This is nothing but my crackpot, untested theory at this point ... but I feel like the voters almost react like they've been "betrayed" by a team full of unknowns this early in the season, haha. If we were ranked #19 and dropped a road game in OT in mid-February, I think we would only drop a couple of spots. However, I bet some voters feel like they took some sort of chance on us after beating Arkansas, and then they have a knee-jerk "Fool me once!!" reaction to us losing and shift their support to some other unknown team instead (e.g., Michigan).
Either way, if we want a shiny ranking ... the opportunities are right there in front of us. I think if we simply win one this week, we will be ranked again, depending on what other ranked teams lose this week. If we win two, though ... we will skyrocket up the polls. LET'S GO DO IT!
EDIT: I will say this, though ... even understanding how the AP Poll works, it is still pretty wild to have Arkansas that far ahead us at this point, haha. Like, we had an OT road loss. Arkansas has done nothing in that timeframe to counteract us manhandling them on a neutral floor.