The results are not unexpected. James did, as he alluded to himself, things that had never been done (well) on AI prior. He appeals to a segment of the public that in the past, and to a great extent still, eschews AI. So essentially had to survive week-to-week without the advantages that Scotty has, i.e., a large, embedded base of voters who were just looking to vote for the best of the singers of a specific sort (personally, not necessarily musically).
In a way, this mirrors Billboard, where they have a Hot 100 chart, and then a set of genre-specific charts, including the Rock chart: I think it is clear that James is this year's "Rock Idol", i.e., the winner of what could conceptually be considered AI's Rock competition. That does mean, however, that he can't claim the "Pop Idol" crown (i.e., analogous to the Hot 100).
I think Scotty will win, in the end, and that's puzzling and a bit disappointing. While I do expect him to show well on the country charts, his songs will never top Billboard's Hot 100 chart. (And before anyone replies: They don't "need" to.)
I think it is unfortunate how things have gone down, though - this is going to be 2006 all over again, with some musically niche-oriented guy winning based on an embedded voting bloc of folks who are going to vote for a certain type of guy, no matter what. Second place will go to a gal who is a good fit for "runner-up", someone who either exudes sex appeal or girl-next-door charm, and, of course, McPhee provided both. Note that neither of these positions have anything to do with musicality.
That's not to say that any of these people aren't talented. The entire Top 4 are all worthy of winning American Idol. It's just that once we get to Top 4, there is no longer any competition for the votes of the big voting blocs, and there simply aren't enough spots under the big voting bloc's chosen picks for the singers who were getting by on just their talent.
The main difference between this year and 2006 is, as I alluded to above, that Katherine McPhee has essentially been split in half. For that matter, so has Kellie Pickler. The parallel gives me hope: James was eliminated in the "Chris Daughtry slot". He's clearly the Chris Daughtry of this year, with as much potential as Daughtry imho. If he goes on from here to have the same post-AI experience as Daughtry did, then it is all for-the-good, eh?
And if either Lauren or Haley win, I'll be rather surprised. And very happy. Because either of them could have a career that would be much more likely one that I would personally follow, and enjoy the fruits of.