I own all of the HP lego sets from the 2010-2011 releases, and they are all very nice.
I really think the Burrow (4840) is a unique set. It has a good collection of mini-figures and the structure is a nice interpretation of the Burrow in the movie/books. It was fun to build. MSRP is $60. They have it online at Barnes & Noble.
If you want bang for your buck, the Quidditch set (4737) is nice because you get a lot of mini-figs for a low cost. It isn't much of a building set, though, since it has fewer bricks in it. It would be more geared towards someone who wants to collect the mini figures, or display a nice looking set, for a lower price point. MSRP is $20, and this one is on shop.lego,
amazon, etc.
Like a PP mentioned, Diagon Alley (10217) is a very nice set if you are looking to get something with a lot of bricks to build with at a higher price point. The buildings are very cool and it comes with a lot of neat bricks, plus a bunch of mini-figs. This is a Lego store exclusive, MSRP $150.
The Forbidden Forest (4865) is another good one for a starter set because it's cheap and comes with a good number of mini-figs. I love the little box sets because of this reason -- they are easy to collect. Not really much of a brick building set, though. MSRP $12 but it's on amazon right now for under $11.
I love Lego vehicles, so I also like the Knight Bus (4866) and the Hogwarts Express (4866). Both of these are re-releases of sets they came out with in prior years. I have the older Hogwarts Express and the new one, and they are very similar. The Knight Bus is great because it's a unique vehicle to begin with -- triple-decker bus :-D Very cool. The Hogwarts Express MRSP is $80, but it looks like it might be easier to find at a brick&mortar store right now -- seems out of stock online right now except for Toys R Us (where it's price is inflated). MRSP on the Knight Bus is $35 and they have it at shop.lego.
I saw the smaller Hogwarts Castle Expansion set the PP mentioned at Costco for $25 in my local area. That's essentially half price, so that was a steal. I would suggest checking your local Costco or equivalent membership stores if you can, just in case they have any left in stock.
Toys R Us occasionally runs BOGO 1/2 off sales on legos, but it's only worth while if you buy sets that are about equally valued. Also, Toys R Us inflates the price of Legos quite a bit higher than MSRP, so unless they have a set you want that you can't get anywhere else, I don't ever buy from them unless they have their sale.
I always check Amazon. When they aren't low on stock or in super high demand, they often have things cheaper than MSRP and it's tax free for most states. Just be careful because they have 3rd party sellers on there who will sometimes have crazy inflated prices, and who also sell the older sets that are no longer made (and therefor they charge a fortune for them)
If I am looking to get a set and not worried about waiting for a sale, I always default to the Lego store. You can get a membership card and earn some rewards. This is convenient for me, though, because I have one locally near me. They have a nice website you can order online, though.
I also check Walmart, Target, and KMart for Legos -- all of those stores run sales and clearances on Legos occasionally.
If you like the hunt, you can also comb craigslist and ebay for loose lego bricks and minifigs, used sets, etc.