I've been to NYC 4 times, although never with kids.
FAO Schwarz is a lovely place to while away a few hours - although it can get very crowded. Arrive just before opening time to see the opening ceremony. The staff then line the red carpet entrance and clap as you go in - makes you feel like a celebrity!!
Tried to go and visit the real Winnie the Pooh at the New York Public Library but it was closed when we went (Oct '08) so check before you go. The Public Library is very grand and worth a wander around anyway. Free entry.
Central Park is just over the road from FAO Shwarz so that would make a nice respite from all the noise and crowds of the street. Try the Zoo there.
I'd be interested to know how you actually get on in Manhattan with a 3 year old. Mine are 8 & 10 and I haven't yet been brave enough to book a NYC trip as I don't think my younger one could handle the pace! Even if we did use the subway a lot! It is the most amazing, exciting and vibrant city you won't want to miss a thing!