IMHO I would think that with a really "tight plan" you could do it in about 8 days and see EVERYTHING. With a 3 year old and two older children you may be slowed down a bit because of meeting up for dinners and trying to be at the "kiddie" things and also experience some of the fun with your older children. With Grandma along (sorry Grandma) you may also be slowed down depending on Grandma's tolerance for running around a theme park. In order to fully enjoy the experience, I would really recommend 10-14 days, probably leaning more toward the 12-14 day trip. If this is your first trip you probably are imagining Disney World alot smaller than it really is. It is HUGE! So much to do, don't want to miss anything from the biggest thrill rides to the tiniest "streetmosphere" at MGM or MK. You want time to have your little one ride Dumbo twice if he wants and have your older boys be able to ride Rock N Roller Coster more than once too. Although you mentioned not wanting to "waste time" at the hotel, I believe you will need it! I regret not spending more time at my hotel the first few trips. We used to just run and run and never make use of the pool at the hotel. Finally after being there a few times we took time to sit back and enjoy a bit and WoW what a difference a day of rest makes in the middle of a trip. Also if you go for a longer trip you will have time to do the Water Parks, which I'm sure your older kids will just LOVE. They have a section for your little guy too! So cute! Grandma will love the lazy river! (My mom and I spend all our time floating around in it as my Husband and Daughter go off on the "fun" stuff). You'll have time to play miniature golf if you wish, do Disney Quest for the Boys and have time to shop and just sit on a bench and enjoy the atmosphere. Make sure to do some table service meals if you can as it cuts the stress( for me anyhow) of going up and trying to get our own food and not knowing where we will eat. Plan ahead for those.
Most of all have a plan in hand, read these boards, read The Unofficial Guide, and anything else you can get your hands on. Staying on site makes all the difference. If I were you I would book either two rooms at one of the Value resorts or one of the new family suites at the All star. Two bathrooms will make all the difference. Have a wonderful trip. ENJOY!