Hi Jenny and Welcome to the DIS!!!

Wow! That would be fun - but so frustrating - so close, but so far away.

Of course I would always spend some time at Downtown Disney.
And what about "Resort Hopping?" - checking out all of the different resorts? (And their Restaurants?

)
Don't know this would be in your budget, but how about breakfast or dinner at Chef Mickey's for a real dose of WDW without going into the parks?
Also, there is Pleasure Island BEFORE 7 pm - no charge.
Just reread your post and see that your DH will be at the Swan and Dolphin for his conference. That reminds me of something else to do. You can go to the Swan And Dolphin, and then go and "do the Boardwalk." You could take a stroll by the Yacht and Beach Club and have dinner at Beaches and Cream. And then there MUST be a place to check see the fireworks from Illuminations at night. I would ask.
ALSO, you could go to the Grand Floridian (there is a Pooh Character Breakfast, lunch and dinner there) in the evening and catch the Electric Boat Parade that passes every day.
I hope that you have a great time!
