We didn't feel the least bit rushed on our 3-day last May. The pace of everything was calm and relaxing. However, having said that, we definitely will do longer cruises in the future. It's just so much fun we want it to last longer. (At the time 3 days fit our budget, schedule and DH's Disney apprehension. Since then, he's a total convert.)
Regarding your trip, I'd suggest going to WDW first, then cruise. IMO the park portion of a land/sea adventure is not to be missed. But it's full of rushing, crowds, queues, etc. Get that over first, then spend your last 3 days full of R&R at sea.
Kelley