Mike_M, what was your original reason for being there? It sounds like there's a big personality difference going on, b/c I, for one,
love just being alone with my thoughts. I enjoy watching my son have fun, but I also don't enjoy worrying that others are having a bad time while following my plans. (in September nearly had a panic attack walking to DCA b/c I wasn't sure people were having a good time)
I had a good long time of solo travel, so I know what I'm missing when I travel with others. I also know what I'm gaining when with family, don't get me wrong!
OP, from doing the math, it sounds like you are the age now I was when I had my kiddo, and I was married only the year before that...different life experiences! Because of my single years (I was days away from 31 before I even met the fellow destined to become my hubby), I know how much I love solo travel (I have been to New Orleans twice, and each time I was alone. AWESOME).
But you don't know if you would like it, and you want to know! So I think you should go! I don't know how old your younger two kids are, but my guy was over 3.5 when I went (lol, he still is, of course) and even though he's an extended nursing kiddo, he was fine with his dad.
Because I was alone for so long and wishing for children for so long, I scoffed at the people who urged that I leave my infant with others so I could get time to myself. I devoted myself 24/7 (OK so I did go shopping, I went to a baby shower, I wasn't a total hermit, LOL) to him. And then after our second trip of 2007, when I was making my family nutty b/c our touring styles don't mesh, I finally reached my limit and really felt the need to go alone.
Are you free in late April? Perhaps joining the Ladies Only trip being organized here would be good for you?

I planned on going on that trip, but ultimately my January trip (under 48 hours long) ruined that plan b/c we kept having unique, unplanned, expenses that kept us from saving up for the April trip. sniffle sniffle. But it might be good, for a first trip away, to meet up with others!
I'm telling all of this sort of backwards.
Our first trips to DLR as adults were done as parts of a bigger visit to my brother. We had only 1 day for each of those visits. When I was a kid, the trips were all about the whole family. When I was with my mom, it was about my brother, and mom's migraine (dang teacups), my odd stepdad...when I went with my dad, well, everywhere my dad took us it was all about him! So now these adult trips, that was being dictated by others, too! Making them happy, trying to maximize their (and my) fun.
We went on our first BIG trip there in September that also involved my 20 year HS reunion and visiting my family. We spent 6 nights at DLR and had so much fun! Decided we just had to visit again in December. DS and I went ahead and had two days in the parks before DH arrived late at night. We spent another 3 nights, and then DH went back to work and my son and I went to San Diego to visit my brother. Spent way too long there, away from my husband, and it nearly drove my poor son over the edge. Even my childfree brother, who has always disliked children below his age, even when he was a child (though he actually does quite like my guy, otherwise we would not be invited there all the time LOL), understood why DS was acting the way he was acting, while I was mystified...
I was trying to have fun, why wasn't my son?
Well, after that experience, I had even MORE of a desire to get away alone. The 9 days at my brother's (told you it was too long!) was SO hard on me, and I was just beyond exhausted. Hubby took great pity on me, and suggested that I take a quick trip by myself, if I wanted to.
That way I could tour the way I want to tour, without worrying about anyone else (DH is smart and knows that if there's another person with me, that I worry more about their good time than mine, and then later I get sad that I didn't enjoy myself more). I could get up early and stay late. I could sit and have a coffee without worrying about, well, anything! Eat where I wanted, go on whatever ride I wanted, wander if I wanted, sit down if I wanted, watch the people get soaked on Grizzly for an HOUR if I wanted!
And that trip, although hard at times, was really very fun.
**********************
The things I worried about...
DH and I, in our early days, vowed to Never Fly Without The Other, and here I was, ignoring that. Well, actually, it had already been done, b/c of the December trip.
Since DS is an extended nurser, I worried about his happiness at bedtime and my body's happiness as well!

I did NOT want to experience mastitis.
DS and DH having fun. Ah, not to worry, they had a blast! DS has been more bonded to DH than to me since before he arrived...DH could tell how big the baby was and what baby's mood was in utero, DH was amazing like that (he can sense energy fields, which is really wild but very cool...and helpful, too), and then in the early days I had had a very hard time and really coudln't move, so in those days ALL I was good for was milk, and so the two of them bonded tighter than just about any father/son duo I can imagine. With the exception of some movies DH has shown DS, I can always trust my hubby to be safe and have fun with our son.
I experienced some turbulence on the flight down, and major turbulence on the way home. I did NOT like that.
**********
When I saw happy families, or even unhappy families, I got a sadness, a melancholy. I would feel the feeling, experience it, and let it pass. I had *just* been there a month before with my family, experiencing those same joys or those same annoyances, and I knew we would be there again! So the feeling would subside and I would move along to the next thing that I wanted to do.
Did my guy know where I was? He was told, and he did NOT care. Because of my go-go-go mentality in December, going back to DLR was the LAST thing on hubby or son's minds. They did NOT want to go again. I talked to my son about it when the plans were in their very early stages, he said he didn't want to go, he said he would be fine if I went, he would have fun with his papa, and that was it. Later he called it my "party", so there's a possibility that he forgot in those intervening weeks, but I don't really know.
Now that I have done my touring style, and have done it well I believe, the plan is that I can relax and be a nice, normal human at DLR in May for Eamon's 4th birthday. Since hubby knows that's the plan, I'm sure he'll comment if I start being tweaky again. I also think that they will have no problems letting me go off to run around again, or to just sit on a bench in NOS, if I need to.
I felt no guilt for being there without him/them, and I continue to feel no guilt. The momentary "awww" moments of seeing other families was gotten past, especially since I knew they didn't want to be there!
If DS were older and I needed to go again, I'd make sure that a solo trip was planned around the same time as a family trip (for me, I'd go before the family trip)
OK I gotta go, I've been working on this reply WAY too long. My kiddo took an extra-long, un-expected, nap today (he's been sick and now I'm sick so we're having a rough time), and it's caused a very very late night for him. Now DH is getting irked at him, and that irks me (DH has been at work all day, away from his beloved boy! his threshold for annoyance shouldn't be as high as mine is by the end of the day! argh!) so I gotta help them out.