These "Which one should I get?" threads are kind of funny cause naturally people gravitate toward the one they know best which is generally the one they use themselves!
First, I think either is a good choice.
Second, I'll divulge I'm an Olympus user. I use Olympus dSLRs and have lots of large, heavy lenses, and I also have an Olympus PEN Mirrorless which has smaller micro 4:3 lenses. I think it
is an advantage to have auto focus with lenses when you use an adaptor on your mirrorless system. With that said, I bought an Olympus adapter but I have yet to use it! Turns out, I really like the micro 4:3 lenses. They just seem to go better with the smaller camera. This wasn't something I anticipated before I started using it. I figured I'd want to use all the big guns but honestly, I haven't had a need as of yet. (And I just got the two kit lenses, not the better quality ones.) I'm sure I will someday and I certainly like having the option, but I use that camera mainly for fun or when I'm traveling really light, so keeping it simple is working out well. The 14-42 kit lens is retractable and makes the system fairly small when it's on. Nice feature.
This statement of yours is telling:
If I were to get any of these cameras and use any of my existing lenses, I would only use the small ones since it would defeat my purpose of trying the mirrorless system.
I think you're wise to think this way, based on my own experience. Other people may be different but you know yourself best.
Regardless of which camera you buy, you know you'll be using the smaller lenses. Check. With your smaller Sony lenses, I guess you'll have auto focus with an adaptor on the NEX. (Don't know all the ins and outs of the NEX and Sony lenses, but I think so.)
With an adaptor on an Olympus you can use those lenses also (I have read that you can use
any lens on a PEN or other m4:3 with an adaptor), but they will not auto focus. Only 4:3 lenses with an Olympus adaptor will autofocus.
If you are considering the OMD, however, you would want to get the better m4:3 lens with it anyway. It is in a different class than the PEN. There are becoming a lot of m4:3 lenses to choose from, and they are nice glass. Supposedly the sensor in the OMD is a Sony sensor, too

and doesn't have as many of the issues with noise at higher ISOs that previous sensors had. (Personally, I try to shoot without using higher ISOs so it hasn't been a huge issue for me anyway.)
I've read a lot of reviews of the OMD from people using it and they are leaving their dSLRs at home and not worrying about it. Amazingly, too, these are not just Olympus loyalists but users of all brands as well. I'm not giving up my dSLR but I would have zero hesitation about getting an OMD. I probably will eventually when I have some extra funds.
Like we always tell people here, whatever you buy has to feel comfortable to you in your hands, and you should find it pleasing to the eye, etc. I don't know if there are OMDs around to play with, but if you can find one you should check it out. With either one, if your not happy, you can always exchange it. Some people seem really happy with their mirrorless systems, and others don't like them, or prefer the simplicity of a point and shoot. Everyone's different. All you can do is give it a whirl.
Good luck!