We spend a week in Cape May every summer. There is no other Jersey shore town like it... It is very Victorian with lots of greenery (trees, gardens, etc). The beaches are beautiful, very clean and you can watch dolphins from the beach nearly every morning!
There are fun, but low key, things to do. You can collect Cape May diamonds at Cape May Point, go to the kids' shows at Convention Hall (I think that is what it's called), have a tea at the Emlick Physick estate, shop at the town square, rent surreys and/or bikes...the list goes on and on... There is a small boardwalk, but not much on it. It is great for a morning bike ride or jog!
One night during our week, we travel to Ocean City to play on the boardwalk. Our kids really look forward to this! Ocean City is really geared toward the family. Interestingly, it is a "dry" town. There are no bars, liquor stores, etc. So it has the boardwalk energy but it quite tame.
It is important to note that we went to WDW last August because it was cheaper than a week at the Jersey shore. (We had the
free dining plan and stayed at POR.) If you are interested in staying close to the beach, Cape May and OC can be pricey. (So can Avalon and Stone Harbor...) It is fun, to us. Most of our friends from other countries and other states do not understand why people in the tri-state area love the Jersey shore so much. I think it is just part of the culture that we all grew up with in the Phila area!
Good luck with your planning. Be sure to download maps of the area to make sure you are renting in a convenient location!
I forgot to add...many of the Jersey shore beaches are shrinking. I was really surprised at some of the "skinny" beaches at OC and Avalon last year. Wildwood's beaches seem to grow as the others lose sand. So that is a plus for Wildwood. We don't do Wildwood's boardwalk because it is a bit too wild for us... Lots of teenagers, a few too many fights... public drunkeness, etc. BUT, we haven't tried WW for a few years so that may have changed--I know they have rides/attractions that teens love...