In the past we never found that much to do on the Boardwalk. It took us all of 30 minutes to walk from one end to the other. There are more performers out on the weekends though. And you can rent surrey bikes which are fun.
DTD is great for LOTS of restaurants, and a wider variety of shopping.
My personal favorite is DTD. I have never been blown away with the Boardwalk. I would not however, take the kids to Pleasure Island.