There is another side to Asbury Park, but I am not sure I would bring a family to stay there...... I live close to Asbury, and I could say many things. It has been a struggle for this town, and many of the problems there are not the town's "fault". The state has caused problems for AP, and of course the problems with the economy has hurt them more than most other towns.
That said, it IS slowly making a come back. Has many new clubs and restuarants, and new condos are going up. The beach itself, is nice and wide, and attractive. Younger people are moving in, many with $$, and they are renovating buildings and opening new businesses. There is a small gay community there, and they are helping the town out as well, a very positive influence, IMHO.
Nearby, you would have more for the kids to do at Pt Pleasant, Manasquan, Belmar, Avon-by-the-Sea, or further south, Long Beach Island......