I have stayed at both, and while the Hilton is more of a true deluxe resort, I much prefer AKL. Will you have a car or will you be relying on hotel transportation? Car would make things easier, but you have to pay for parking.
The bus situation at the Hilton was the deal breaker for me. It was a major hassle. I was there about a year and a half ago, so things may have changed since then, but when I was there, you had to make a reservation for the bus time that you wanted. That meant standing in line at the concierge desk. The early times fill up fast.
Loading the bus took a while because strollers had to go underneath in the luggage hold. The bus picks up at the Waldorf Astoria after leaving the Hilton, so it took some time to get to the park. I never made it to rope drop at MK because after a long bus experience, you get dropped off at the TTC, so you still have to take the ferry or the monorail to get to MK front gate. Travel time was anywhere from 1 to 1.5 hours for me.
If you were planning on spending a lot of time at the hotel and not rope dropping parks, I would say go for the Hilton for sure. Since you say you aren't going to be spending a lot of time at the hotel, I would look for something else.
Have you considered the Swan or Dolphin? They give you more of a deluxe experience without the Disney overload and are usually cheaper than Disney Deluxe resorts. Another option might be the new Gran Destino Tower at Coronado Springs. Reviews are overwhelmingly positive and is really more a deluxe rather than moderate experience.
Hope you have a great trip no matter what you choose!