Our 3YO daughter is allergic to bananas (I know, a tad odd).
As is always the case at Disney dining the Ohana staff was fantastic in working with us on her allergy. Obviously the dessert wasn't a possibility. The chef came out to talk to us about alternatives. It was incredibly sweet when he got down at my daughters level and asked her what kind of sweet foods she likes. She said to the chef, "I Wooove Ice Cweam." The chef asked her if she liked sprinkles. She was a little confused (I guess we'd never given her ice cream with sprinkles). We assured he she'd love them.
When he brought out vanilla ice cream with multi-colored Mickey-shaped sprinkles, her face lit up. She continued to eat every single sprinkle before she ate her ice cream. Now, whenever we go anywhere for Ice Cream, she asks for sprinkles.
Long story to say, I'm pretty sure you can do something different.
That said, the bread pudding and ice cream is my favorite Disney treat!