it's just in the sauce which is served on the side. you could pick them out, they are not many and what is in there is in slices, or skip the sauce or ask for the sauce without the bananas... i'm not sure if there are cross contamination issues if there is an allergy.
Disney is always accommodating! If you tell your waitress or call ahead, they will work something out for you. Also, I saw the words Ohana bread pudding and instantly got hungry! Haha