that's a tough one... I personally would not bring a 3 year old there. I am bringing my 3 kids when we go but they are older. My youngest is 6 and loves fine dining and I know I can depend on her to be a complete lady while there.
Your the only one that knows your kids and really the only one that can answer this question. Just remember all of the other people there are paying a high price for fine dining and it isn't a character meal. In that I mean the place isn't geared for kids. Yes, they do serve kids but its not what I would call kid nation
Are you going to be at the parks all day before dining? Is so take your son at his worst behaved day and ask yourself if he could manage a fine dining establishment at his worst. He will be tired, cranky, overstimulated and hopped up on Disney magic.
Depending on where you are staying, it might be best to use that money for a nice room service dinner. That way your son can pass out and you and DH can relax on the balcony of your resort and have dessert/coffee/wine while he is sleeping.
Please don't take what I say the wrong way ... its just that I don't think a 3 year old would have a whole lot of "fun" at this place.