Your question is almost impossible to answer without some kind of idea of what food you like, what kind of surroundings, and what you mean by "budget". Move away a block or two from the Times Square area, and you can get very good food for ten dollars or less. And as far as I know, there is no where else in the world where in the space of just a block or two you get excellent restaurants featuring Ethiopian, Argentinian, Vietnamese and Nepali food.
Start by picking up the most recent copy of Zagat's "New York Restaurants". It lists restaurants by cost, area, cuisine and so on, and is a very, very good basic guide.
From there, go on to the New York menus website, which gives you listings by cost, area and cuisine, but also user reviews. Be warned, however, that some of these reviews definitely must be taken with a few grains of salt. And the down side of the website is that there are just so many restaurants to go through.
One thing neither of the above two primary sources list is the food carts, which are the one of the better kept secrets when it comes to dining in New York. Yes, yes, I know - you associate food carts with overcooked franks, greasy things-that-look-kinda-like-meat-on-a-stick, and those tourist-priced pretzels.
Suspend your disbelief and take a look at
http://midtownlunch.com/ which is a website that provides a great overview of food carts that serve not only decent, but in many cases quite good food for ten dollars or less. The man who runs the website was profiled in the New York Times just a few days ago:
http://www.nytimes.com/2009/06/03/dining/03lunch.html?ref=dining
And the good thing about these food carts is that they are perfect for tourists on the go - good food, cheap, and you can eat on the go.
It's hard to make specific suggestions when it comes to individual restaurants. I think everyone in New York has their favorites, which they swear beats the competition hands down. (And yet other people have visited those same places, got stuck with a server or a cook who had had a bad day, and swear they would never go back again.) You might start at Restaurant Row on 46th Street -
http://en.wikipedia.org/wiki/Restaurant_Row_(New_York_City) . If the prices on the menus look too high (I think they're OK!) continue on to Ninth or Tenth Avenue; the eateries there are rather cheap, and the chances are the quality is still good. But like I said, stay away from Times Square and anything that looks tourist-oriented.
My own favorites at the moment tend to be located down in Greenwich Village, or over in Little Italy or Chinatown. Chinatown in particular has wonderful food at bargain prices!
But you might also splurge at least once. And when DW and I want to splurge, we go to Bappo (
http://www.babbonyc.com/mariob2.html ). Their tasting menu in particular is the best we have ever had, and we've eaten our way through France and Italy. Bappo is among the top Italian restaurants in the U.S., and deservedly. Reservations are reputedly hard to get, but I chanced to find out what the secret is. You simply walk in during the afternoon, before they open at 5 pm, and ask for a table - if not that same day - then the next, at 5 pm. We have never had a problem, and we have always had absolutely wonderful food.
