When it comes down to it, you don't need any license, filing, business listing, separate tax ID, certification, or anything. You can do freelanced, self-employed photography, keep track of any income made from photography both with records and saving receipts, then when tax time comes, simply file a schedule C alongside your normal 1040 for any income earned through photography.
Only when you intend to get a bit more 'advanced' about it - deducting camera equipment and travel expenses, setting up a studio or office, etc would you want to go through the business filing or licensing option...mostly when you are deciding photography is going to be your career choice.
But if by setting up a business, you mean to occasionally make some side income from selling photos, or occasionally getting paid to shoot someone, it can be done as a fully self-employed pursuit, and you can label yourself on business cards or in ads as 'your name, Photographer'. Just be sure to file that schedule C.
As for how to go about starting? Lots of self-promotion, lots of photo taking, plenty of initially underpricing your services to gain reputation, not judging any job as beneath you, a good dollup of luck, plenty more self promotion, and then plenty more luck!