i would look very closely into the zoning restrictions for the area to ensure that the type of home you want to build is even permitted. some areas also have restrictions that prohibit "temporary structures" which include modular homes, trailers and the like. also check and see what the future plans for the area are-sometimes the city manager's office has good information-what may seem like a relaxed, sleepy little place now could be earmarked for massive construction within a few years (we had neighbors who were livid when a strip mall was built on land behind thier homes, had they taken the time-like those of who bought new-to check the city growth plan they would have known that the entire neighborhood we lived in was mixed zoning and that strip mall had been in the works for several years).
also check to see if there are any "endangered" or "protected" wildlife prone to the area-several news stories have been on around here wherein folks bought hugely expensive land not realizing they were prone to certain critters being looked at for protection. when it came time to build the environmental impact reports prevented construction all together.
one way to keep construction costs down (at least it's popular around here) is to buy land with even the most falling down building on it. some places charge far less for building permits and have much more lenient laws re what can be built if it's going to be a "re-model" vs. new construction. a town near us is famous for this, and people buy lots that have a fallen down shack so they can use one wall, reinforce the studds, and justify their entire construction as a re-model.