If you're shooting something stationary (like the castle or some other building), there's simply no way around
some sort of support for the camera - something that takes the camera out of your hands.
A nice solid tripod is the best all-around solution, photographically-speaking, but obviously can be a pain to carry around.
There are smaller options like the GorillaPod, ClamperPod, bean bags, pocket tripods, etc, as well. These generally work OK with smaller cameras (though you can often make them work with bigger ones if you're careful.)
Once you have that, you have to make sure that the camera doesn't shake when you start taking the photo. If you have a DSLR, a remote shutter release is the best option. If you have a point-n-shoot or a DSLR w/o remote shutter release, use the two-second delay option to give it time to "steady" after you press the shutter. If you're on something really wobbly, you may even need to go to the 10-second shutter.
Finally, as for the camera itself, you want to keep the ISO low and the aperture at the sharpest setting. If your camera supports Av (Aperture Priority) mode, use that and set it somewhere around F8-F10. Set the ISO to something low, generally you want the lowest available. That could be anywhere from 50 to 200 depending on your camera.
Then, just shoot - you'll get great night photos as if by magic.
You may also want to play with white balance settings - many times, the camera will give you a brownish sky, setting the white balance to Tungsten can help correct this.