When my kids were that age, one of the moms made a schedule up for summer of parks. Each week was a different park. We would meet Wednesday at 10 am, kids would play, we would have lunch and then each week a different family would bring supplies for the kids to have "craft time." It was nice having the schedule pre-made for the summer.
How about doing science projects? Volcano's, egg heads (grow grass in eggs and draw faces on the eggs so the grass looks like hair)... I had a great book, "Ultimate Book of Kid Concoctions" that I used regularly. There are lots of those type of books at the library (since you're going anyway).
Garage Sales! Give each kid $3 and when it's gone it's gone. My kids LOVED garage sales. You can get lots of stuff for so little money. And then if they only play with it for five minutes, who cares? Then you can resell it at your sale.
Day Camping. We would go to our local state park, set up the tent, hike, fish, etc, cook lunch, yada yada, cook supper, have a fire... then take down the tent and go home. For fishing - we only had one rod, and I knew that wasn't going to work. So I had them find a nice stick on our hike, tied fishing line to it, put hooks on the end and wa la! My daughter caught 8 fish that day and my son 11! It also taught them you don't need expensive equipment. Granted, they only caught blue gill, but that didn't matter. It was exciting to catch a fish.
I made my kids read 20 minutes a day. If your 4 year isn't reading yet, have him look at books for the time. I think it makes a difference in their reading level when they start school again in the fall. I believe it's one of those, use it or lose it skills.
Cook. Again, there are books at the library for cooking with your kids.
Hope you like some of these ideas, and enjoy your summer. This is such a great age! Soon they'll be teenagers.