No, I don't feel guilty when the kids have to fend for themselves for entertainment for a little while. I don't have the time, energy, or inclination to organize every aspect of their day. And having some "free play" time stimulates their creativity and independence.
I'm around to oversee my kids, of course, and make certain that they are safe. And I'll read a book or two or play a game, if that's requested. And I'll give suggestions of things the kids can do when they get tired with their current activity. But, in general, the kids spend a lot of time choosing their own activities -- usually DS4 chooses and DD2 wants to do whatever big brother is doing.