We are lucky in that my ds is 7, so he is in school most of the day. I work flexible hours - I'm out the door before he even gets up. This has actually been very nice for dh and ds, getting up is a special time just for them. I get home about an hour after ds's school day ends. This also works out great. His school has an aferschool program that is staffed by teachers. He has time to get help with any homework, have a snack, and play for awhile with friends. As an only child, this gives him an opportunity to be with other kids of different ages he would not have otherwise. I pick him up, and then we have our special time before my dh gets home from work.
On Thursdays and Fridays, I work 5 hour days. This allows me to pick ds up when school is over, so he can participate in activities - he's done gymnastics, acting and singing, etc.
During the summer, ds attends a summer day camp that wants children dropped by 9 and not picked up until after 4. This is perfect for us. We continue on our same schedule, and I am able to pick him up when he's done.
What works and is comfortable for every family is different. You may have to experiment some before you find the situation that is most right for you. The most important thing is for you to feel comfortable, and be happy.
If you do decide to return to work, try to find something that you value, and are happy doing. Try to get your kids excited about, rather than resentful, that you are working. My ds enjoys school, so it's easy for him to understand that I enjoy my work as well.
Balancing work and family can be a chore or a challenge. Attitudes make a big difference!