All of this has been covered in briefings by Ron Dittermore or other NASA reps:
The Astronauts were not equiped to do a space walk under the belly of the shuttle -- they never are. Walking under the belly of the shuttle has the potential of doing more damage to the shuttle than already is there, and has a greater risk of losing the astronaut than attempting re-entry. Unfortunately, in this case, that risk turned out to be a loss.

But, even if they had done the walk and found the damage, there wasn't anything they could do. There isn't a "tile fix-it kit" in existance -- tiles are examined and repaired after every mission individually. They couldn't have stayed in space to await a repair or rescue team because there wasn't enough time. They would have died due to lack of oxygen and we still would have lost them and the shuttle. They couldn't have gone to the Space Station because they were in a different orbit, not equiped to dock with the Station, and putting 10 people on 3 peoples' worth of supplies, they'd use them up very, very fast. They really only had one option, and that was to try coming home. If luck was with them, they would have made it. Unfortunately, it wasn't this time.
The arm wasn't included this time due to weight issues. It's rarely included on research missions because the point of the mission is research, not arm use. There's only so much weight that can be lifted into space, at least at a certain cost. And the arm is rather heavy.
The problem with an escape pod is the only way it could be made would be to totally re-design the front of the shuttle and put more stresses on an already-stressed structure. There's a greater risk of something that is designed to break away during an emergency instead breaking away when it's not supposed to than there is risk of having this happen again. Also, retro-fitting the existing shuttles to include the escape pod would greatly increase their weight, again limiting what can be taken up, not to mention make them less safe because you're jurry-rigging something to an existing vehicle that wasn't designed for the jurry-rigging.
It's a catch 22 in a risky business. But, even that being said, there have only been 2 major disasters (at least that I know of) in 22 years of performing this risky business (specifically the Space Shuttle). Considering all the risks involved, that's a durn good record.