As you know, London is very spread out. You will not get a hotel that is near to everything. Here are the benefits of staying at County Hall - hopefully someone else will be able to give you an idea of the Tower Bridge location.
County Hall (it was still an office building until about 30 years ago and was where my mother had her first job in the 1940s!) is fairly well placed for a lot of things especially if you don't mind a little bit of walking.
County Hall is close to Waterloo Station giving you access to mainline and underground trains. It is right next to the London Eye (we could see it from our room, but I do know that some rooms don't have a view but look onto an internal courtyard) and the London Dungeon.
It is just across the river from Westminster - an easy walk to Westminster Abbey, Houses of Parliament (Big Ben), a little bit further to Westminster Cathedral (which has a fairly quiet viewing tower with good views of London), and not far from the Churchill War Rooms.
You can walk from Westminster to Trafalgar Square -past the end of Downing Street (prime minister's residence), past Horse Guards (where there are usually mounted soldiers on guard duty), you will see the Cenotaph (war memorial) and the Banqueting House (where Charles I was beheaded). There is a free Rick Steves audio guide for this walk.
https://www.ricksteves.com/watch-read-listen/audio/audio-tours/london (Westminster Walk).
At Trafalgar Square is Nelson's Column and the National Gallery. Then you are near to Leicester Square (cinemas), Piccadilly Circus (Eros) the Strand (some theatres) and Covent Garden (shops, cafes, street entertainment)
A little bit further, but still walkable, you can go through St James' Park to Buckingham Palace if you are lucky you may spot the pelicans on the way (there's a children's playground in St James' Park which may be useful depending on the age of your child.)
Across from the hotel is Westminster Pier which is where a lot of the river cruises go from, which give you great views of London. These will take you to the Tower of London, Greenwich and sometimes a bit further to the London Barrier (Flood defences).
There is a hop on/off bus stop just behind the hotel, if you do this I strongly suggest not doing the "hopping" but stay on for the whole trip as the traffic can be really bad so you can waste a lot of time waiting for the next bus.