The underground river really isn't exactly underground - it's not claustrophobic at all, and there are many places where you can get out if you want and walk. ...
There are two routes that you can take - one is much more open than the other........we did the more open one and I agree, it wasn't clautrophobic at all.....the passages going through caverns were fairly short, and had some really neat roof-lights in places...our DD6 enjoyed it.
Yes, the water is cold...but its certainly not freezing, and its perfectly fine once you are in it.