I have heard this too but really, have you ever seen anyones cart tip over from having an infant carseat on it? And like a PP said, I have a hard time believing that a 10 pound infant can tip a cart, but a 6 year old cannot.
Yes!! Well, okay, the cart didn't tip, but the infant seat flew out of the grocery cart and onto the pavement in the parking lot. Not my kid, but when I was pregnant with my 3rd this summer I was grocery shopping and was talking to a mom and little baby (she was 5 months old) who were in front of me in line. She was wheeling her cart, with the baby in the carseat on top, through the parking lot when I came out and there was a small bump in the road and the seat literally flew out of the cart and landed face down.

That poor little baby and her mom were screaming hysterically. Her face was scratched and bleeding, but I don't know if she had any significant injuries as I left when the paramedics came. Either way, it reminded me of why I never felt comfortable placing my kids on top of the cart in their carseats. I always put them in the basket and do now with my 3rd as well. No, I can't fit as much as if the carseat weren't there, obviously, but I manage just fine.
Alternatively, if I need to buy a lot of stuff, I put her in a baby carrier and wear her through the store so I have the whole cart.
So anyway, I don't know if it is a danger of the cart tipping or the carseat coming out like that. There was also a similar story in Parents magazine in the fall, I think, where someone bumped the cart in the store and the carseat fell out or something like that.
But back to the OP, I would definitely get an infant seat. My first HATED her infant seat so we never used it as a carrier, but she was born in February so it was way nicer to get her in the seat in the house, cover her in blankets, then head to the car, rather than carry her bare little body in a blanket, or try to stuff her limbs into a snowsuit/coat.
We did buy a Britax convertible seat when she was a young infant (Marathon), but like others said, she simply didn't fit properly in that the strap slots were too high for her. She was 75th percentile in height from birth till now (she's 6yo) but still, she was too short for the lowest slots. She outgrew the infant seat height-wise around 10 months. She hit 26 inches (the max of the seat) by 6 months old, but her head was still way more than an inch below the seat so she was fine in it.
My 2nd one liked the infant seat (same one my 1st hated) so we did use his out of the car if we were shopping, in a restaurant, etc. He outgrew the seat around 7 months in height, and was close to the weight limit then as well. That seat went to 20lbs and 26in.
With my 3rd, we had to buy a new infant seat as the old one expired a month before she was born. We got the Britax Chaperone, which was very pricey, but I liked the side impact head protection and the anti-rebound bar. As our 3rd kid, the infant seat is invaulable as we are in and out of the car a lot and she would be constantly disturbed if I had to take her in and out. She is not a fan of being buckled so this would result in much crying.
Like someone else said, the seat holds much larger babies than the seats made a few years ago. Ours goes up to 30lbs and 32in. Neither of my kids exceeded that size at a year so I assume that our newest won't either.
But since I know she is my last baby, I didn't want to buy a 2nd base for my husband's car. We tried using one of our Britax Marathons for her, but she was way too tiny (8.1lbs and 20.7in at birth, so decent size). So we just belt the infant carrier in with the seatbelt when we use my husband's car.
Also, this one was born in January, so we bought a "Cozy Cover" and that eliminated the need for a coat. We did put a blanket under the cover when it was frigid, but otherwise the cover will do. Plus no worries about her getting wet when it rains cause I can put the cover on.
Regarding the angle being incorrect when not in the base (or properly installed without the base), that is true, but it is such that the baby is more reclined than should be, not more upright. So their head can't flop forward onto their necks, it just lays straight back. Not sure that is any better really, but they can't asphyxiate in the way they can if their head flops onto their chest.