I've seen all the movies - and you can probably follow along if you at least have an idea what the characters are.
While it's a good movie, I am not sure I would even put it in the top half of Marvel movies...The last 3 were all better IMO. (Spiderman, Thor Ragnorak, Black Panther) I mean don't get me wrong it was an impressive feat to wrangle all those characters into a coherent story that's for sure, but certain ones, in particular Winter Soldier and Civil War I think do the character beats so much better - this movie has so much to show that it doesn't really have much time for characters, allowing the other movies to do that heavy lifting. I think other than Thor, Gamora, and maybe Tony there's so little character development, I can't imagine if you are not a fan of the movies that watched many of them, that this movie would just seem really silly. It would be sort of like watching the last Harry Potter movie without watching any of the earlier one and just be like "who are these people?"