If you have kids and don't mind letting them explore a little on their own, they'll almost certainly have an amazing time on the island. If you don't have kids but love the Pirates of the Caribbean films, or want a nice place to eat lunch or relax away from the crowds, I'd figure to visit in the afternoon when crowds are heavy anyway. (If you have kids and they love the Pirates of the Caribbean films then you MUST GO.)
If none of those things apply/appeal to you, then you can probably miss it. In fact, if you have kids and don't want to let them explore the island on their own and don't want to explore with them, either, you probably shouldn't go.
I was always one of those people who liked to eat at
Disneyland Resort instead of bringing food in, because they have some delicious food options that I can't get at home so I thought it was special. But my most memorable, maybe my favorite meal, was sitting at a table on Tom Sawyer Island with my friend and her mom (who were going on the cheap and had brought their lunch but shared their chips and cherries with me). We were away from the crowds, had no wait to get our food, and a woman who took care of the island stopped by and chatted with us about the baby ducks.
But on a 2-day visit, it's a judgment call, unless you want to give your kids some independent playtime or love the Pirates of the Caribbean films. (There are enough references to the film to make it something a fan should probably visit at least once.)