Legally speaking, these people are not the child's grandparents. They are her legal great-uncle and great-aunt. Within YOUR family, however, you can consider them to be grandparents, aunt/uncle, or whatever, but legally they can't sue for grandparents' rights since they aren't grandparents.
If they had sued closer to the time of adoption, there is a chance they might have had a chance at getting visitation ordered through the Court, but the chances are low this many years afterwards.
Even if they had standing to sue, which they don't, another thing going against them winning a court case is that you and your husband, as the child's parents, have allowed them access to the child. You have not kept them from the child, therefore they don't need to sue to see the child. They have the opportunity to see the child at family functions, however THEY choose not to take advantage of those opportunities. As her parents, you and your family have the right to make your own plans, such as sports, family outings, etc, without having to change plans to meet their demands/wishes.
Why do they want alone time? Are they taking your child to see her biological parent(s)? Are you OK with that? Are they telling your child things, or making promises to her, such as a warped view of reality for what led to the adoption? Are they trying to get her to chose them over you, and trying to undermine your authority or parental decisions?
Whatever you do, if they sue or threaten to sue, don't give in and agree to any sort of legal agreement. Stand your ground. If you given in and they get legal visitation, that could affect your family in a big way. You might not be able to move from where you are now, since that could affect their court-ordered visitation.
Personally, I'd only let them see the child at supervised events right now, since they're being pushy about it, and not let them take the child alone. When she's older, maybe I'd change my mind, but not right now.
As someone else said:
I'd cut the one on one visits and let them visit with you and your husband or other family functions. Be very sweet and say oh tgat doesn't work for us us - but you're welcome to see her at so and so and so's pic nic or we'd love to have you join us for dinner.