Did you have a hysteroscopy done to see the inside of your uterus to see if there is scarring? That could rule out that problem.
My oldest DD is the result of 4 rounds of IVF. We started with Clomid and IUI because our issue was male factor (low count and motility). I hyperstimmed on the Clomid at 50mg, which is the lowest dose they use, generally. I had my 3rd IUI canceled because I had 7 follicles and my RE was terrified of higher-order multiple (a la Jon and Kate). It's really important to be monitored on Clomid, it's nothing to mess around with. Please, don't let some doctor hand you Clomid and not monitor you....the risks are huge and the side effects can actually hinder your ability to get pregnant (cysts and thinning of the lining of the uterus).
We moved to IVF because it was too dangerous to stim me and do an IUI. With IVF, they had more control. The shots weren't bad. Not fun, but I did 2 or 3 a night and got used to it. Given my history of hyperstimming, I was at the doctor every single day for blood work. I looked like an addict.

My arms were all bruised, my veins collapsed. It wasn't pretty and that was the worst part. I enjoyed my egg retrievals. I love me some Fentanyl and Demerol. I was knocked out cold, woke up, and they told me how many eggs they got. Fun!
Cycle 1 and 3 were failures. Cycle 2 was an ectopic. Cycle 4 gave me my oldest DD. With cycle #4, we did a few things different. The other cycles, we just adjusted the medication a bit. Cycle #4, I refused to do it without further testing. I had an SHG done (normal). I had a recurrent loss panel done that required 22 tubes of blood be drawn from me (where it took 5 people to get all of it, one of whom asked me why I was there because she'd never seen someone have so much blood drawn...when I told her, she told me to go on vacation...I told her, "I thought about that, but this is really so much more fun!"

). All normal, except I tested positive for MTHFR. Over 40% of the population has an MTHFR mutation that can cause implantation failure and early losses. Unfortunately, it can also cause horrible late term losses before people realize they have it. I started mega doses of Folic Acid and Aspirin therapy. I began Metformin, which I read can help reduce hyperstimming (and, it did!). I got OHSS and was hospitalized, but that cycle, it was much less severe. I have PCOS, but I'm pretty much asymptomatic for it. My only symptom is elevated LH and my problem with hyperstimming on ovulation-inducing drugs, none of which I ever would have known about if I just got pregnant easily. Thankfully, unlike other people with more severe PCOS, I don't have to be on the Met forever because it's a tough drug to take. I will, however, have to take the Folic Acid and the Aspirin forever.
All of this would be useful without IVF, so it's something to think about checking out whatever route you take.
My 2nd and 3rd DDs were total surprises. People always say that once I got pregnant my body would somehow KNOW how to do it. Since our biggest barrier was sperm, I'm not sure how MY being pregnant helped his sperm, so I just nod politely and let it go.
So many of us have been there. You're not alone.
My BFF had a traumatic c-section that caused horrible damage to her uterus and she was able to get pregnant again. When they did her repeat section, the doctor said he was shocked, given what her uterus looked like. That kid found the one open spot in her uterus and stuck around (born on my b-day).
So, yeah, you never know. I suppose if I had just had more patience (as people have told me), I would have eventually gotten pregnant with DD2 and DD3, but I'd be missing DD1. She had to be created special in the lab. I don't regret for a second what I did to have her and I'd do it 100x more, even if someone said to wait and I'd eventually get pregnant....I do IVF 100x for her.
I wish you so much luck with everything! Definitely hook up with a Resolve group. I'm a volunteer counselor for Resolve (specializing in people going through multiple IVFs) and I also do some work in their insurance coverage section where I help people in my area navigate IF coverage on their insurance and pharmacy plans. They're a great resource and have group meetings and publications with information.