No, I do not think you are crazy! If breakfast ADRs work for you better than dinner ADRs then that is exactly what you should do! Everyone needs to do what works best for their family. In fact, I think you are very smart to be doing this so that everyone has a good time.
I do agree though, with a PP who said they don't enjoy it so much to eat a lot of buffet food and then ride things. I'm the same way. But then, it depends on what you ride. When we went to DHS we did the character breakfast before the park opened, and we all ate a lot. However, I don't go on thrill rides, and DH was not sure if he would have time for a thrill ride but he knew that if he did it would be later in the day anyway. He also eats a full meal and then could ride a roller coaster with no problem! He did this at Six Flags a few weeks ago (he ate a Nathan's hot dog, fries, and soda and then rode Nitro!). Also, the first thing I did at rope drop at DHS was grab fast passes for us for Toy Story Mania, while DH and the boys headed to wait in line for the Playhouse Disney show. Then I met up with them and we went to the first showing if Playhouse Disney Live On Stage. After that we rode Toy Story Mania. DH did get to ride Tower of Terror, his all-time favorite ride in all of WDW. He did this shortly after eating lunch at Pizza Planet too!
So, like I said, you have to do whatever works for you and your family. For us, we prefer to have a TS dinner every night, with a few character breakfasts thrown in here and there. On a 5-night trip last summer, we did 2 character breakfasts: the one at DHS, and also Chef Mickey's on the last morning right before leaving, so no theme park that day. The other mornings we had breakfast in our room before heading to the parks (we brought some breakfast foods with us like cereal and cereal bars).
By the way, the character breakfasts are awesome! And the food is very good too...lots of variety as well! Have a wonderful time and enjoy!