I think the differences between the Little Mermaid ride format and Pirates or Haunted Mansion is this:
With Nemo, Little Mermaid, Peter Pan etc, the movies were made, and were very popular, prior to the ride being designed. They were not designed to be E ticket rides.
With Pirates and Haunted Mansion, the rides were developed several decades before the movies. With the rides being designed as E Ticket attractions in and of themselves.
I think Disney has plenty of data on guest expectations and desires, and future E ticket rides are in the works, BUT they will more of the thrill ride nature than the gentle boats and omnimover (doom buggy) design. Unlike my generation, most today are not awed by the technolgy of lifelike birds and humanoid audioanimatronics...they want big thrill rides and "cute" childrens attractions for the family. I agree, kind of sad, but theme parks have to make a profit based on guest expectations.