If there weren't replacement refs, people would be complaining about the regular refs. Perhaps not so many issues at one time, but we would still see complaints about the refs.
Anyone who has played the game can tell you that there is holding on every play, yet it isn't called on every play. The refs have a thankless job and the replacements just magnify the difference.
These cannot be D1, D2 or D3 refs, as they are under contract for the season. The replacements are a group of retirees, lesser leagues, and non-contracted refs. The NFL needs to face the issues and get the regulars back under a long term contract.
To those suggesting boycott, you will only hurt your local economies. Think stadium workers, parking attendants, hotels, eateries, food services, delivery services, taxis, shuttle drivers, local taxes, etc. The best way to impact the NFL owners is to complain to sponsors, advertisers, and the networks. A constant stream of complaints will force the networks to convince the NFL owners that they must improve the product that the networks are paying for the rights to air.