It’s the inability to do anything about it.
Part of the frustration for merchants is that it’s virtually impossible to negotiate with the card associations, according to Kenneth Posner, a Morgan Stanley analyst. Merchants can choose between two options: pay the fees and abide by the card association's rules, or simply don't accept credit cards.
“They have shown an ability to raise prices even when merchants complain,” he said. “The big frustration is not necessarily the magnitude of the cost. It’s the inability to do anything about it.”
Source: Quad City Times, An increase of ‘intercharges’