Folks who are libertarians fundamentally do not believe in natural authority.
If you don't believe in natural authority, then the government CANNOT have any. and therefore, the government is functionally identical to the mafia. Pay up, or else. Full stop.
If you do believe in natural authority, the libertarian position looks insane. Bob, the boss, correctly has authority...and his actions are justified by his natural authority.
Contrarily, if you do not believe in the existence of justified natural authority...then the non-libertarian position looks insane. Some dude Bob has no right tell worker Willy what to do, unless it's a chosen exchange. Doesn't matter if Bob is a conqueror, a technocrat, a capitalist or a union leader. There are exactly two interactions in the world...freely chosen and thug-based. Mafia with guns and trading. Full stop.
I think it explains a lot of the miscommunication between libertarians and others.