The government retains a monopoly on force because we (implicitly) agree that its use of force, in the form of institutions, is preferable, in its reliability and consistency, to individual use of force. Outside of the context of the concession of the monopoly of violence to a central authority or authorities; immediate self-defense is not the only valid use of violence.

Outside of that context; that is to say, regarding the morality and not the legality of an act, one would have to have a fucking death wish to disregard the use of violence outside of the context of immediate self-defense. That’s the whole reason cultures of honor get started - because if you do not react to threats and advocacy with force, because if you sit there and meekly let Clan McNazi from across the highlands whisper that all of ‘you people’ in your clan should be killed while you’re trying to work the fucking fields, because if you try to play tit-for-tat, all that ends with is you and all of your family in a shallow grave, or in chains.

In a civilized society, the position of reacting with force is taken up by the state, however flawed and imperfect this system is. We haven’t stopped reacting with force in non-self-defense contexts, we’ve merely outsourced it to a (theoretically) representative body.