How to handle misunderstandings without blaming myself or another person, and without sounding like an asshole?

What is the right thing to do when me and another person misunderstand each other? I tend to take the blame even in conversations when I believe I wasn't wrong. I don't think this arrangement makes sense. It ruins my confidence to be called stupid and not know how to fix that without being rude or obsessive over a small detail. Someone calls me dumb and when I disagree according to them I can't admit I was wrong and they were just joking anyways. But can't they do the same by this logic? They're not admitting to anything either. I just want to find a way to handle misunderstanding in a way that doesn't blame anyone. I don't want to blow a small disagreement out of proportions but I don't like being the person who takes the blame every time.

A follow up question: When me and someone else disagree on a piece of factual information what am I supposed to do? What's the right thing to do when someone claims I'm incorrect when it's the other way around? I can't explain my point because the point is stuff like The shop closes at 8 PM not at 10 PM. I tend to back down as well is that the right thing? Thank you for your answers.