The versions of yourself that you present and others present to you are simple and easy to understand. It isn't difficult to for someone to believe that they know you when who you are is a presented set of simplified traits. It's also why people seem easy to read. In reality, who you truly believe you are and who others believe themselves to be is generally internalized and imperceptible to others:
Researchers asked people to describe a time when they feel most like themselves. Most subjects, 78 percent, described something internal and unobservable like the feeling of seeing their child excel or the rush of applause after playing for an audience. When asked to describe when they believed friends or relatives were most illustrative of their personalities, they described internal feelings only 28 percent of the time. Instead, they tended to describe actions. Tom is most like Tom when he is telling a dirty joke. Jill is most like Jill when she is rock climbing. You can't see internal states of others, so you generally don't use those states to describe their personalities.
This phenomenon—what psychologists call the illusion of asymmetric insight—creates a lot of problems. For instance, it allows you to completely reject what others believe because you think you understand it, and remain convinced that they'd agree with you if only they understood your point of view. Basically, you think you can understand everyone else and nobody can understand you. It gets even worse in groups. If you need an example, take a look at politics.
In reality, nobody's really right or wrong. We're never going to be able to paint a full picture of ourselves even if we're completely honest. Because much of who we are is internalized, we'll always project an incomplete version of who we are. When we look at other people, however, we have to remember that they're doing the same thing. It's easy to argue and disagree with what we see, hear, and observe, but the full truth is often often beyond our reach. Next time you disagree with another person or group, remember that you probably don't truly know and understand their point of view. You may agree on more than you're able to perceive.