Q: I seem to be upsetting my co-workers, and it's completely unintentional. They think that I'm unfriendly and that I think I'm better than they are. I'm just a quiet person and not very social. How can I get things on a better track?
A: Take a close look at unintended messages you may send, and find ways to help them understand you.
The inner game
When you're upset and feeling misjudged, it can be hard to see clearly. Take some time to get calm, focusing on your breath until you feel settled, and letting yourself relax.
How did you become aware of the issue? If a co-worker raised it, you have a good opening to find common ground. If it was your boss, you should be able to get more specifics so that you can consider ways to address it, but you should also determine if it's seen as a performance issue.
What do your co-workers see? If they lunch together, chat during the workday, even socialize outside of work and you always decline, this may be sending the message that you don't care about them. Or perhaps it's in the way you demur — thinking objectively, could your tone be taken as superior, as though their activities are beneath you?
What do you want? Yes, a "better track," but that could mean a number of things. Your most likely path to achieving it is a combination of explaining yourself to them and increasing your engagement with the group at least by a little.
The outer game
The situation you're in is really a matter of introversion vs. extroversion — there are people who get energy from others, and those, like you, who get energy from within. As an introvert, you can develop strategies to shape interactions to be more comfortable. For example, bring in a treat and let everyone know there are cookies in the break room. It sends a friendly message in a low-interaction way.
You can also use this perspective to help building shared understanding. If it has come to you directly that there's an issue, use this as a chance for discussion, figuring out what you'd like people to know about you. For example, it may be that you're uncomfortable in large groups — so skipping lunch outings isn't because you don't like your co-workers, it's that going out with a big group wipes you out.