“I’m not voting”

What is a stigma between the people that say this, and why is there typically a negative connotation? I have my own ideas why but want to hear others.