I have lately noticed that as soon as I ask a question or post an answer, someone almost always downvotes it, and then often no one else ever does again. Sometimes, the downvotes go away after sometime, maybe because someone upvotes me then. How do I determine what is really going on? Is anyone else experiencing this?

EDIT: Someone is even down voting my super old answers note.