I'll just add some rules I'm trying to follow:. I've been guilty of all these things (and sometimes still am, probably), but at some point I've recognized them as harmful for the community.
When downvoting or VTC, always try to leave a comment
Is the question unclear? Let OP know what information is missing or which part you don't understand. Is there no info whatsoever? Tell them that you can't just guess what their issue is and what details they should provide. I know, I know: "it doesn't work plz help" and other users with magical thinking grind our gears, but bashing them doesn't get them any closer to understanding the problem with the way they ask for help.
Is the formatting bad? Let them know that the question is hard to read and they have to improve it, otherwise people who could help them won't even bother with reading it. This tells them why good formatting is important and that it's in their best interest to care about it. Tip: [edit] in a comment magically turns into an edit link.
Is it slightly off-topic and there are no sites where it could be migrated? Consider at least nudging OP in the right direction in the comments. Maybe recommend a non-Stack* community if you're aware of one.
The question is chatty and doesn't fit the Q&A format? Heck, consider abusing the comments sections and being helpful.
TL;DR: Use downvotes and VTC to be helpful, not toxic.
Think twice before closing as unclear
Is the question actually unclear, or is it just unclear for you because you're not an expert on the topic?
On multiple occasions I was able to provide a complete, detailed solution and just solve someone's problem, but the question got closed as unclear when I was typing the answer. I can vote to reopen, but my one vote is not enough. Even if it actually gets reopened later, I probably won't get back to it and it may remain unanswered.
At the end of the day maybe it's better to leave a question opened and unanswered, rather than closed while it could be answered?
Recognize if OP tries to improve
We're not here to discipline users, but to help them. Don't punish them for not making the question perfect or not knowing something. Work with them towards that goal instead. You're more knowledgeable than them, so try to get them on your level.
If OP keeps editing the question, maybe it's better to abstain from the final VTC to avoid discouraging them?
Avoid emotionally loaded adjectives
Pointing at specific problems with a post is much nicer and more productive than expressing how you feel about it.
Rather than call a question "badly written", you could say it's not cleanly formatted or that it lacks proper punctuation and capitalization and is therefore hard to read.
An answer may be a "terrible advice", but it's better to say that it's factually incorrect, based on false premise, risky or potentially harmful.