Often I come across perfectly valid questions which have been down-voted for no apparent reason, sometimes with no comments or answers at all.
I don't think a down-vote should ever be made without explaining the reason, but if you want to continue to allow it, there should at least be a penalty for doing so. We are not in the world of Franz Kafka, are we?
If the comment and down-vote are tied, then the down-vote can be reversed if the comment is deleted.