Under certain circumstances, closed questions are automatically deletedautomatically deleted by the Community user. Otherwise, they can be deleted with varying numbers of votes from 10K+ users (depending on the score of the question and its answers). Finally, moderators can instantly delete any post.
As for why they're removed, people in favor of deleting closed questions generally refer to the broken window theory, which basically states that bad things accumulate at a rate proportional to the number of existing bad things. Following this reasoning, leaving unacceptable questions around encourages more bad questions.
Personally, I disagree. Closing, in my view, already boards up the broken window. The closure notice is a very good and very clear indicator that the question is not currently acceptable (if it ever was). Deletion makes it much more difficult for any useful information to be saved. We should close bad questions fast to prevent answerers from wasting effort on them, but once a good answer is posted, we should do some serious thinking before hiding it. The rules for automatic deletion already consider that - even a very heavily downvoted question won't be roomba'd if it has even one upvoted answer.
A special aspect of moderator insta-deletion is that such posts will not accept undelete votes from normal members of the community. Even though 10K+ users can see them in the "recently deleted" section of the 10K tools, we can't bring them back if we disagree with the deletion. In my opinion, that should be changed (isn't absolute freezing what locks are for?), but that's a different discussion.