Now, should I delete my answer (as it adds nothing that hasn't been mentioned in answers to duplicates of that question)?
Not necessarily. It's not a mistake to answer a question, per se.
But, unless you've added something completely new – or your answer was accepted and upvoted already, you might just want to remove it. Naturally, you're not obliged to search for duplicates before you answer, but at some point – if you have a feeling there might be tons of duplicates – why not help clean up the site and find one of those. Then, flag the question for closing, or vote to close (if you have the privilege to).
Not answering obvious duplicates over and over again will help keep the important information together in one (hopefully canonical) topic*, and not spread it across a series of posts, endlessly repeating the same old.
* If the original "reference" question is quite old and outdated, we might have a problem though. I've discussed this here: How can we take care of our old duplicates?
Or will the whole (closed) question be removed at some point anyway?
No, in most cases it won't. Duplicates stick around and point to the "original" question for reference. If they're 1:1 the same, they might even be merged. We try to keep them around so search engines might give us more traffic.
Not everybody uses the same wording to ask a question, so therefore it's justified to keep them on the site – as long as they're reasonably good questions, of course.