Can I see which Answers/Questions count towards a badge? E.g. for the badge "Sportsmanship", can I see somewhere which votes counted towards that badge or would the only option be to manually go through all my upvotes (or write a tool to do the job)?
For some badges it's possible through the Stack Exchange Data Explorer but since your votes aren't public, it won't work for Sportsmanship.
It is possible through the API but it's rather cumbersome and will require multiple calls (first to obtain a list of your answers with positive score, then for each of them a list of other answers on the question, if you use an authorized token you could see if you upvoted them). You can rely on the badge tracker on your profile page that it indicates the right number of qualifying answers.