As Sathya [linked to](http://meta.stackoverflow.com/questions/111393/posts-ive-already-reviewed-show-up-in-the-review-section-again) this bug was also reported on Meta Stack Overflow.  

It has since been fixed, just cleaning up this bug report. (: