Try this: if you're in a comment thread with someone, upvote them for keeping up the conversation. That way, if they let you down or lose your interest, you can simply not give them their upvote. Save the downvote for especially galling, hopefully rare moments.