Can a third button be added for "I disagree"? If the "I disagree" button is chosen then neither the upvote or downvote button could also be chosen so that a user disagreeing with a statement would not affect the vote count -- or it could be that if you choose the "I disagree" button then you could still upvote (but not downvote) to promote a good response that you just happen to disagree with.

That sounds kind of negative so maybe it could be called something better than "I disagree" or also include an "I agree" button if having four different buttons isn't too crazy.