Unfortunately, I don't think you are wrong, and the problem stems from having the downvote as an anti-upvote. If you upvote posts you like, you're going to downvote ones that you don't. That's a natural association that takes effort to break. I would prefer no downvoting at all, but if you're really worried about shitposts, have an up button and a report button instead. If something gets "reported" enough, then it can be hidden. I've seen this system used on other sites, and it works great. Users don't "report" posts or comments just because they don't like them. It's taken a lot more seriously, because you know that somebody with authority (a mod or admin) will see it.
Unfortunately, I don't think you are wrong, and the problem stems from having the downvote as an anti-upvote. If you upvote posts you like, you're going to downvote ones that you don't. That's a natural association that takes effort to break. I would prefer no downvoting at all, but if you're really worried about shitposts, have an up button and a report button instead. If something gets "reported" enough, then it can be hidden. I've seen this system used on other sites, and it works great. Users don't "report" posts or comments just because they don't like them. It's taken a lot more seriously, because you know that somebody with authority (a mod or admin) will see it.