Edit: obligatory explanation (thanks mods for squaring me away)…
What you see via the UI isn’t “all that exists”. Unlike Reddit, where everything is a black box, there are a lot more eyeballs who can see “under the hood”. Any instance admin, proper or rogue, gets a ton of information that users won’t normally see. The attached example demonstrates that while users will only see upvote/downvote tallies, admins can see who actually performed those actions.
Edit: To clarify, not just YOUR instance admin gets this info. This is ANY instance admin across the Fediverse.
There was already bullying by people, and it didn’t need downvotes for it to happen. To solve it, it just needs moderation.
Maybe people can also be nicer and downvote less, for a change.
Some instances don’t allow downvotes by people logged into that instance, which I think helps. (From both sides: I find that when I can’t downvote, I have a lot less motivation to read anything that makes me angry. I just keep scrolling).