Its not that easy because users can undo their votes, or change an upvote to downvote. Thats why we store every individual vote in the database. Also votes from defederated instances are discarded, which wouldnt work if there is only a total vote count from the community. And in general Activitypub is push-based, not pull-based.