You should just never lose it. Voting isn’t a privilege to be awarded for good behavior, never mind that laws are sometimes themselves unjust and often disproportionately applied. Should people in prison for weed possession be unable to express their opinion about whether weed should be illegal?

Voting is both important and generally not that critical. The idea matters a lot more than the actual impact most votes have. There’s just not much in the way of compelling benefit to denying prisoners their vote and plenty of risk of the law, either intentionally or unintentionally, being an implement that can affect who gets to vote on the people that make and enforce the law.