Writing this down because I’m not the only one having trouble with the unusual CMOS battery holder of certain HP OEM boards, in my case a HP Dorado OC Rev B motherboard from an Omen 30L or Omen 25L PC. Their battery holders are weird compared to others in that one cannot just push some lever to release the battery (cf. this Reddit thread). It turns out that the correct method apparently is to just wedge the battery out from below (there is ample of room between it and the board to fit a tool), as is shown in this video from minute 40:00 onward. It seems that a bit of force is actually necessary:
(The video is linked on HP’s support pages here (under “Service & Repairs (2) > Removing & Replacing Parts: OMEN by HP 40L Gaming Desktop PC”), so it seems to apply not just to Omen 40L’s, but in fact also to Omen 30L’s. The CMOS battery holder of my 30L does in fact look identical to the one from the 40L in the video. Everything at your own risk, obviously.)
For me, it worked like a charm and didn’t even require as much force as I feared.