If you’re communicating the fact that the books and things can’t be removed from the water without damaging them to the players, you could play into that with the loot. Things like powerful scrolls or even entire spellbooks, for example - but if they’re removed from the water, they quickly become useless. Let your party’s wizard figure out how they’re going to copy that spellbook when they have to do it underwater. Maybe they’ll keep the scrolls in bottles filled with water until they need them.

You could also include traps specific to that area. Think like, for example, a magical barrier that traps anyone who steals the books in that section - and underwater - until they put the things they took back. Maybe it malfunctions, and doesn’t de-activate.

It’s hard to give specifics, because you didn’t mention what level the party is expected to be when they get to this section.

Tridents,

conch shells of communication (pair of talking shells),

horn of blasting (big noise goes big boom)

Big old shell shield

Or shell plate (magic it as you’d like)

Pirate stuff

The location I may have put the most work into in my first campaign was an enormous hidden library and I loved it.

I was particularly inspired by the artwork of Giovanni Battista Piranesi and funnily enough years later I read the amazing novel Piranesi which takes place in an infinite, beautiful space with flooded districts.

I think my experience sounds less helpful, your idea sounds a little more mystical like The Library from Avatar: The last Airbender while mine was closer to that space at the end of Raiders of the Lost Ark where the government keeps all their anomalies.

But regardless, this is a library, so for the few who can access it, it’s still used for study. Information may not just be in books, but there may be marine alchemy stations, underwater instruments and weapons kept like a museum. Another option is times that let you learn how to overcome the resistances or immunities of underwater monsters, or deal extra damage to them, or even control them.

Also if people often get lost in the library, what happens to them? Are there corpses in the library or does someone clean them up? What happens to their items?

Finally I’m gonna pitch something I heard way back on a blog that I thought was really cool, and I’m assuming you’re running 5e, apologies if I’m way off the mark. The ethereal plane has no water, so it’s “oceans” are filled with ladders, ropes and bridges. Throughout this section of the library, you can lay down this lore, and eventually reveal a spell that lets you ignore water while submerged in favour of this mythic space of snakes and ladders. If some enemies or even a couple of PCs get their hands on this spell (or magic boots or whatever), underwater combat suddenly becomes very dynamic, so some creatures swim and others run and climb in the same space.

Libraries are mainly places of knowledge.
They could also include second that aren’t just books: a museum or natural museum, an art gallery, a meeting space for academic conferences, etc.
Was that part of the library used by an underwater civilization? Or was it about one?

If it’s the former, even our idea of a book or scroll might be wildly different from them.
If they lived underwater, instead of using ink on paper/vellum, maybe they carved on things, or arranged shells or sand, tied knots in strands of rope like material, or tapestries, or influenced the growth of corals in ways to display their ideas, etc. If they used ink, maybe it was more like a 3d holographic representation but with pigments suspended in water.
Turbulence, moving through might simply make it ineligible (or maybe it can be reversed, see below)

1 minute science video about fluids

Did they even have light?
Bioluminescence might be something cool here, where the “books” themselves might literally be live bio things that are alive.
Or maybe just the ambient light can be provided by bioluminescent plankton as you move around.
Maybe their way of displaying an idea was recreating specific movement in the water so the plankton is simulated in a specific pattern to display.

You can probably also steal and reskin some of the stuff in the candlekeep adventure if you need side content. Won’t be a drop-in replacement but the general vibe is there.