A cycle cape to keep the rain off.
Microfibre cloth to dry the seat on rainy days.
Gloves for warm hands and protection.
Spare inner tube & a few tools.

Rack and panniers obviously, but specifically panniers you can remove and re-attach in a hurry.

I found it very frustrating to have pannier bags that were attached with these long velcro straps that went between the two bags, it made just moving the bike around a lot more difficult than it aught to have been when I had a bunch of groceries onboard or whatever. Now, obviously being able to detach the bags hasn’t made the ride back from the shops any easier, but hauling the lot up a few flights of stairs has become dramatically less unpleasant.