I can confirm this drives the one behind you crazy.

The other day I drove behind someone that did like that, first they lowered the speed for a few minutes then they passed a speed limit sign and increased the speed.

I ended up trying to match their lowest speed and giving them plenty of space.