Cursor pagination, and why offsets lie

Offset pagination quietly skips rows under writes. Here is the failure and the fix.

Every offset-paginated export has a version of this bug, and almost nobody has noticed theirs. It only shows up when a delete lands mid-export, and the symptom is a row that simply is not there.

Page two with an offset means rows 51 to 100 as the table stands right now. Delete row 30 while a client is paginating and every later row shifts up one: the row that was 51 becomes 50, and the client that asks for 51 never sees it. Nothing errors. The data is simply gone from the export.

Cursors fix this by naming a position instead of counting rows. Since v2.2.1 our cursors are anchored to a monotonic sequence, so a deletion in front of you cannot move what is behind you. Pass the cursor from the previous response and stop when it comes back null.

The cost is that you cannot jump to page nine. We think that trade is honest: nobody who needed page nine ever trusted what they found there.