I like library providers that can provide mechanical upgrade instructions. For example:

model.adjust(x,1,y) is now model.single(Adjustment.Foo, x).with_attribute(y)

Or whatever. Then people can go through your instructions find-and-replacing the changes, or even better, have an automated tool do it.

Also you pay some of the maintenance burden by writing all this documentation, so you have a some stake in keeping the changes minimal.

The chess engine’s training is anchored by the win/lose outcome of the game. LLM training is anchored by what humans like to read and write. This means that a human needs to somehow be in the loop.