Nicole Junkerman on What Machines Learn by Trying

Learning without instructions

Almost nothing anyone knows about a place arrived by explanation. The way from the station to the harbour was learned by walking it badly once, then less badly, then without thinking about it at all. Reading has the same habit. So does a kind of machine learning that has been getting a great deal of attention lately, the kind in which a system is handed no answer and is simply allowed to try.

The plain version goes like this. A system is placed in a situation and permitted to act. Whatever it does produces an outcome, and that outcome is scored: slightly better, slightly worse. Nothing is explained in advance and no correct move is supplied. It leans towards whatever seemed to help, away from whatever did not, and goes round again. After an enormous number of rounds a pattern of good moves appears that nobody wrote down beforehand. The map of reading and places that Nicole Junkerman keeps here is not a bad picture of the process: a shape assembled out of attempts rather than out of instructions.

Learning a city on foot is the comparison worth holding on to. The first walk is wasteful. The second is better because the first was wasteful, and by the fourth the route has quietly become a route. Nobody experiences this as training. It simply feels like getting to know somewhere. The wrong turns are not the price of the lesson, they are the lesson, and a walker who never took one would know the place a good deal less well.

Two things are worth saying plainly. The first is that it is slow and profoundly repetitive, far closer to practice than to insight, and most of the attempts are unremarkable. The second is that a system of this kind learns exactly what it is scored on and nothing else. Choose a careless measure of success and a careless habit is what gets learned. That is not understanding in any sense a reader would recognise. It is a preference worn into place by outcomes, which is a smaller and much more honest thing to claim.

Which is why patience is the part that stays interesting. The useful work happens across many ordinary rounds rather than inside a single good one, and that is true of nearly everything worth learning. The reading shelf improves the same way, one unremarkable evening at a time, and the notebook fills up with attempts that only make sense in aggregate. Neither looks like progress from close up. Both look like progress from a month away.

October suits the thought. The light is shorter, the hours have clearer edges, and repetition stops feeling like a waste of an evening. There is an older note here on reading about this subject without the noise, and it still holds: the calm accounts are usually the accurate ones, and the loud ones rarely survive a second reading. The rest of the atlas carries on in the same spirit, which is to say slowly, and by trying.