A cyclist weaving through traffic is not reacting to the street as it is. The body is already braced for the pothole a second ahead, already leaning into a turn that has not started yet. Psychologists call this constant, small-scale forecasting nexting, and a group of reinforcement learning researchers borrowed the word to describe a robot that learned to do something similar [1]. Reinforcement learning is the branch of machine learning in which software improves through trial and error, and one of its quieter strengths turns out to be learning to predict.
What made the robot practical was not a large neural network. It was tile coding, a simple technique with roots in an older control method known as CMAC [2], and one that most students meet briefly before moving on to deep learning. Looking at where tile coding has actually been put to work suggests it deserves a second look, less as a way to make machines act and more as a way to help them anticipate.
Overlapping Grids
Imagine laying a sheet of graph paper over a map. Any location now falls in one square, and everything learned about that square applies to the whole of it. That is a coarse way to describe a place, since two points just either side of a line end up in different squares and share nothing.
Now add a second sheet, shifted a little, and a third shifted a little more. A location falls in one square on every sheet. Two nearby locations share most of their squares, two distant ones share none, and the overlap between sheets lets the system tell apart points much finer than any single square. That is tile coding. Each sheet is a tiling, each square a tile, and the set of tiles a reading lands in becomes its description [3].
The same trick works for any measurement, not just position on a map. A motor's current, a joint's angle or a room's temperature can all be described this way. Richard Sutton's reference software for the technique, called tiles3, is short enough to fit on a single page [2].
From an engineering point of view, the appeal is predictability. Every reading lights up the same small number of tiles, so each learning step does the same amount of work, and the software caps how much memory the tiles can use [2]. Learning about one situation also leaves unrelated situations untouched, which matters a great deal for a system that has to keep learning while it runs.
Asking Questions About the Future
The second idea is about what to predict. Reinforcement learning usually estimates how much reward lies ahead. The nexting research kept that machinery but pointed it at ordinary sensor readings instead, so the robot could learn how bright its surroundings were about to get, or how hard a motor would be working over the next few seconds [1]. A single setting controls how far ahead each question looks, from a fraction of a second to several seconds.
Because every question reads from the same tiles, asking one more costs very little. There is no new model to build, only one more small set of numbers to keep up to date.
Thousands of Forecasts on a Laptop
Every wire on a hobby robot like this one carries a signal that could be forecast, battery level, wheel effort, the distance reported by the sensor up front. The research robot in the nexting study had several dozen such signals. Its learning system made more than two thousand forecasts about them at once, from an ordinary laptop, and still finished each round of updates comfortably inside the robot's tenth-of-a-second rhythm [1]. Most of the learning was done within about half an hour of driving around its pen.
What the Robot Learned
Speed alone would not make the result interesting. The researchers also compared the robot's forecasts, learned on the fly, against the best forecasts that could be worked out afterward with all the data in hand, and the live versions held up well [1]. A single set of tiles, built once, was enough to forecast light, motion and motor signals across every time horizon tested. Nobody had to design separate inputs for each kind of signal.
One small detail in the paper hints at a practical use. The robot's motors had overheating protection and would stop to cool down every so often [1]. In the experiment that was just a nuisance. For anyone responsible for motors, pumps or fans, a forecast of the next few seconds of effort and heat, learned from nothing but the machine's own sensors, starts to look like an early warning.
Anticipating the Next Joint
A powered hand like this one has more joints than its wearer has muscle signals to drive them. Many prosthetic arms controlled by muscle signals solve this with a switch that steps control from elbow to wrist to hand in a fixed order, an approach widely regarded as slow and difficult to use [4].
Predictions That Help a Person
Patrick Pilarski and colleagues at the University of Alberta asked whether the same kind of forecasting could reorder that list as a person used the arm, offering first whichever joint the person was most likely to want next. Their system tile coded the arm's position and other signals, learned its forecasts in real time on a laptop, and on a standard skills test beat even the best fixed order, one identified afterward from the same session, cutting the expected wait between joints by more than 14 percent [4].
Later work moved closer to everyday use. Real-time prediction learning of this kind was demonstrated with an amputee using a training robot arm [5], and a follow-up case series found that adaptive switching reduced both how often and how long people spent switching, for an amputee and for able-bodied participants [6]. These are small studies, and clinicians are far better placed to judge what they mean for people who wear prostheses every day. From a machine learning perspective, the notable part is that the system learned from each person's own signals while the arm was in use, with no separate training phase beforehand.
What This Suggests for Builders
For anyone building software for physical equipment, this work points to a pattern that seems underused. A handful of sensor readings, overlapping grids, and a set of simple forecasting questions can give a machine a running sense of what is about to happen, on hardware no bigger than a laptop and at a cost that is known in advance. The tiles3 manual covers most of the practical setup, such as choosing how wide each tile should be and which measurements to combine [2].
The catch is that the number of tiles explodes as more measurements are combined. The nexting robot, for instance, tiled most of its sensors on their own or in small pairs rather than all together [1]. Deciding which measurements belong together is where knowledge of the equipment enters the design. For richer, more complex data, one research team working on laser welding first used a neural network to boil the raw data down to a few key features, then handed those to the same kind of lightweight forecasting [7]. Whether that pairing, a learned summary in front of a cheap and predictable forecaster, becomes a common recipe for small devices is still an open question. It looks like a reasonable bet for machines that need to keep learning from their own sensors without ever stopping to retrain.
References
- J. Modayil et al., "Multi-timescale Nexting in a Reinforcement Learning Robot," in From Animals to Animats 12 (SAB 2012), Lecture Notes in Computer Science, vol. 7426, Springer, 2012, pp. 299–309. DOI: [Online]
- R. S. Sutton, "Tile Coding Software, Reference Manual, Version 3.0," Reinforcement Learning and Artificial Intelligence Lab, University of Alberta, accessed Nov. 6, 2026, [Online]
- R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction, 2nd ed., Cambridge, MA, USA, MIT Press, 2018, sec. 9.5.4, p. 217. [Online]
- P. M. Pilarski et al., "Dynamic switching and real-time machine learning for improved human control of assistive biomedical robots," in Proc. 4th IEEE RAS/EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob'12), Rome, Italy, 2012, pp. 296–302. DOI: [Online]
- P. M. Pilarski et al., "Adaptive artificial limbs: a real-time approach to prediction and anticipation," IEEE Robotics & Automation Magazine, vol. 20, no. 1, pp. 53–64, 2013. DOI: [Online]
- A. L. Edwards et al., "Application of real-time machine learning to myoelectric prosthesis control: A case series in adaptive switching," Prosthetics and Orthotics International, vol. 40, no. 5, pp. 573–581, 2016. DOI: [Online]
- J. Günther et al., "Intelligent laser welding through representation, prediction, and control learning: An architecture with deep neural networks and reinforcement learning," Mechatronics, vol. 34, pp. 1–11, 2016. DOI: [Online]
The Guard at the Front Door
A language model that declines a harmful request looks like a model that cannot help with it. Research on prefilled replies, a single internal refusal switch, and cheap fine-tuning suggests that safety training mostly shapes the first few words of a reply and leaves the underlying ability in place, and that telling a refusal from an inability is the harder problem.
The Part You Meant to Keep
AI help tends to look good in the moment and cost something later. The evidence so far offers no single cutoff, and the line seems to move with what a person wants to keep. Could a cognitive offloading scale find it?
Working on something related?
Thinkata builds production AI systems of the kind these articles discuss. A 20-minute call is the fastest way to find out whether it fits.