How a set of failed attempts to improve training led to a more useful question, and the company that grew out of it. In short: Aethos trains open models for companies that want their own, and a few epochs into each run it tells them whether the rest of the
Why I started deciding what counted as success before seeing the data — and why that turned out to be only half the problem. The hardest experiments to interpret weren't the ones that failed outright. They were the ones that almost worked. A scheduler that finishes a fraction of
Five attempts at per-sample interventions, the mechanism each one tripped on, and why uniform sampling kept winning. Not all training examples are equally useful to a model, which makes it tempting to train more efficiently by spending more of the model's attention on the valuable ones — sampling
Notes from a multi-dataset NLP study on how aggressively training data can be trimmed before accuracy starts to slip. Anyone who's trained a model at production scale has done some version of the math: GPU-hours times dollars per hour times how many runs you'll
What I learned trying to use per-sample data quality signals to control machine learning training — and what they appear to do well instead. There's a common idea in machine learning that goes something like this: If we could measure what each data point contributes to learning, we