autonomous research
Fully autonomous research systems are inevitable.
The state of AI-driven research today is where AI for coding was 8 to 12 months ago. Inconsistent moments of amazement with spiky task and prompt dependent performance.
Right now, AI-driven research is very primitive. You need a human handling the systems end-to-end, similar to what coding is to software engineering. Most AI-driven research today is from humans fitting the AI within a narrow enough problem space for them to make progress. The mistake that some people make here, is assuming that humans will always be necessary for this and there is some kind of abstract "research taste" that will be necessary in the near term future to do SOTA research. We are at a temporary moment in time where humans are needed to get the most out of these systems as they scale and iterate, but said scaling and iterating is converging on replacing the handler - not the other way around.
When I do research with AIs, this is what I provide them:
- Mistake analysis
- Verification
- Coordination
- Compute access
- Process
- Research Strategy
- Continual learning over long contexts
- Pattern recognition on long horizons (basically all of the above haha)
I am not providing any value that cannot eventually be solved. Everything that I am doing and more is verifiable and will therefore be solved. Some are compute constrained, some are capability constrained, some are RL resource constrained, some are tooling constrained - but in this industry, a constraint has almost always implied solvability.
After my experience using these systems, I do not think an autonomous frontier research agent can exist today. Most labs or companies claiming otherwise right now have not built a true harness. Most are using hack-y harnesses that force certain behaviors that do not scale or generalize with model behavior. In the near future, the best harnesses are those that use minimal task-specific tooling, the ideal harness for coding will also be the ideal harness for research, marketing, etc. The greatest lesson of my harness engineering endeavours is that a generalized harness is what the future will seek. This is because as we race to make every domain in society verifiable, we want the harness used to hit verifiable targets to be generalized as well (including for research). It's very likely that the great task for diffusion, acceleration, society, etc. over the coming decade with AI advances will be making all domains verifiable to AI systems to fit this trend line of how real progress gets made with AI. You are doing your harness engineering correctly if, when a model upgrade comes and you swap it in, that the system is immediately upgraded and then extensible by just adding more tools, context, and verifiability on its work rather than changing the core primitives the system operates on.
This is similar to the vast swath of coding agents made in 2024 that have all now centralized to a core agent tool loop + file tree system. Therefore, in my personal research ventures I apply the same base primitives that coding agents use and then just expand upon them:
AI requires scale by its very nature, even with every algorithmic advancement in the world it is the great polarizer of any distribution. The best AI systems will only be more centralized and resource intensive going forward. Attempting to argue that the best near term systems will prefer different harnesses is a fight against the centralizing and verification tendencies of this entire domain by its nature.
File trees are a horribly inefficient way for models to build continual learning - but it works with human guidance (for now) and can generalize a model's ability far more than its in-context learning window and is already existing proof that the value that humans provide to these systems is going to diminish with time as we find more processes to eliminate the work that we do. The design is pretty simple, a continuous orchestrator that follows the repo's guidelines and principles to synthesize and analyze the work so far. There is nothing a part of the research program that does not exist in the file tree. If anything exists outside of the file tree it is lost forever. Until we get unlimited and perfect attention algorithms for context (or continual learning) this must be a core principle for any system. If it exists in the file tree you apply many fun agent techniques over it to make the research process streamlined. It has enabled helpful techniques like front-matter, skills, third opinions, custom tools, etc. that help make the manual process that humans do prompting GPT Pro in ChatGPT easier. These are pretty basic primitives and nothing revolutionary is done here - because there doesn't need to be. The models are not trained to run on your custom research loop that guides it in a certain way to approach research, that will not work long-term and just be deprecated as model capabilities expand.
I'd like to write a long piece on my experience using the systems today and the specific weaknesses but I don't think that will be valuable for long. I think the best assessment will be, for people like me who cannot prove any of the math in the papers I write myself, what value do I provide? Those will be the best tell of the current bottlenecks and work to be done - which labs are already racing to (attempt) to do.
I can envision a day <1 year where a mid tier unsolved Erdős-level problem is put into a box and a lean verified solution returns later. Math, and to some extent, STEM research, is one of the easier ones because it is arguably even more verifiable than most of the day-to-day consumer uses for AI. This doesn't mean I don't think the other problems won't get solved either; simulated worlds, scaling RL (and letting different environments generalize), etc. there are many roads to solving more "abstractly" verified domains.
It sounds cringe to write out and sounded cooler in my head but I can definitely see a world where all "first-time events" in the modern day like an AI in a position of government, running businesses, etc. have all been simulated countless times in worlds against trillions of AI agents recursively improving themselves through RL and the best one wins out and comes to human land. Lol. What a beautiful world we live in where I can write that and it's a genuine possibility.
We should realize that the bottleneck to progress is how to conform our human systems to these, not the other way around, and then operate on that principle for future changes. I'm sorry that I have very little to personally offer to frontier research, if I did maybe I could help get my point across better. But I do hope that my work helps at least someone see the value in utilizing these systems to advance humanity in any domain and the good that can come of it. If you think there is a mistake in any of my work or there is some major way to improve please email me at sensho@sensho.xyz, I will try my best to fix it.