What is CodeEvolver?
CodeEvolver is a coding agent that reflects on the errors in your AI application and makes changes to the prompts and code to improve your system.
The Problem
For many use cases, AI feels like a magic wand that delivers impressive results out of the box. One-off issues are a prompt away; simply tell your coding agent about an issue and let it code its way into an acceptable solution.
However, building reliable AI systems involves infinitely more work. Reliable AI systems are built by identifying failures, experimenting with different solutions (prompts, context pipelines, and code), and iterating until you're happy enough with your system. Then, you get a new customer, new AI model, or new use case, and your accuracy drops. Not only is this process time-consuming, but the outcome we settle on is often far away from the optimal solution.
Evals to the rescue
Reliability is achieved through eval-driven engineering. An eval instantly measures results and should be used to test various approaches. It is often unclear which prompt, skill, memory configuration, or context pipeline will perform the best.
Combining evals with self-improvement: CodeEvolver
The vision behind self-improving AI is to provide a system with a long-term objective, and the AI will improve against that objective until an optimal solution is found. When you have good eval data (e.g., with ground truth labels), the eval itself is the objective. CodeEvolver brings this vision to life. You provide an eval dataset and it modifies any codebase until it can no longer improve against its reward metric.
How does CodeEvolver work?
CodeEvolver combines our own proprietary optimization algorithm (the artificial selection algorithm) with autonomous coding agents powered by Claude Agents SDK and/or OpenCode.
Our approach combines:
- Artificial Selection Algorithm: Our Artificial Selection Algorithm (ASA) balances exploration and exploitation through an evolutionary algorithm that's driven by agent-driven hypotheses and candidate selection.
- Reflection Agent: A reflection agent analyzes past failures in your system and proposes changes to the prompts and the code. Insights from experiments are aggregated over time in memory.
- Coding Agent: A coding agent executes changes to the code base.
- The evaluation pipeline measures improvements against your reward function, and GEPA saves the best performing candidates.
Getting Started
CodeEvolver is offered as part of our AI engineering services. Sign up here!
Before running CodeEvolver, we will support you with three pre-requisites:
- Reward function: We define an objective metric that the system attempts to maximize, along with any constraints
- Evaluation dataset: We define a representative sample of inputs into your AI system. In some cases, this dataset also includes labeled ground truth, if that is required for your reward function.
- AI system: An AI system is defined by a series of one or more language model calls. We work with any AI framework!