Solipher SHARD CodeContext
Compiles the smallest slice of your repository a coding task actually needs, under a hard token budget, with every line traceable to the reason it was included.
The problem, and what SHARD CodeContext actually does.
The problem
Ask a model to change code it cannot see and it will answer anyway. In our own measured run, five of six answers came back written in Python for a Rust repository, confidently describing an API that does not exist. The usual fix is to paste whole files, which works and costs thousands of tokens per request.
The solution
SHARD CodeContext treats this as a constraint problem rather than a search problem. It works out which code the task genuinely requires, guarantees that code is present, fills the remaining budget with whatever else helps most, and refuses to answer at all rather than quietly return a slice that is missing something required.
It also decides when to send nothing at all
Measuring the product honestly produced a result that changed its design. For a self-contained request like “write a memoized Fibonacci function”, attaching repository context costs about 4,600 extra input tokens and buys nothing, because the answer does not depend on your code at all. So requests are routed. If the task resolves to real code, it compiles. If it resolves to nothing, it sends no context and constrains the answer instead. If it is not a coding request at all, it passes through untouched, because forcing a code-only answer onto a question makes the answer worse rather than cheaper.
codecontext compile --repo . --task "fix the IDF floor in bm25_score" --explain
route: compile (task resolves to 1 span in this repository)
bundle: 16 spans, 2995 tokens
codecontext compile --task "write a memoized fibonacci function"
route: minimise (no repository supplied)
no context compiled; max_output_tokens=2048, reasoning=lowThe full evidence
Every number below comes from a real run against a real checkout, with token counts read from the tokenizer rather than estimated. The benchmark is not a fixture: continuous integration clones all four repositories and re-measures these figures on every push, so a regression in the compiler shows up as a failed build.
Read this before the numbers below
These results measure tokens, and whether the answer used the repository’s real API. They do not measure whether the code the model then wrote actually works. That is the metric that would matter most and it has not been evaluated. The sample is 11 tasks across 4 repositories, which is enough to expose real bugs and not enough to generalize from. Three of the eleven cases are a small loss rather than a win, and they are shown here alongside the wins rather than dropped.
Model
google/gemini-2.5-flash via Vertex AI, OpenAI-compatible endpoint
GCP project
weighty-planet-500504-k6
Tokenizer
cl100k_base, real token counts from the tokenizer, never estimated
Benchmark repositories
4 real open-source and internal repos: SHARD Context (Rust), ripgrep (Rust), Flask (Python), zod (TypeScript)
Codebase
4-crate Rust workspace, about 350 automated tests
Continuous integration
GitHub Actions, fmt, clippy -D warnings, full suite, and the benchmark re-measured on every push
Languages supported
Rust, Python, TypeScript, JavaScript
Tokens sent, compared against pasting the one relevant file
11 real tasks, 4 repositories. The baseline is the single most obviously relevant file for that task, pasted whole, which is what a developer or a coding agent does today. Every case served a bundle with complete coverage of the code the task required.
The result depends on how the codebase is written
Averaged per language, the three disagree sharply, and the disagreement is the useful finding rather than a problem to smooth over.
Large files holding many small definitions, so extracting one discards almost everything
Mixed file sizes, mixed cohesion
Small, already-focused modules, so there is very little to discard
What that actually means
A tool that selects a slice can only save you as much as the file contained code your task did not need. zod’s checks.ts is 1,293 lines of many small definitions, so pulling one out discards almost all of it: 632 tokens against 9,665. Flask’s modules are small and cohesive, so there is very little to discard, and two of the three Flask cases land within 2.5% of simply sending the file.
That is not a weakness in Python. It is a property of well-factored code, and the honest reading is that this product pays for itself on large or loosely-factored files and roughly breaks even on small ones. It detects that case and sends the whole file rather than spending your budget to lose.
Did the answer use the repository's real API?
Six real tasks against the SHARD Context repository. Same model, temperature 0, identical decoding settings. The only difference is whether the compiled slice was present. Scored by how many of that task's real symbols the answer actually used.
The counting understates it
On the last task both arms score 1 out of 5, so by symbol count they look equal. They are not. Without context the model produced a Python class with a Pydantic validator, for a Rust codebase. With context it produced this:
--- a/contracts/src/policy.rs
+++ b/contracts/src/policy.rs
-pub const MAX_MANDATORY_ATOMS: usize = 12;
+pub const MAX_MANDATORY_ATOMS: usize = 8;Correct constant, correct file, correct value, 182 bytes. Five of the six no-context answers were written in the wrong language entirely.
When context is the wrong answer
Real token usage returned by the API, for two self-contained tasks that do not depend on any repository. Output includes reasoning tokens, which are billed as output. Attaching context costs about 4,600 input tokens and buys nothing; routing the request away from the compiler and constraining the answer instead is cheaper than both alternatives.
Write a memoized Fibonacci function
Add memoization to factorial_recursive
Four real bugs the benchmarks found, and what was done about them
None of these were visible to hand-written test fixtures. They appeared only once the compiler was pointed at real repositories, which is the argument for benchmarking against real code rather than examples written to pass.
A bundle went over its own hard budget
The benchmark caught a 1,514-token bundle served against a 1,500-token budget. The solver only ever saw per-span costs, but the rendered request also carries the system contract and the task itself. Fixed by subtracting that envelope before budgeting, measuring each span's marginal rather than absolute cost, and re-measuring the finished bundle and withholding it if it still exceeds. The budget is now verified, not assumed.
Most modern TypeScript was invisible
Benchmarking zod showed that an exported const arrow function was not treated as a definition, so a large part of a modern TypeScript codebase could not be seen at all. zod's own safeParse could not be resolved. A second defect: the dollar sign was treated as a word separator, making every $ZodCheck symbol unreachable. Both were invisible to hand-written test fixtures and only appeared against a real repository.
Overloaded Python functions broke the compile
Benchmarking Flask surfaced that Python's @overload declares each signature separately plus the implementation, so one file held three definitions named stream_with_context. All three received the same internal identifier, and the solver correctly rejects duplicates, so any task touching that file failed outright. Fixed, and overloads in a single file are now treated as one symbol rather than an ambiguity.
The compiler used to lose to simply pasting the file
On small files the compiled slice cost more than the whole file. bm25.rs lost by 21.8%. The compiler now compares against that baseline after compiling and sends the whole file when the file is genuinely cheaper, taking the worst case from 21.8% worse to 2.9% worse, which is the rendering envelope and irreducible if the output is to carry provenance.
What's real today, and what isn't yet.
The command-line tool and the editor integration are genuinely usable. The HTTP proxy is a correct demonstration, not a service you should deploy. Both things are true, and the difference matters more than a single readiness label would.
Real today
- A working compiler, not a prototype: repository walk with real safety filters, syntax-aware chunking at definition boundaries, a symbol index with a one-hop dependency graph, mandatory-atom extraction, and a budget-bounded selection stage, all with real test coverage
- 11 real tasks across 4 real repositories in Rust, Python and TypeScript, all served with complete coverage of the code each task required, re-measured by CI on every push rather than quoted from a fixture
- Answer quality measured against a live model, not asserted from architecture: with the compiled slice present the answer used the repository's real API in every case, and without it five of six answers came back in the wrong programming language
- A hard token budget that is verified rather than assumed. The finished bundle is re-measured after rendering and withheld if it exceeds, which is how a real 1,514-against-1,500 budget violation was caught and fixed
- Deterministic output: the same repository, task and budget produce byte-identical results, verified against a real repository where filesystem ordering is the thing most likely to break it
- Refuses rather than guesses. An ambiguous symbol reports every match instead of picking one, a budget too small to cover the required code abstains instead of returning a partial slice, and a task that resolves to nothing is routed away from the compiler rather than given irrelevant context
- A persistent, per-file index so an edited file is the only thing re-processed, plus an optional watcher that absorbs that work in the background, taking the call right after an edit from 423ms to 190ms on a real repository
- Secret redaction before any code is stored or sent, covering cloud keys, private key headers, tokens and obvious credential assignments, with a deliberately narrow, high-precision rule set rather than a claim of complete coverage
- An editor integration over the Model Context Protocol, driven end to end over a real connection, which tells an agent honestly when a task needs no repository context instead of returning a pointless slice
- An OpenAI-compatible HTTP endpoint, live-verified against a real model, with authentication, per-tenant access control checked twice per request, signed output, and a startup refusal to bind to a public address without a token configured
- Four real defects found by benchmarking against real repositories and fixed at the root, each with a regression test, rather than a claim of first-try correctness
Not yet, honestly
- Task success is not measured. Every number on this page is about tokens and about whether the answer used the real API. Whether the resulting code actually passes the repository's own tests was never evaluated, and that is the metric a serious claim would need.
- The evaluation is 11 tasks across 4 repositories. That is enough to show the mechanism works and to expose real bugs, and nowhere near enough to generalize. A published result would need hundreds of tasks on a standard benchmark such as SWE-bench or RepoBench.
- No comparison against published retrieval baselines. There is no BM25-only, embedding-RAG, repo-map or prompt-compression arm, so the honest claim is against pasting a file, not against the state of the art.
- One model, one run per case, no variance reported. Everything was measured on gemini-2.5-flash at temperature 0.
- The task classifier that decides whether a request needs repository context is a hand-written word list. Every other component has measurements behind it; this one does not, and its miss rate on real prompts is unknown.
- The HTTP proxy is not production software. It serves one request at a time, has no metrics, no rate limiting, no TLS, no graceful shutdown, and keeps its run history only in memory, so a restart loses the audit trail. The command-line tool and the editor integration are the parts fit to use today.
- Latency numbers were measured locally and are not re-measured in CI, unlike the token numbers. They indicate the shape of the cost, not a reproducible result.
- Three languages, not more. Go, Java and C# are not implemented.
- The project depends on a private repository and carries no licence file, so it cannot currently be built or used by anyone outside this account.
Looking for repositories to be wrong about.
The most useful thing right now is a codebase this has not seen. Every real repository pointed at it so far has produced a real bug, and the language results say the benefit depends heavily on how a codebase is written, which is exactly the kind of claim that needs more than four repositories behind it.