How GitHub Makes AI Coding More...

GitHub says efficient AI coding should be measured by the completed task, not by the token count of an individual interaction. Overly concise tool responses can force agents to repeat commands or retrieve missing information, increasing total cost and execution time.

The company evaluated four Copilot improvements with offline agentic coding benchmarks and then validated the strongest candidates in controlled online experiments. The examples come from GitHub Copilot CLI, while other products using the same harness, including the GitHub Copilot app and Copilot code review, can also benefit.

Chart showing 3.1% 'Remove view previxes', 5.5% 'Selective output compaction', 2.9% 'Compact task-tool prompt', and 2.3% 'Reduce notification roundtrips'. — Figure 1: Four independent A/B experiments using the same AI-credit metric. The segments are shown together for comparison; their effects are not necessarily strictly additive.

Avoiding the local metric trap

GitHub tested RTK (Rust Token Killer), which shortens shell output before an agent reads it. Although RTK reduced individual responses, agents sometimes reopened the original output or reran commands when omitted details mattered. The resulting extra turns caused tasks to use more tokens and take longer overall.

GitHub therefore measures efficiency across the full task, from the user request to the final result, rather than optimizing tokens per tool call.

Flow chart showing: RTK, compresses shell output > Local win, tool output gets shorter > Useful detail is missing > Recovery, reread or rerun > More turns and context carried forward. Then the option of finishing at 'End-to-end result, Tokens and cost up, Task duration up, Task completion: steady,' or 'Recovery repeats' going back to 'useful detail is missing'. — Figure 2: A shorter tool response can make the completed task more expensive when missing details force the agent to reread output, rerun commands, and carry more context forward.

Compressing repetitive output selectively

Benchmark analysis found that install, build, test, and lint output often contains repetitive noise, while source-like output and arbitrary command results are more likely to contain essential context. Early compressor versions were too aggressive: they caused agents to repeat work or read saved output, increasing cost and reducing success. GitHub removed an initial git diff filter after agents repeatedly reopened the original output.

The shipped policy:

Preserves source-like and arbitrary output, including results from cat, git diff, git show, and arbitrary scripts.
Reorganizes search results from tools such as grep without dropping matches or file lists.
Compresses repetitive install, build, test, and progress output only when savings are substantial.

Complete originals remain available through a direct recovery path. On offline tasks where compression activated, GitHub detected no statistically significant task-success regression, and agents rarely opened the saved originals. An online experiment slightly reduced average cost without a material regression in tracked quality metrics.

Flowchart showing how GitHub Copilot handles shell-command output. Copilot calls a shell command, classifies the output, then chooses one of three paths: keep arbitrary/source output unchanged, reorganize search results without losing any matches, or selectively compress repetitive noise (like install/build/test logs) while preserving full output and providing a recovery path. The processed result is returned to Copilot. — Figure 3: The shipped compressor preserves source-like output, reorganizes search results without loss, and compresses only predictable repetitive noise while retaining the full original.

Removing unused formatting

Copilot’s view tool previously added line numbers to every file line. Current editing tools match surrounding code instead, leaving those prefixes without a useful role in normal file reading. Removing them lowered model-inference cost by roughly 5% in offline agentic coding benchmarks, with success rates within expected run-to-run variance and no increase in edit failures.

In an online Copilot CLI experiment, average daily model-inference cost per user fell by about 3%, with no material regression in measured quality or satisfaction. The file contents themselves remained unchanged.

Before-and-after image of code snippets. The line-number prefixes re removed from the 'After' image. — Figure 4: Removing line-number prefixes preserves the source exactly while eliminating formatting that was repeated across every file read.

Shortening prompts while preserving behavior

Copilot’s task tool launches specialized agents for parallel work, but its guidance had accumulated across descriptions, schemas, agent definitions, system instructions, and companion tools. A meta-prompting process reduced the guidance by roughly half.

An initial online experiment exposed a problem that offline testing missed: cautious parallelism guidance had become a hard scheduling policy, causing independent custom agents to run sequentially. GitHub stopped the experiment, added a regression evaluation, and replaced explicit allowlists and denylists with the sentence: Independent agents can run in parallel; consider side effects.

The revised prompt removes about 1,300 task-tool prompt tokens per turn, equal to approximately 1.8% fewer total prompt tokens per session and 2.9% lower normalized cost per active hour. No quality regression was detected in the measured evaluations.

Three-stage diagram labeled Compression → Regression + fix → Completed. Left panel shows an original prompt compressed by about 50%. Middle panel highlights a regression where agents became serialized, then a fix by editing one sentence to restore parallelism. Right panel shows final shipped prompt with restored behavior and cumulative savings of about 1,300 fewer tokens per turn across steps. — Figure 5 Prompt compression became safe only after a regression test exposed serialized agents and a one-sentence fix restored parallelism; the resulting token savings recur on every model turn.

Delivering completed background work directly

Agents can run independent background tasks such as shell commands and sub-agent investigations. Previously, completion notifications told the agent that work was ready but did not include the result, forcing an additional retrieval turn. Copilot now batches eligible completion notifications and supplies completed results in the existing tool-result format.

For a shell command and sub-agent completing near the same time, the old flow required four model calls before work could continue: one call to request each result and another to process each. The updated harness can deliver both results in one model call. Explicit reads for work that is still running remain unchanged.

By eliminating retrieval detours without compressing or withholding results, the change reduced average token-related usage, measured in AI Credits, by about 2.3%.

Before-and-after sequence diagram comparing orchestration behavior.

Before: model waits on separate shell and sub-agent completions, causing retrieval detours and four LLM calls to process two results.
After: a harness batches related completions and emits synthetic tool events so background work continues while waiting; both results are processed together in a single LLM call.
The visual emphasizes reduced latency and fewer model round trips. — Figure 6 Before, each background completion could wake a retrieval-only model turn. After, the harness batches eligible completions and delivers completed results in the existing tool-result format.

Measuring changes in their actual workflows

GitHub found that an instruction change that worked well in Copilot code review increased cost in a Copilot CLI experiment and was not shipped. Separately, removing line-number prefixes and selectively compressing output each reduced average prompt tokens per review by roughly 5% in independent evaluations of a large set of production-model Copilot code review tasks, with no material change in tracked review-quality metrics.

These results are separate from the earlier migration of Copilot code review to shared file tools, which, combined with review-instruction tuning, reduced code review cost by about 20%.

Lessons for efficient AI coding agents

Optimize the completed task rather than an individual tool call.
Improve orchestration by eliminating model turns that the harness can complete deterministically.
Compress according to what output represents, favoring lossless transformations and monitoring recovery use.
Test prompt rewrites for unintended behavioral changes.
Re-evaluate every optimization across offline benchmarks, online experiments, and each product surface.

GitHub says these changes do not make the model smarter; they remove work the model does not need to perform. They are shipping across GitHub Copilot experiences that use the shared underlying harness.