Enforcing policies on agentic systems is getting harder. The input is no longer a single message. It is a long transcript or a full agent trace. The policy is no longer one rule. It is a bundle of conditions, exceptions and counts that must be assessed together. Fine-tuned guardrails degrade on these compound policies. Frontier LLM judges handle them better, but they are expensive and their verdicts flip between runs.
We present SCOPE (Symbolic Composition Over Predicates and Extractors), a framework that compiles a natural-language policy into an executable program. The program is built from narrow yes/no questions answered by weak classifiers, deterministic extractors for exact computation, and Python code that composes their outputs into a verdict. The program is optimized jointly with its own training data, using adversarial counterexamples that target where it departs from the policy.
Programs compiled with weak classifiers match frontier reasoning models, at a fraction of the cost, with far more stable predictions. Every verdict traces back to the predicate outputs that produced it.
SCOPE is available as part of Plurai SDK. Full details in the paper. The benchmark is on HuggingFace.
Motivation
Real policies are compound. A customer support agent must escalate only when the user re-raises the same unresolved intent three times, and a turn that adds new information does not count. A research agent’s plan is valid only if it references available tools, refers to them abstractly, and ends with a specific tag, somewhere inside a 45k-character trace. One missed clause produces a false violation or a missed obligation.
Existing approaches present a trade-off:
A fine-tuned small model is fast and accurate on one well-scoped rule. Its capacity is the ceiling. Once a policy bundles interacting conditions, a single forward pass cannot resolve them. A frontier judge handles the compound case, but its variance grows with the number of clauses, and it costs a frontier call on every gated interaction.
Neurosymbolic methods are the natural alternative. Narrow sub-decisions are more stable and can run on shallower models. Existing methods fail on two axes. Expressiveness: checklists, weighted rules and Boolean formulas cannot encode conditionals, iteration or counting. Bias: the symbolic layer is fit to a fixed dataset, a proxy for the policy, not to the policy itself.
SCOPE addresses both. The composition is code, so it can express what real policies contain. The training data is grown adversarially against the policy, so the program is fit to the specification rather than to the samples.
Method Overview
(1) Policy and seed samples. (2) The optimizer compiles them into (3) a program of weak-model predicates, deterministic extractors and composing code. (4) Adversaries add counterexamples where the program departs from the policy. The loop repeats.
SCOPE takes a policy and a few labeled seed samples. It searches for a program that reproduces the policy’s verdicts when every predicate is answered by a weak model.
The program. Three building blocks: predicates, self-contained yes/no questions answered by a small model; extractors, deterministic functions for spans, counts and thresholds; and composition code, Python that feeds each predicate the right slice and combines the labels into a verdict. Because the composition is code, it can loop, branch and count the way real policies do.
The weak proxy. During the search, every predicate is answered by a sub-frontier model with no reasoning and temperature 0. It returns only the label. No reasoning keeps each call cheap; temperature 0 keeps its answer stable across runs. If the proxy cannot answer a predicate reliably, the search rewords or splits it. Weaker proxies produce more, simpler predicates.
The optimizer. Every run produces a trace: each predicate call, its input and its label. The analyzer works backward from wrong verdicts to the predicate or code at fault, then proposes a fix grounded in a policy clause, never in a phrase seen in a sample. The search starts from the policy as a single predicate, so it falls back to prompt optimization when decomposition does not help.
The adversaries. A fixed pool rewards fitting the pool. After each revision, a systemic adversary targets clauses the composition misroutes, and a local adversary targets predicates that drift from their clause. A strong teacher labels each counterexample blind. It enters the pool only if both labels agree.
Strong models are paid for once, at compilation. The compiled program runs entirely on weak models.
Experimental Setup
We evaluate SCOPE on four policy-enforcement tasks drawn from real agentic systems. Each poses a distinct core challenge:
As part of this work, we release the benchmark with human-verified labels on HuggingFace.
We compare against three families of methods:
- Prompt optimization: GEPA and MIPROv2 (DSPy), which rewrite a single monolithic prompt.
- Task decomposition: CheckEval (checklist questions) and RLIE (rules combined by logistic regression). Both are binary only, so they run on Escalation and Plan.
- Workflow generation: AFlow, which searches over multi-call LLM programs. It is the only method whose intermediate instructions can elicit reasoning.
Results
Accuracy across executors
Across all three executors, SCOPE attains the highest accuracy, or ties for it, in 9 of 12 task-model cells. AFlow is second, with 3, despite being the only method allowed intermediate reasoning.
Decomposition compresses the capability gap
The gain over prompt optimization widens as the executor weakens. Moving from the weakest proxy to the strongest buys 0.29 accuracy on average under prompt optimization. Under SCOPE it buys 0.09. Once a policy is expressed as predicates a small model can answer, the choice of executor matters much less.
The program structure adapts to the executor. Weaker proxies yield more predicates, with the largest jump from gpt-5.4-mini to gpt-5.4-nano. When the executor is strong enough, the search can converge to a single predicate. On Coaching with gemini-3.5-flash it did, raising accuracy from 0.87 to 0.93.
Stability
Tile-match is the hardest stability test: every sample requires several independent decisions, and one flipped decision changes the output. Each SCOPE verdict also traces to explicit predicate outputs, not a post-hoc rationale.
Conclusion
SCOPE treats a policy as a specification to compile, not a prompt to interpret. The result is a program of narrow predicates, exact extractors and explicit code, optimized against counterexamples that target where it departs from the policy. Compiled once with a weak proxy, it runs on inexpensive models, matches frontier reasoning models, and produces verdicts that are stable and traceable.
SCOPE complements our previous BARRED work. BARRED turns a single policy into a fine-tuned classifier. SCOPE breaks a compound policy into predicates small models can answer.
