Docs chevron_right Accuracy Transparency — duplication percentages and known issues

Accuracy Transparency

Deslop's percentage is exact arithmetic over the findings in the current report. It is not a statistical estimate, and it is not a claim that detection has perfect precision or recall.

It is a raw coverage figure: it measures how much of the codebase the detector currently flags, taking every surviving finding at face value. Nothing in it is discounted for how strong the evidence behind a finding is.

How the percentage is calculated

duplication_percent = clamp(100 × duplicated_loc / analysed_loc, 0, 100)
  • analysed_loc is the number of physical lines in every analysed source file. Blank and comment lines count; an empty file contributes zero. Files excluded before analysis contribute nothing.
  • duplicated_loc is the per-file union of physical line numbers touched by non-hidden occurrences in clusters that survive the report filters. A line covered by overlapping occurrences or clusters is counted once.
  • report_hide and generated-header occurrences do not enter the numerator. Beyond that, no finding is discounted: rank, size, bucket and confidence all carry equal weight. A line whose only evidence is a matching code shape — the kind the report itself labels "verify before extracting" — counts exactly like a line in a copy proven identical byte for byte. That is what keeps the number reproducible, and it is also why it can read higher than the duplication you would actually act on (#344, #355).
  • Per-file percentages use the same calculation. Folder percentages sum the files' duplicated and analysed line counts, then divide; they are never averages of file percentages.
  • A zero-line corpus reports 0%. JSON carries the full floating-point value; human-facing reports round it for display.

The implementation is public: render_report selects the visible cluster set, compute_repo_metrics unions the covered lines, and percent performs the division and clamp.

What we are building next

A second, evidence-weighted percentage: the same line sets, with each line priced by the strength of the evidence behind the finding that covers it, so a proven copy weighs more than a shape-only resemblance. It gets its own opt-in CI gate and reports beside the raw figure — which keeps its exact meaning and stays the default, so no existing threshold shifts under you. The design, the weights, and the reasoning behind them are in weighted-metrics-plan.md, tracked in #344.

How the CI gate works

Set --fail-over <percent> or [threshold] max_duplication_percent in .deslop.toml. The CLI flag overrides the config value; --no-fail-over disables the gate for that run.

The gate fails only when the full-precision measured value is greater than the ceiling. Equality passes. A breach writes the reports, then exits 3; without a threshold, duplication alone never fails the run. Thresholds must be finite values from 0 to 100.

The exact comparison is in ThresholdSummary::resolve, CLI precedence and exit handling are in main.rs, and the GitHub Action preserves the reports before re-raising the exit code.

Known open accuracy risks

Reviewed 13 August 2026. This table covers every open issue that can currently change or misstate a finding, signal, bucket, rank, percentage, CI verdict, or cross-surface result. Some are limited to a language, optional embeddings, a release version or a particular configuration.

Issues Possible effect
#359 Several engine and VS Code defects can promote unrelated members, suppress real logic, lose a TypeScript clone, misclassify Python assertions, or show a stronger verdict than the evidence supports.
#358, #356, #351 With embeddings enabled, real Python matches can be suppressed, ANN bridges can erase or relabel structural findings, and measured cosine evidence can be discarded.
#357 Duplicate subtrees are all indexed by ANN. This is primarily a scale defect, but its repair must preserve every original pair or it can create false negatives.
#355 A Dart family of one-statement delegating methods can surface as duplication and inflate duplicated_loc and duplication_percent.
#344, #343 The percentage gives low- and high-confidence visible lines equal weight, while fused confidence can saturate at 1.0; gates and rankings can therefore overstate weak shape matches.
#342 A repository beneath an ancestor named dist, build, target or another built-in exclude can be scanned as zero files and falsely pass clean.
#339, #336, #286 F# token evidence can depend on byte ranges, data tables can dominate the report, and failed embeddings can leave a recall blind spot.
#301 Identical input can produce different cluster sets and percentages between runs, making a gate near its ceiling flaky.
#298, #292 Generated out/, coverage, VS Code test, or Playwright report assets can enter the corpus and inflate or destabilise findings and percentages.
#285, #284, #283 Unrelated TypeScript test scenarios and object-literal tables can be promoted to high-confidence nearly identical code (Type-3).
#103, #79, #71 Python test idioms, already-extracted helper calls, and independent HTTP endpoint tests can be reported as actionable duplication.
#309, #264, #263, #262 find-similar can miss unique or tracked code and may reject or omit JavaScript/TypeScript, weakening the pre-write duplicate gate.
#276, #228 Different or stale CLI, LSP, MCP and VS Code results can show conflicting clusters, percentages, rankings, and pass/fail verdicts.
#167 Experimental Dart declarative constructors produce parser error nodes, creating a narrow potential detection gap.
#345 Documentation and code disagree about fused-score admission, ranking order and the embeddings default, which can mislead interpretation of otherwise real figures.
#347 The scheduled real-repository accuracy gate fails before scanning, so accuracy regressions can escape that assurance layer.

We are actively working to fix every accuracy issue. Until an issue is closed with regression coverage, treat the percentage as an exact measurement of Deslop's current visible findings—not as ground truth about every duplicate in the codebase.