AI Writing Detection for Developer Text

Detect patterns characteristic of AI-generated text in developer artifacts. These patterns reduce trust, add noise, and obscure meaning.

Pattern Categories

Scope

Scan these artifact types:

What NOT to Scan

• Generated code (lock files, compiled output, vendor directories)
• Third-party content (copied license text, vendored docs)
• Code itself (variable names, string literals used programmatically)
• Test fixtures and mock data

Detection Rules

High-Confidence Signals (Always Flag)

These patterns are strong indicators of AI-generated text:

1. Chat leaks — “Certainly!”, “I’d be happy to”, “Great question!”, “Here’s” as sentence opener
2. Cutoff disclaimers — “As of my last update”, “I cannot guarantee”
3. High-signal AI vocabulary — delve, utilize (as “use”), whilst, harnessing, paradigm, synergy
4. “This noun verbs” in docstrings — “This function calculates”, “This method returns”
5. Synthetic openers — “In today’s fast-paced”, “In the world of”
6. Sycophantic code comments — “Excellent approach!”, “Great implementation!”

Medium-Confidence Signals (Flag in Context)

Flag when 2+ appear together or pattern is repeated:

1. Low-signal AI vocabulary clusters — 3+ words from the low-signal list in one section
2. Formulaic structure — Rigid intro-body-conclusion in a README section
3. Heading restatement — First sentence after heading restates the heading
4. Excessive hedging — “might potentially”, “could possibly”, “it seems like it may”
5. Synonym cycling — Same concept called different names within one section
6. Boldface overuse — More than 30% of sentences contain bold text

Low-Confidence Signals (Note Only)

Mention but don’t flag as issues:

1. Emoji in technical docs — May be intentional project style
2. Filler phrases — Some are common in human writing too
3. Generic conclusions — May be appropriate for summary sections
4. Commit inflation — Some teams prefer descriptive commits

False Positive Warnings

Do NOT flag these as AI-generated:

Integration

With beagle-core:review-verification-protocol

Before reporting any finding:

1. Read the surrounding context (full paragraph or function)
2. Confirm the pattern is AI-characteristic, not just formal writing
3. Check if the project has established conventions that match the pattern
4. Verify the suggestion improves clarity without changing meaning

With beagle-core:llm-artifacts-detection

Code-level patterns (tautological docstrings, obvious comments) overlap with llm-artifacts-detection‘s style criteria. When both skills are loaded:

• review-ai-writing focuses on writing style (how it reads)
• llm-artifacts-detection focuses on code artifacts (whether it should exist at all)
• If .beagle/llm-artifacts-review.json exists, skip findings already captured there

Output Format

Report each finding as:

[FILE:LINE] ISSUE_TITLE
- Category: content | vocabulary | formatting | communication | filler | code_docs
- Type: specific_pattern_name
- Original: "the problematic text"
- Suggestion: "the improved text" or "delete"
- Risk: Low | Medium
- Fix Safety: Safe | Needs review

Risk Levels

• Low — Filler phrases, obvious comments, emoji. Removing improves clarity with no meaning change.
• Medium — Vocabulary swaps, structural changes, docstring rewrites. Meaning could shift if done carelessly.

Fix Safety

• Safe — Mechanical replacement or deletion. No judgment needed.
• Needs review — Rewrite requires understanding context. Human should verify the replacement preserves intent.