Orchestration Log: When this skill is activated, append a log entry to
outputs/orchestration_log.md:### Skill Activation: Review Engine (Round [N]) **Timestamp:** [current date/time] **Actor:** AI Agent (review-engine) **Input:** [N] review points from [source] (annotated PDF / pasted text / reviewer report) **Output:** [N] changes implemented, paper recompiled, latexdiff generated **Human decisions:** [list of QUESTION items resolved by orchestrator]
Review Engine
Core Principle
Academic revision is a structured, repeatable process: extract feedback -> interpret -> classify -> implement -> verify. This engine automates the entire loop. The human orchestrator approves the change plan; the engine executes it.
This skill was derived from 4 actual co-author revision rounds on a real paper (Blask & Funk, 2026). Every step reflects what we learned works — and what fails — when processing human feedback on AI-generated manuscripts.
When to Activate
- User provides an annotated PDF (co-author or reviewer comments)
- User pastes reviewer comments from a journal decision letter
- User says "implement these changes", "process this review", "revision round"
- User says "respond to reviewers", "R&R", "revise and resubmit"
- After self-review (Phase 6) produces a critique that needs implementation
Prerequisites
paper.texexists inlatex/and compiles successfully (this is the baseline)pdflatexandbibtexare availablelatexdiffis available (for visual change tracking)- PyMuPDF (
fitz) is available for PDF annotation extraction (pip install pymupdf)
Step 1: EXTRACT Review Points
The first step is to get all review points into a structured format, regardless of how the reviewer provided them.
From Annotated PDF (Co-Author Review)
This is the most common case: a co-author annotates the PDF with highlights, sticky
notes, and text comments. Use scripts/extract_annotations.py:
import sys
sys.path.insert(0, "scripts")
from extract_annotations import extract_annotations, annotations_to_markdown
annotations = extract_annotations("path/to/annotated.pdf")
print(annotations_to_markdown(annotations))
print(f"\nTotal: {len(annotations)} annotations found")
If scripts/extract_annotations.py is not available, use inline PyMuPDF:
import fitz
doc = fitz.open("path/to/annotated.pdf")
annotations = []
for page_num, page in enumerate(doc, 1):
for annot in page.annots() or []:
entry = {
"page": page_num,
"type": annot.type[1], # "Highlight", "Text", "FreeText", "StrikeOut"
"content": annot.info.get("content", "").strip(),
"author": annot.info.get("title", ""),
"highlighted_text": "",
}
# Extract highlighted/marked text via quadpoints
if annot.type[0] in (8, 9, 10, 11) and annot.vertices: # Highlight, Underline, Squiggly, StrikeOut
quad_count = len(annot.vertices) // 4
text_parts = []
for i in range(quad_count):
quad = annot.vertices[i * 4:(i + 1) * 4]
rect = fitz.Rect(
min(p[0] for p in quad), min(p[1] for p in quad),
max(p[0] for p in quad), max(p[1] for p in quad),
)
text_parts.append(page.get_text("text", clip=rect).strip())
entry["highlighted_text"] = " ".join(text_parts)
if entry["content"] or entry["highlighted_text"]:
annotations.append(entry)
doc.close()
Common pitfall: The user may send the wrong PDF (without annotations). If extraction returns 0 annotations, tell the user immediately and ask them to re-send. Do NOT proceed with an empty annotation list.
From Pasted Text (Journal R&R)
Parse reviewer comments by detecting common patterns:
Reviewer 1:
Major Comments:
1. The authors should clarify...
2. The methodology section lacks...
Minor Comments:
1. On page 5, the reference to...
Split into individual review points by:
- Numbered items ("1.", "2.", "-", "a)")
- Reviewer sections ("Reviewer 1:", "Reviewer 2:")
- Category headers ("Major Comments:", "Minor Comments:", "Questions:")
- Page/line references ("page 5", "line 42", "Section 3")
From Self-Review (Phase 6 Output)
Parse the structured self-critique from self_review.md or the Phase 6 output.
Each bullet point or numbered item becomes a review point.
Output Format
Every extraction method produces the same structure:
{
"id": 1,
"source": "pdf_annotation" | "pasted_text" | "self_review",
"page": 5, # PDF page (if applicable)
"type": "Highlight", # Annotation type (if PDF)
"reviewer": "Burkhardt Funk", # Reviewer name (if known)
"reviewer_text": "just delete, I think ref to tab 1 is also wrong here",
"highlighted_text": "Following Hevner et al.'s guidelines, we evaluate in three steps",
"section_ref": null # Will be filled in Step 2
}
Step 2: MAP to Source
For each review point, locate the corresponding position in paper.tex.
Mapping Strategy
-
Primary: Text match. Take the
highlighted_text(or key phrases fromreviewer_text) and searchpaper.texusing grep/search. Handle:- LaTeX commands embedded in text (
\textit{...},\citep{...}) - Line breaks (the PDF renders continuous text that spans multiple LaTeX lines)
- Ligatures and special characters (fi, fl, -- vs. --)
- LaTeX commands embedded in text (
-
Secondary: Page-to-section estimation. If no text match is found, use the PDF page number to estimate which LaTeX section it corresponds to. Compile the paper, check page breaks, and narrow down the section.
-
Tertiary: Manual flag. If neither works, flag the review point with
mapping_confidence: lowand present it to the user for manual location.
Identifying Section Context
For each matched location, also extract:
- The
\section{}or\subsection{}it belongs to - The line number in
paper.tex - 3 lines of surrounding context (for the change plan display)
Output
Each review point is enriched:
{
...,
"tex_line": 256,
"tex_section": "\\subsection{Pipeline Implementation}",
"tex_context": "Following Hevner et al.'s guidelines, we evaluate in three steps...",
"mapping_confidence": "high" | "medium" | "low"
}
Step 3: CLASSIFY Actions
For each review point, determine what the reviewer is asking for and how to respond.
Action Types
| Type | Description | Automation | Example |
|---|---|---|---|
DELETE | Remove text entirely | Full auto | "just delete" |
REPLACE | Change specific text or terminology | Full auto | "systems -> agents" |
MOVE | Relocate content to different section | Guided auto | "belongs in conclusion" |
RESTRUCTURE | Reorganize section flow/argument | Needs plan approval | "Better: briefly describe each agent" |
FIX | Correct references, typos, formatting | Full auto | "is it 7.1?" |
FIGURE | Modify or regenerate a figure | Invoke figure-engine | "extend orchestrator zone" |
SHORTEN | Reduce or condense text | Guided auto | "zu kleinteilig" |
EXPAND | Add more detail or explanation | Invoke writing-engine | "elaborate on methodology" |
APPROVE | No action needed (positive feedback) | Log only | "fine with phases" |
QUESTION | Ambiguous -- needs human decision | Block + ask | "do we do that?" |
Priority Levels
- CRITICAL: Factual error, wrong reference, misattribution, missing content
- MAJOR: Structural change, unclear argument, missing justification
- MINOR: Wording improvements, terminology, formatting
- COSMETIC: Typos, spacing, style preferences
Classification Process
For each review point:
- Read the
reviewer_textin context ofhighlighted_text - Determine intent (what does the reviewer want changed?)
- Assign
action_typeand `p