Scientific Peer Review

Rigorously evaluate scientific work for quality, validity, and reproducibility.

When to Use

• Self-reviewing manuscript before submission (REVIEW phase)
• Evaluating methodology and experimental design
• Checking statistical analyses and reporting
• Assessing reproducibility and data availability
• Reviewing others’ manuscripts for journals
• Evaluating grant proposals
• Quality checking your own work during ANALYSIS phase

Review Workflow

1. INITIAL SCAN      → Overall impression, scope, significance
2. SECTION REVIEW    → Detailed evaluation of each section
3. METHODOLOGY       → Rigor, assumptions, controls
4. STATISTICS        → Appropriate tests, effect sizes, reporting
5. REPRODUCIBILITY   → Data, code, materials availability
6. FIGURES/TABLES    → Clarity, integrity, accessibility
7. ETHICS            → Approvals, consent, conflicts
8. WRITING           → Clarity, organization, accuracy
9. SYNTHESIZE        → Major/minor issues, recommendation

Stage 1: Initial Assessment

Quick Questions (5 minutes)

1. What is the central research question?
2. What are the main findings?
3. Is the work scientifically sound?
4. Are there any immediate major flaws?
5. Is it appropriate for the intended venue?

Initial Summary Template

## Initial Assessment

**Research Question**: [One sentence summary]

**Main Findings**: [2-3 key results]

**Initial Impression**: [Sound/Concerning/Major issues]

**Significance**: [Novel contribution to field?]

Stage 2: Section-by-Section Review

Abstract & Title

Introduction

Methods

Critical Details to Verify:

• Sample sizes and power calculations
• Randomization and blinding
• Inclusion/exclusion criteria
• Software versions
• Statistical tests and corrections

Results

Common Issues:

• Selective reporting
• Inappropriate statistical tests
• Missing error bars
• Over-fitting
• Batch effects or confounders
• Missing controls

Discussion

Red Flags:

• Overstated conclusions
• Ignoring contradictory evidence
• Causal claims from correlational data
• Mechanistic claims without evidence

Stage 3: Methodological Rigor

Statistical Assessment

Experimental Design

Stage 4: Reproducibility Assessment

Data Availability

Code and Materials

Reporting Standards

Check adherence to discipline-specific guidelines:

Stage 5: Figure and Table Review

Quality Checks

Integrity Checks

Stage 6: Writing Quality

Structure and Organization

Writing Quality

Structuring the Review Report

Summary Statement (1-2 paragraphs)

## Summary

[Brief synopsis of the research]

**Recommendation**: [Accept / Minor revisions / Major revisions / Reject]

**Key Strengths**:
1. [Strength 1]
2. [Strength 2]
3. [Strength 3]

**Key Weaknesses**:
1. [Weakness 1]
2. [Weakness 2]

**Bottom Line**: [Overall assessment of significance and soundness]

Major Comments

Issues that significantly impact validity or interpretability:

## Major Comments

1. **[Issue Title]**
   - *Problem*: [Clear statement of the issue]
   - *Why it matters*: [Impact on conclusions]
   - *Suggestion*: [How to address it]

2. **[Issue Title]**
   ...

Major issues typically include:

• Fundamental methodological flaws
• Inappropriate statistical analyses
• Unsupported conclusions
• Missing critical controls
• Reproducibility concerns

Minor Comments

Less critical issues that would improve the manuscript:

## Minor Comments

1. [Page/Figure X]: [Issue and suggestion]
2. [Methods section]: [Missing detail]
3. [Figure 2]: [Clarity improvement]

Review Tone Guidelines

Do ✓

• Be constructive and specific
• Acknowledge strengths
• Provide actionable suggestions
• Focus on the science
• Be thorough but proportionate

Don’t ✗

• Use dismissive language
• Make personal attacks
• Be vague or sarcastic
• Request unnecessary experiments
• Impose personal preferences as requirements

Self-Review Checklist (Before Submission)

Use this during your REVIEW phase:

Methodology

• Methods are reproducible
• Controls are appropriate and documented
• Statistical methods are justified
• Sample sizes are adequate

Results

• All results support conclusions
• Effect sizes are reported
• Negative results are included
• Figures are clear and accessible

Reproducibility

• Data will be available
• Code is documented and available
• Protocols are detailed
• Reporting guidelines followed

Writing

• Abstract accurately summarizes the work
• Conclusions are supported by data
• Limitations are acknowledged
• References are current and complete

Integration with RA Workflow

REVIEW Phase Activities

1. Run self-review using this checklist
2. Document issues in tasks.md
3. Address each issue systematically
4. Re-review until checklist passes
5. Update .research/logs/activity.md

Pre-Submission Verification

Before calling a manuscript complete:

• Self-review completed
• All major issues addressed
• Figures meet journal requirements
• Data/code deposited
• Reporting checklist complete
• Cover letter prepared