Reviewer Guidelines

This comprehensive guide helps reviewers provide high-quality, constructive reviews for conference submissions through PaperFox.

Overview

As a reviewer, you play a crucial role in maintaining the quality and standards of academic conferences. Your expert evaluation helps authors improve their work and assists conference organizers in making informed decisions.

Your Responsibilities

• Provide fair and thorough evaluations
• Offer constructive feedback for improvement
• Maintain confidentiality and ethical standards
• Meet review deadlines consistently
• Participate in discussion phases when required

Getting Started as a Reviewer

Accepting Review Invitations

Initial Invitation:

• Review the invitation email carefully
• Check conference scope and your expertise match
• Verify you have time to complete reviews by deadline
• Consider potential conflicts of interest

Accepting the Invitation:

1. Click the link in your invitation email
2. Create an account if you don't have one
3. Review the conference details and expectations
4. Accept or decline the invitation promptly

Setting Up Your Reviewer Profile

• Expertise areas: List your research specializations
• Conflict of interest information: Declare potential conflicts
• Availability: Confirm your availability for the review period
• Contact preferences: Set notification preferences

Understanding the Review Process

Types of Review Processes

Single-Blind Review:

• You know author identities
• Authors don't know reviewer identities
• More context about authors' previous work
• Potential for unconscious bias

Double-Blind Review:

• Both reviewer and author identities hidden
• Focus purely on content quality
• Reduced bias but less context
• Standard for most academic conferences

Open Review:

• All identities visible
• Transparent and accountable process
• Less common but growing trend
• Promotes constructive dialogue

Review Timeline

Typical review process stages:

1. Assignment: Papers assigned to reviewers (1-2 weeks)
2. Review Period: Complete reviews (4-8 weeks)
3. Discussion Phase: Reviewer discussion (1-2 weeks)
4. Final Decisions: Track chairs make decisions

Review Criteria and Standards

Key Evaluation Areas

Technical Quality:

• Methodology: Is the research approach sound?
• Execution: Are experiments/analyses properly conducted?
• Validity: Are conclusions supported by evidence?
• Reproducibility: Can results be replicated?

Novelty and Significance:

• Originality: What is new or different?
• Contribution: How does this advance the field?
• Impact: What is the potential influence?
• Innovation: Are approaches creative or insightful?

Clarity and Presentation:

• Writing quality: Is the paper well-written?
• Organization: Is the structure logical?
• Figures/tables: Are visuals clear and informative?
• Understanding: Can experts follow the work?

Relevance and Scope:

• Conference fit: Does this match conference themes?
• Audience interest: Will attendees find this valuable?
• Timeliness: Is this current and relevant?
• Completeness: Is the work sufficiently developed?

Rating Scales

Common scoring systems:

• Numerical scales: 1-5 or 1-10 ratings
• Categorical ratings: Accept/Minor Revisions/Major Revisions/Reject
• Confidence levels: How certain are you of your evaluation?
• Overall recommendation: Final accept/reject recommendation

Writing Effective Reviews

Review Structure

Summary Section:

• Brief summary of the paper's main contributions
• Demonstrates you understood the work
• Helps authors see how their work is perceived
• Usually 2-3 sentences

Strengths Section:

• Highlight what the paper does well
• Acknowledge good methodology, writing, or insights
• Recognize significant contributions
• Be specific and constructive

Weaknesses Section:

• Identify areas needing improvement
• Focus on fixable issues when possible
• Be specific about problems
• Suggest potential solutions

Detailed Comments:

• Page/line specific feedback
• Technical corrections
• Clarification requests
• Improvement suggestions

Minor Issues:

• Typos and grammatical errors
• Reference formatting
• Figure/table improvements
• Small technical corrections

Writing Best Practices

Be Constructive:

• Focus on improving the work
• Suggest specific improvements
• Acknowledge effort and intent
• Maintain respectful tone

Be Specific:

• Reference specific sections, figures, or lines
• Provide concrete examples
• Explain reasoning behind criticisms
• Offer actionable feedback

Be Balanced:

• Include both strengths and weaknesses
• Avoid purely negative reviews
• Recognize good aspects even in rejected papers
• Maintain objectivity

Be Professional:

• Use respectful language throughout
• Avoid personal attacks or dismissive comments
• Focus on the work, not the authors
• Maintain academic standards

🔬 Technical Review Guidelines

Methodology Assessment

Experimental Design:

• Are controls appropriate?
• Is sample size adequate?
• Are variables properly controlled?
• Is the approach suitable for the research questions?

Data Analysis:

• Are statistical methods appropriate?
• Are assumptions met?
• Is analysis thorough and correct?
• Are results interpreted properly?

Reproducibility:

• Is methodology described clearly enough to replicate?
• Are datasets and tools available or described?
• Can results be verified independently?
• Are limitations acknowledged?

Literature Review Evaluation

• Completeness: Are key references included?
• Currency: Is recent work cited appropriately?
• Context: Is prior work properly positioned?
• Gaps: Are research gaps clearly identified?

Results and Discussion

• Clarity: Are results presented clearly?
• Completeness: Are all important results included?
• Interpretation: Are conclusions supported by data?
• Limitations: Are weaknesses acknowledged?

Ethical Considerations

Confidentiality

• Never share submitted papers with others
• Don't discuss submissions publicly or privately
• Protect author anonymity (in double-blind review)
• Secure submitted materials appropriately

Conflicts of Interest

Declare conflicts if you:

• Collaborated with authors recently
• Have personal relationships with authors
• Work at the same institution
• Have financial interests in the work
• Cannot review objectively

Fair and Unbiased Review

• Evaluate work on merit alone
• Avoid discrimination based on author characteristics
• Consider diverse perspectives and approaches
• Acknowledge your own limitations and biases

Intellectual Property

• Don't use ideas from submitted papers in your own work
• Don't share innovative approaches with others
• Respect authors' intellectual contributions
• Maintain embargo until publication

⏰ Time Management and Deadlines

Planning Your Review Time

• Start early: Don't wait until deadline
• Allocate adequate time: Usually 4-8 hours per paper
• Multiple readings: Plan for several review passes
• Buffer time: Allow for unexpected issues

Review Process Timeline

First Reading (30-45 minutes):

• Get overall understanding
• Identify major strengths/weaknesses
• Note initial impressions

Detailed Review (2-3 hours):

• Thorough evaluation of all sections
• Technical assessment
• Note specific comments

Writing Review (1-2 hours):

• Structure your feedback
• Write clear, constructive comments
• Review for tone and completeness

Final Review (30 minutes):

• Proofread your review
• Ensure consistency
• Check all required sections completed

Meeting Deadlines

• Calendar reminders: Set multiple alerts
• Progress tracking: Monitor your review progress
• Early submission: Submit before deadline when possible
• Communication: Contact chairs if delays unavoidable

Discussion Phase Participation

When Discussions Occur

• Conflicting reviews: When reviewers disagree significantly
• Borderline papers: For papers near acceptance threshold
• Technical questions: When expertise varies among reviewers
• Policy issues: For papers raising special considerations

Effective Discussion Participation

• Stay professional: Maintain respectful dialogue
• Be open to other perspectives: Consider different viewpoints
• Provide evidence: Support arguments with specific examples
• Focus on the work: Keep discussions about the paper
• Reach consensus: Work toward fair resolution

Discussion Etiquette

• Read other reviews carefully before responding
• Acknowledge valid points from other reviewers
• Explain your reasoning clearly
• Be willing to adjust your position when appropriate
• Keep discussions focused and productive

Common Review Challenges

Handling Difficult Papers

Papers Outside Your Expertise:

• Acknowledge limitations in your review
• Focus on areas you can evaluate
• Be honest about your confidence level
• Suggest alternative reviewers if possible

Poorly Written Papers:

• Distinguish between language and content issues
• Provide specific examples of unclear writing
• Suggest improvements where possible
• Consider if content has merit despite presentation

Borderline Papers:

• Weigh strengths and weaknesses carefully
• Consider conference standards and expectations
• Be decisive in your recommendation
• Explain reasoning thoroughly

Managing Review Load

• Accept appropriate number of reviews
• Balance quality and quantity
• Decline when overcommitted
• Communicate limitations to organizers

Becoming an Excellent Reviewer

Developing Review Skills

• Learn from feedback: When available, see how your reviews are received
• Read exemplary reviews: Learn from experienced reviewers
• Attend workshops: Participate in reviewer training when offered
• Practice regularly: Skills improve with experience

Contributing to the Community

• Mentor new reviewers: Share knowledge with junior colleagues
• Provide reviewer training: Offer workshops in your community
• Serve as meta-reviewer: Take on additional responsibilities
• Advocate for good practices: Promote review quality standards

Recognition and Career Benefits

• Professional development: Reviewing enhances your expertise
• Network building: Connect with researchers in your field
• Early access: See cutting-edge work before publication
• Service recognition: Important for academic career advancement

---

Ready to begin reviewing? Check your reviewer dashboard for assigned papers.