Reviewing Guidelines

Contents

• Introduction
• Reviewing Timeline
• Reviewing in a Nutshell
• Ethics for Reviewing Papers
• How to Write Good Reviews

Introduction

These reviewing guidelines are inspired by the best practice from CVPR 2024 - for further details, please see the CVPR Reviewer Guidlines.

Thank you for volunteering your time to review for BMVC 2024! To maintain a high-quality technical program, we rely very much on the time and expertise of our reviewers. This document explains what is expected of all members of the Reviewing Committee for BMVC 2024.

BMVC will implement a scoring system this year, where Area Chairs will rate each review submitted. Reviewers that fall below a quality threshold will not be acknowledged as being part of the Reviewing Committee for BMVC 2024. It is, therefore, important to familiarise yourself with these guidelines to ensure that you are providing a quality review that the research community would benefit from.

To contact the Programme Chairs, please send emails to: programme-chairs@bmvc2024.org

Reviewing Timeline

• 13th-15th of May: Check your papers assigned to you
• 20th of May: Review period starts
• 3rd of June: Reviews due
• 14th-21st of June: ACs and reviewers discussion period
• 21st of June: Final review recommendation due

Reviewing In a Nutshell

Each paper that is accepted should be technically sound and make a contribution to the field. Look for what is good or stimulating in the paper and what knowledge advancement it has made. We recommend that you embrace novel, brave concepts, even if they have not been tested on many datasets. For example, the fact that a proposed method does not exceed the state-of-the-art accuracy on an existing benchmark dataset is not grounds for rejection by itself. Rather, it is important to weigh both the novelty and potential impact of the work alongside the reported performance. Minor flaws should not be a reason to reject a paper. Above all, you should be specific and detailed in your reviews. Your discussion, more than your score, will help the authors, fellow reviewers, and Area Chairs understand the basis for your recommendation. You should include specific feedback on ways the authors can improve their papers.

It is worth noting that there will be no rebuttal period for BMVC 2024. Therefore reviews should not ask for revisions and/or additional experiments but only evaluate the submitted paper as it stands.

Check your papers (13th-15th of May)

As soon as you get your reviewing assignment, please go through all the papers to make sure that (a) you have no obvious conflict of interest (see “Avoid Conflicts of Interest” below) and (b) you feel comfortable reviewing the paper assigned. If issues with either of these points arise, please get in touch with the Area Chair immediately as instructed in the detailed emails you will receive during the process.

Know the policies

Please read the Author Guidelines carefully to familiarise yourself with all official policies the authors are expected to follow. If you believe that a paper may violate any of these policies, please get in touch with the Area Chairs or Programme Chairs. In the meantime, proceed to review the paper, assuming no violation has taken place.

Ethics for Reviewing Papers

Respect anonymity in the review process

Our Author Guidelines have instructed authors to make reasonable efforts to hide their identities, including omitting their names, affiliations, and acknowledgements. This information will, of course, be included in the final published version of the manuscript. Reviewers should not actively discover the authors’ identity and make all efforts to keep their own identity invisible to the authors.

With the increase in popularity of arXiv preprints, sometimes the authors of a paper may already be known to the reviewer. Posting to arXiv is NOT considered a violation of anonymity on the part of the authors, and in most cases, reviewers who happen to know (or suspect) the authors’ identity can still review the paper as long as they feel that they can do an impartial job. An important general principle is to make every effort to treat papers fairly whether or not you know (or suspect) who wrote them. If you do not know the identity of the authors at the start of the process, DO NOT attempt to find it out by searching the Web for preprints.

Protect Ideas

As a reviewer for BMVC, you are responsible for protecting the confidentiality of the ideas presented in the papers you review. BMVC submissions are not published documents. The work is considered new or proprietary by the authors; otherwise, they would not have submitted it. Of course, their intent is to publish to the world ultimately, but most of the submitted papers will not appear in the BMVC proceedings. Thus, it is likely that the paper you have in your hands will be refined further and submitted to some other journal or conference. Sometimes the work is still considered confidential by the authors’ employers. These organisations do not consider sending a paper to BMVC for review to constitute a public disclosure. Protection of the ideas in the papers you receive means:

• You should not show the paper to anyone else, including colleagues or students, unless you have asked them to write a review or to help with your review.
• You should not show any results, videos/images, code, or supplementary material to non-reviewers.
• You should not use ideas/code from papers you review to develop your own ideas/code.
• After the review process, you should destroy all copies of papers and supplementary material and erase any code that the authors submitted as part of the supplementary and any implementations you have written to evaluate the ideas in the papers, as well as any results of those implementations.

Avoid Conflicts of Interest

As a reviewer of a BMVC paper, it is important for you to avoid any conflict of interest. There should be no question about the impartiality of any review. Thus, if you are assigned a paper where your review would create a possible conflict of interest, you should return the paper and not submit a review. Conflicts of interest include (but are not limited to) situations in which:

• You work at the same institution as one of the authors.
• You have been directly involved in the work and will be receiving credit in some way. If you're a member of an author's thesis committee, and the paper is about their thesis work, then you were involved.
• You suspect that others might perceive a conflict of interest in your involvement.
• You have collaborated with one of the authors in the past three years (more or less). Collaboration is usually defined as having written a paper or grant proposal together, although you should use your judgement.
• You were the MSc/PhD advisor or advisee of one of the authors. Most funding agencies and publications typically consider advisees representing a lifetime conflict of interest. BMVC has traditionally been more flexible than this, but you should think carefully before reviewing a paper you know to be written by a former advisor or advisee, especially a recent one.
• While the organisers make every effort to avoid such conflicts in the review assignments, they may nonetheless occasionally arise. If you recognise the work or the author and feel it could present a conflict of interest, contact the Area Chair as soon as possible so they can find someone else to review it.

Be Professional

Belittling or sarcastic comments have no place in the reviewing process. The most valuable comments in a review help the authors understand their work’s shortcomings and how they might improve it. Write a courteous, informative, incisive, and helpful review that you would be proud to sign with your name (were it not anonymous).

How to Write Good Reviews

• Take the time to write good reviews. Ideally, you should read a paper and then think about it over the course of several days before you write your review.
• Short reviews are unhelpful to authors, other reviewers, and Area Chairs. If you have agreed to review a paper, you should take enough time to write a thoughtful and detailed review. Bullet lists with one short sentence per bullet are NOT a detailed review.
• Be specific when you suggest that the writing needs to be improved. If a particular section is unclear, point it out and suggest how it can be clarified.
• Be specific about novelty. Claims in a review that the submitted work “has been done before” MUST be backed up with specific references and an explanation of how closely they are related. At the same time, for a positive review, be sure to summarise what novel aspects are most interesting in the Strengths section.
• Do not reject papers solely because they are missing citations or comparisons to prior work that has only been published without review (e.g., arXiv or technical reports).
• Do not give away your identity by asking the authors to cite several of your own papers.
• If you think the paper is out of scope for BMVC's subject areas, clearly explain why in the review. Then suggest other publication possibilities (journals, conferences, workshops) that would be a better match for the paper. However, unless the area mismatch is extreme, you should keep an open mind because we want a diverse set of good papers at the conference.
• The tone of your review is important. A harshly written review will be resented by the authors, regardless of whether your criticisms are true. If you take care, it is always possible to word your review constructively while staying true to your thoughts about the paper.
• Avoid referring to the authors in the second person (“you”). It is also best to avoid the term “the authors” because you are reviewing their work and not the person. Instead, use the third person (“the paper”). Referring to the authors as “you” can be perceived as being confrontational, even though you may not mean it this way.
• Be generous about giving the authors new ideas for how they can improve their work. You might suggest a new technical tool that could help, a dataset that could be tried, an application area that might benefit from their work, or a way to generalise their idea to increase its impact.

Finally, keep in mind that a thoughtful review benefits not only the authors but also yourself. Your reviews are read by other reviewers, especially the Area Chairs, in addition to the authors. Unlike the authors, the Area Chairs and Programme Committee know your identity. Being a helpful reviewer will generate goodwill towards you in the research community—and may even help you win an Outstanding Reviewer award.