Call for Papers

Aims and Scope

The 21st annual ACM Conference on International Computing Education Research (ICER) aims to gather high-quality contributions to the Computing Education Research discipline. The “Research Papers” track invites submissions describing original research results related to any aspect of teaching and learning computing, from introductory through advanced material. Submissions are welcome from across the research methods used in Computing Education Research and related fields. Each contribution will be assessed based on:

• the appropriateness and soundness of its methods
• its relevance to teaching or learning computing, and
• the depth of its contribution to the community’s understanding of the question at hand.

Research areas of particular interest include:

• design-based research, learner-centered design, and evaluation of educational technology supporting computing knowledge or skills development,
• discipline based education research (DBER) about computing, computer science, and related disciplines,
• informal learning experiences related to programming and software development (all ages), ranging from after-school programs for children, to end-user development communities, to workplace training of computing professionals,
• learnability of programming languages and tools for learning programming and computing concepts,
• learning analytics and educational data mining in computing education contexts,
• learning sciences work in the computing content domain,
• measurement instrument development and validation (e.g., concept inventories, attitudes scales, etc) for use in computing disciplines,
• pedagogical environments fostering computational thinking,
• psychology of programming,
• rigorous replication of empirical work, relevant to computing education, to compare with or extend previous empirical research results,
• professional development for computing educators at all levels.

While this above list is non-exclusive, authors are also invited to consider the call for papers for the “Lightning Talks & Posters” and “Work-in-Progress” tracks if in doubt about the suitability of their work for this track.

Submission instructions will be made available soon.

By submitting your article to an ACM Publication, you are hereby acknowledging that you and your co-authors are subject to all ACM Publications Policies, including ACM’s new Publications Policy on Research Involving Human Participants and Subjects. Alleged violations of this policy or any ACM Publications Policy will be investigated by ACM and may result in a full retraction of your paper, in addition to other potential penalties, as per ACM Publications Policy.

Please ensure that you and your co-authors obtain an ORCID ID, so you can complete the publishing process for your accepted paper. ACM has been involved in ORCID from the start and we have recently made a commitment to collect ORCID IDs from all of our published authors. We are committed to improve author discoverability, ensure proper attribution and contribute to ongoing community efforts around name normalization; your ORCID ID will help in these efforts.

All questions about this call should go to the ICER 2025 program committee chairs at pc-chairs@icer.acm.org.

Important Dates

All submission deadlines are “anywhere on Earth” (AoE, UTC-12).

What	When
Titles, abstracts, and authors due. (The chairs will use this information to assign papers to PC members.)	Friday, March 14th, 2025
Full paper submission deadline	Friday, March 21st, 2025
Decisions announced	Thursday, May 15th, 2025
“Conditional Accept” revisions due	Thursday, May 29th, 2025
“Conditional Accept” revisions approval notification	Thursday, June 5th, 2025
Final versions due to TAPS	Wednesday, June 11th, 2025
ICER 2025 Conference	Sunday, August 3rd through Wednesday, August 6th 2025
Published in the ACM Digital Library	The official publication date is the date the proceedings are made available in the ACM Digital Library. This date will be the first day of the conference. The official publication date may affect the deadline for any patent filings related to published work.

Submission Process

Submit at the ICER 2025 HotCRP site.

When you submit the abstract or full version ready for review, you need to perform the following actions:

• Check the checkbox “ready for review” at the bottom of the submission form. (Otherwise it will be marked as a draft).

• Check the checkbox “I have read and understood the ACM Publications Policy on Research Involving Human Participants and Subjects”. Note: “Where such research is conducted in countries where no such local governing laws and regulations related to human participant and subject research exist, Authors must at a bare minimum be prepared to show compliance with the above detailed principles.”

• Select the appropriate option under “Research Involving human participants / subjects” describing the ethics approval for your work. Below, you will also be asked to provide details of the ethical approval for your study; this will be hidden from reviewers but may be used by the program chairs if a query arises from the reviewers.

• Check the checkbox “I have read and understood the ACM Policy on Plagiarism, Misrepresentation, and Falsification; in particular, no version of this work is under submission elsewhere.”. Make sure to disclose possible overlap with your own previous work (“redundant publication”) to the ICER Program Committee co-chairs.

• Check the checkbox “I have read and understood the ICER Anonymization Policy” (see below).

ICER Anonymization Policy

ICER research paper submissions will be reviewed using a double-anonymous process: the authors do not know the identity of the reviewers and the reviewers do not know the identity of the authors. To ensure this:

• Avoid titles that indicate a clearly identifiable research project.

• Remove author names and affiliations. (If you are using LaTeX, you can start your document declaration with \documentclass[manuscript,review,anonymous]{acmart} to easily anonymize these.

• Avoid referring to yourself when citing your own work.

• Redact (just for review) portions of positionality statements that would identify you within the community (perhaps due to demographics shared by few others).

• Avoid references to your affiliation. For example, rather than referring to your actual university, you might write “A Large Metropolitan University (ALMU)” rather than “Auckland University of Technology (AUT)”.

• Redact any other identifying information such as contributors, course numbers, IRB names and numbers, grant titles and numbers, from the main text and the acknowledgements.

• Omit author details from the PDF you generate, such as author name or the name of the source document. These are often automatically inserted into exported PDFs, so be sure to check your PDF before submission.

Do not simply cover identifying details with a black box, as the text can easily be seen from under the box by dragging the cursor over it, and will still be read by screen readers.

Work that is not sufficiently anonymized will be desk-rejected by the PC chairs without offering an option to redact and resubmit.

Conflict of Interests

The SIGCSE Conflict of Interest policy applies to all submissions. You can review how conflicts will be managed by consulting our reviewer training, which details our review process.

Submission Format and Publication Workflow

Papers submitted to the research track of ICER have to be prepared according to the ACM TAPS workflow system. Read this page carefully to understand the new workflow.

Starting in 2021, ICER switched to a publication format (called TAPS) that separates content from presentation in support of accessibility. This means that the submission format and the publication format differ. For submission, we standardize on a single-column presentation.

• The submission template is either the single column Word Submission Template or the single column LaTeX (using the “manuscript,review,anonymous” style available in template, which you can see an example of in the sample-manuscript.tex example in the LaTeX master template samples). Reviewers will review in this single column format. You can download these templates on the ACM Master Article Templates page. If you use LaTeX and Overleaf, this is the template you need: ACM Conference Proceedings Primary Article Template, but adjust the \documentclass as specified above.
• The publication template is either the single column Word Submission Template or LaTeX template using “sigconf” and “manuscript” styles in acmart. You can download the templates on the ACM TAPS workflow page page, where you can also see example papers using the TAPS-compatible Word and LaTeX templates. If your paper is accepted, you will use the TAPS system to generate your final publication outputs. This will involve more than just submitting a PDF, requiring you to instead submit your Word or LaTeX source files and fix any errors in your source before the final version deadline listed above. The final published versions will be the ACM two-column conference PDF format (as well as XML, HTML, and ePub formats in the future).

For LaTeX users, be aware that there is a list of approved LaTeX packages for use with ACM TAPS. Not all packages are allowed.

For clarity, your paper should have a single column. If you have two columns, you have done it wrong. Your paper should look like this if you are using LaTeX (click for full-sized image):

Your paper should look like this if you are using Word (click for full-sized image):

Submission Length

Authors may submit papers following the formatting described above up to 18 pages (in the single column format) excluding references. In other words: scroll to the 19th page (and beyond) in your PDF. If they don’t exist, or have only references, you’re fine. Authors who feel the 18 pages are insufficient due to the nature of their research methods can request up to 3 additional pages. If you wish to include the additional three pages, fill in the section on the submission form to justify why you need those additional pages. If applicable, reviewers will be asked to evaluate whether these additional pages are necessary.

ICER papers must be self-contained in the sense that reviewers can assess the contribution without referring to any external material. Appendices in the submitted PDF are considered to be part of the main text and thus are subject to the page count (and should appear before the references). If authors want to provide additional material, e.g., codebooks, they must do so in an anonymized way via an external web resource of their choice; reviewers will neither be required nor asked, however, to consult such resources when assessing a paper’s contribution.

Accessibility

Authors will no doubt want their paper to be read as widely as possible. The way to make your paper’s content accessible to all is to ensure it is accessible. We strongly recommend reading and following the SIGCSE list of tips to make your paper accessible. Most pertinently at submission time, some of your reviewers may have accessibility needs, and may not be able to follow your paper (and thus may judge it negatively) unless you make it accessible.

Abstracts

A good abstract should summarize the question your paper asks and what answers it found. It is not enough to just say “We discuss our results and their implications”; say what you actually discovered, so future readers can learn that from your summary.

If your paper is empirical in nature, ICER recommends (but does not require) using a structured abstract that contains the following sections, each 1-2 sentences:

• Background and Context. What is the problem space you are working in? Which phenomena are you considering and why are they relevant and important for an ICER audience?
• Objectives. What research questions were you trying to answer?
• Method. What did you do to answer your research questions?
• Findings. What did you discover? Both positive and negative results should be summarized.
• Implications. What implications does your discovery have on prior and future research, and on the practice of computing education?

Not all papers may fit this structure, but if yours does, it will greatly help reviewers and future readers understand your paper’s research design and contribution.

Acceptance and Conditional Acceptance

All papers recommended for acceptance after the Senior PC meetings are either accepted or conditionally accepted. For accepted papers, there is no resubmission required other than the final camera-ready version. For conditionally-accepted papers, meta-reviews will indicate one or more minor revisions that are necessary for final acceptance; authors are responsible for submitting these revisions to HotCRP prior to the “Conditional Accept revisions due” deadline in the Call for Papers. The Senior PC and Program Chairs will review the final revisions; if they are acceptable, the paper will be officially accepted, and authors will have one week to submit an approved camera-ready version to TAPS for publication. If the Senior PC and Program Chairs judge that the request for revisions were not suitably addressed, the paper will be rejected.

Because the turnaround time for conditional acceptance is only one week, requested revisions will necessarily be minor: they may include presentation issues or requests for added clarity or details helpful for future readers of the archived paper. New results, new methodological details that change the interpretation of the results, or other substantially new content will neither be asked for nor allowed to be added.

ACM Publications Policies

By submitting your article to an ACM Publication, you are hereby acknowledging that you and your co-authors are subject to all ACM Publications Policies, including ACM’s new Publications Policy on Research Involving Human Participants and Subjects. Alleged violations of this policy or any ACM Publications Policy will be investigated by ACM and may result in a full retraction of your paper, in addition to other potential penalties, as per ACM Publications Policy.

ICER follows ACM’s Authorship Policy and Publications Policy on the Withdrawal, Correction, Retraction, and Removal of Works from ACM Publications and ACM DL. These state that any person listed as an author on a paper must (1) have made substantial contributions to the work, (2) have participated in drafting/revising the paper, (3) be aware that the paper has been submitted, and (4) agree to be held accountable for the content of the paper.

Please ensure that you and your co-authors obtain an ORCID ID, so you can complete the publishing process for your accepted paper. ACM has been involved in ORCID from the start and we have recently made a commitment to collect ORCID IDs from all of our published authors. We are committed to improve author discoverability, ensure proper attribution and contribute to ongoing community efforts around name normalization; your ORCID ID will help in these efforts.

Policy on use of AI in paper writing

The use of AI to check or improve spelling, grammar or within-sentence phrasing is permitted in writing ICER papers and does not need to be declared. The use of AI to generate images for figures is permitted but must be declared in the paper as per the ACM Policy. The use of AI to generate multi-sentence portions of text is not permitted at ICER, and the chairs may desk-reject papers that they believe have been written using AI tools. Reviewers who suspect papers may have been AI-generated should flag this to the program chairs.

Below is the review form that will be used for research papers at ICER 2025, which may be useful for authors and reviewers.

Summary of paper and contribution

Please summarize the paper briefly in your own words. Also provide a one-sentence summary of the paper’s main contribution (a finding, an idea, etc)

Prior Work - The submission is grounded in relevant prior work and leverages available theory when appropriate.

After reading the paper, one should feel more informed about prior literature, how that literature is related to the paper’s contributions, and how the present work contributes above and beyond the existing literature. Note that not all types of research will have relevant theory to discuss, nor do all contribution types need theory to make significant advances.

(Scored 1-5 where 1 is Unacceptable, 5 is Excellent.)

Prior Work - Explanation

Provide a detailed rationale for your score above, identifying any strengths and weaknesses related to this criterion. Reviewers should identify related work the authors might have missed and include pointers. Missing a paper that is relevant, but would not dramatically change the paper, is not sufficient grounds for rejecting a paper. Instead, criticism in reviews that leads to downgrading a paper should focus on missing prior work or theories that would significantly alter research questions, analysis, or interpretation of results.

Soundness - The submission’s methods and/or innovations soundly address its research questions.

The paper should answer the questions it poses, and it should do so with rigor, broadly construed. Claims must be aligned with the scientific findings of the work. This is the single most important difference between research papers and other kinds of knowledge sharing in computing education (e.g., experience reports), and the source of certainty researchers can offer. Note that soundness is relative to claims. For example, if a paper claims to have provided evidence of causality, but its methods did not do that, that would be grounds for critique. But if a paper only claimed to have found a correlation, and that correlation is a notable discovery that future work could explain, downgrading it for not demonstrating causality would be inappropriate.

(Scored 1-5 where 1 is Unacceptable, 5 is Excellent.)

Soundness - Explanation

Provide a detailed rationale for your score above, identifying any strengths and weaknesses related to this criterion.

Advances Knowledge - The submission advances knowledge of computing education by addressing (possibly novel) questions that are of interest to the computing education community.

The authors must convince you that the discoveries advance our knowledge of computing education in a significant way, e.g., by confirming uncertain prior work, adding a significant new idea, or making progress on a long-standing open question.

(Scored 1-5 where 1 is Unacceptable, 5 is Excellent.)

Advances Knowledge - Explanation

Provide a detailed rationale for your score above, identifying any strengths and weaknesses related to this criterion.

Clarity and Conciseness - The submission is written clearly enough to publish, and the methods of the paper are clear enough to be reproduced.

An ICER paper should be self-contained in the sense that readers should be able to understand most of the key details about how the authors conducted their work or made their innovation possible. If authors used additional pages, reviews should say whether those pages are warranted or not. We recognize that not all authors are fluent English writers; if, however, the paper requires significant editing to be comprehensible to fluent English readers, or it is unnecessarily verbose, it is not yet ready for publication. Reviews should address any omissions of research process or innovation details that would significantly alter your judgment of the paper’s validity.

(Scored 1-5 where 1 is Unacceptable, 5 is Excellent.)

Clarity and Conciseness - Explanation

Provide a detailed rationale for your score above, identifying any strengths and weaknesses related to this criterion.

Recommendation - Based on the criteria above, this paper should be published at ICER.

Based on all of the previous criteria, decide how strongly you believe the paper should be accepted or rejected, assuming authors make any modest, straightforward minor revisions you and other reviewers request before publication. Your recommendation should include whether the paper properly grounded itself in prior literature/theory, had sound methods with aligned claims, and is a significant contribution to CER knowledge. Papers that meet all of the criteria should be strongly accepted (though this does not imply that they are perfect). Papers that fail to meet most of the criteria should be strongly rejected. Each paper should be reviewed independently of others, as if it were a standalone submission. There is no target acceptance rate.

(Scored 1-5 where 1 is Strong reject, 5 is Strong accept.)

Recommendation Summary and Rationale

Provide a summary of the rationale for your recommendation, relative to the criteria above, including how you weighed the criteria in shaping your decision.

ICER currently evaluates papers against the following reviewing criteria, as independently as possible. These have been carefully chosen to be inclusive to many phenomena, epistemologies, and contribution types.

• Criterion A: Prior Work - The submission is grounded in relevant prior work and leverages available theory when appropriate.
• Criterion B: Soundness - The submission’s methods and/or innovations soundly address its research questions.
• Criterion C: Advances Knowledge - The submission advances knowledge of computing education by addressing (possibly novel) questions that are of interest to the computing education community.
• Criterion D: Clarity and Conciseness - The submission is written clearly enough to publish, and the methods of the paper are clear enough to be reproduced.

To be published at ICER, papers should be positively evaluated on all of these. The summary of this is another criterion:

• Summary: Based on the criteria above, this paper should be published at ICER.

Below, we discuss each criterion in turn.

Criterion A: Prior Work - The submission is grounded in relevant prior work and leverages available theory when appropriate.

Papers should draw on relevant prior work and theories, and explicitly show how they are tied to the questions addressed. After reading the paper, one should feel more informed about prior literature and how the paper contributes to the literature. Literature and its connection to the current work should be discussed in depth, with particular focus on how this present work adds to existing knowledge; it should not simply be a list of “work done in this area.” Note that not all types of research will have relevant theory to discuss, nor do all contribution types need theory to make significant advances. For example, a surprisingly robust but unexplained correlation might be an important discovery that later work could develop theory to explain. The authors may be drawing from a different interpretation of a theory than the reviewer, and these different perspectives should be considered carefully.

Reviewers should identify related work the authors might have missed and include pointers to the specific papers. Missing a paper that is relevant, but would not dramatically change the paper, is not sufficient grounds for rejecting a paper. Such citations can be added upon reviewers’ request prior to publication. Instead, criticism in reviews that leads to downgrading a paper should focus on missing prior work or theories that would significantly alter research questions, analysis, interpretation of results, or stated novel contributions.

Criterion B: Soundness - The submission’s methods and/or innovations soundly address its research questions.

The paper should answer the questions it poses, and it should do so with rigor, broadly construed. This is the single most important difference between research papers and other kinds of knowledge sharing in computing education (e.g., experience reports), and the source of certainty researchers can offer.
Note that soundness is relative to claims. For example, if a paper claims to have provided evidence of causality, but its methods did not do that, that would be grounds for critique. But if a paper only claimed to have found a correlation, and that correlation is a notable discovery that future work could explain, downgrading it for not demonstrating causality would be inappropriate. Details of the research process should be sufficiently described to be able to determine whether the work was sound. Authors should clearly describe limitations of the work and/or threats to validity.

Some specific guidelines:

• Refrain from applying criteria for quantitative methods to qualitative methods (e.g., critiquing a case study for a “small N” makes no sense; that is the point of a case study).
• Refrain from downgrading work based on a lack of a statistically significant difference if the study demonstrates sufficient power to detect a difference. A lack of difference can be discovery, too.
• Refrain from asking for the paper to do more than it claims if the demonstrated claims are sufficiently publishable (e.g., “I would publish this if it had also demonstrated knowledge transfer”).
• Refrain from relying on inexpert, anecdotal judgments (e.g., “I don’t know much about this but I played with it once and it didn’t work”).

Criterion C: Advances Knowledge - The submission advances knowledge of computing education by addressing (possibly novel) questions that are of interest to the computing education community.

A paper can meet the previous criteria and still fail to advance what we know about the phenomena. It is up to the authors to convince you that the discoveries advance our knowledge in some way, e.g., by confirming uncertain prior work, adding a significant new idea, or making progress on a long-standing open question. In addition, the authors should convince you that their study is an important contribution to the community. Secondarily, there should be someone in computing education who might find the discovery interesting. It does not have to be interesting to a particular reviewer, and a particular reviewer does not have to be absolutely confident that an audience exists.

Some specific guidelines:

• Refrain from downgrading work because another, single paper was already published on the topic. Discoveries accumulate over many papers, not just one.
• Refrain from downgrading work that contributes a really new idea for not yet having everything figured out about it. Again, new discoveries may require multiple papers.
• Refrain from downgrading work because the results do not appear generalizable or were only obtained at a specific institution. Many papers explicitly discuss such limitations and possible remedies. Also, generalizability takes time, and, by their very nature, some qualitative methods do not lead to generalizable results.
• Refrain from downgrading work based on “only” being a replication. Replications, if done with diligence, are important.
• Refrain from downgrading work based on investigating phenomena you personally do not like (e.g., “I hate object-oriented languages, this work does not matter”).

Criterion D: Clarity and Conciseness - The submission is written clearly enough to publish, and the methods of the paper are clear enough to be reproduced.

Papers need to be clear and concise, both to be comprehensible to diverse audiences, but also to ensure the community is not overburdened by verboseness. We recognize that not all authors are fluent English writers; if, however, the paper requires significant editing to be comprehensible to fluent English readers, or it is unnecessarily verbose, it is not yet ready for publication.

An ICER paper should be self-contained in the sense that readers should be able to understand most of the key details about how the authors conducted their work or made their innovation possible. This is key for replication and meta-analysis of studies that come from positivist or post-positivist epistemologies. For interpretivist works, it is also key for what Checkland and Howell called “recoverability” (See Tracy et al. 2010 for a detailed overview for evaluating qualitative work).

If authors requested additional pages for their work, this is where reviewers can address whether those additional pages are necessary. If reviewers agree that the additional space is needed, they should say why in this section. If reviewers disagree, they should either say how the paper should be cut (if the paper would still be acceptable without the extra pages) or downgrade the work due to a lack of conciseness.

Some specific guidelines:

• Refrain from downgrading work based on having easily fixed spelling and grammar issues.
• Refrain from downgrading a sufficiently clear paper because it could be clearer. All writing can be clearer in some way.
• Refrain from downgrading work based on not using all of the available page count. It is okay if a paper is short but significant.
• Refrain from asking for more detail unless you are certain there is space or - if there is not space - you can provide concrete suggestions for what to cut.
• Refrain from downgrading work based on not describing every detail.

Summary: Based on the criteria above, this paper should be published at ICER.

Based on all of the previous criteria, decide how strongly you believe the paper should be accepted or rejected, assuming authors make any modest, straightforward minor revisions you and other reviewers request before publication. Papers that meet all of the criteria should be strongly accepted (though this does not imply that they are perfect). Papers that fail to meet most of the criteria should be strongly rejected. Each paper should be reviewed independently of others. There are no conference presentation “slots”; there is no target acceptance rate. Neither should be a factor in reviewing individual submissions.

Specific guidelines: Because each paper should be judged on its own, (meta-)reviewers are asked to do the following:

• Refrain from recommending to accept a paper because it was the best in your set. It is possible that none of your papers sufficiently meet the criteria.
• Refrain from recommending to reject a paper because it should not take up a “slot”. The PC chairs will devise a program for however many papers sufficiently meet the criteria, whether that is 5 or 50. There is no need to preemptively design the program through your review; focus on the criteria.
• When recommending a paper for acceptance, consider all review criteria. For example, a sound paper that does not advance knowledge should not be recommended for acceptance. Nor should a paper that claims to advance knowledge be accepted if the methods are not sound or if the findings do not support such claims. This does not mean a paper has to be perfect (no papers are); there may be some limitations to the study (e.g. data issues or limits in the study design) but if they are discussed in the paper and the contributions are worded accordingly, this is fine.

Guidelines for (Meta-)Reviewers:

Meta-reviewers will need to write a meta-review at the end of the process. They should briefly summarise the main points from the individual reviews, especially those that were critical for making the final recommendation. The meta-review should make it clear how the paper contributes (or does not contribute) to computing education. The meta review should make it clear to authors why their paper was accepted or rejected. Try to offer encouragement to authors where possible: if they choose to continue working on this paper, what could be done to make it into an acceptable paper?

The recommendations for a paper in a meta-review can be: reject, discuss, conditional accept, accept. All papers except rejects will be discussed by a group from the senior program committee. The difference is that discuss papers are borderline or rely on some critical decision that needs wider discussion (e.g. whether a paper is on topic, or whether it is ethical); conditional accept papers should have a specific set of suggestions in the draft meta-review for what the “conditional” part is (i.e. what needs changing before it becomes acceptable). Accept papers are thought to be good enough to accept as-is, but we will briefly discuss them among the senior program committee to help with consistency of judgement.

Award Recommendations

On the review form, reviewers may signal to the meta-reviewer and PC chairs that they believe the submission should be considered for a best paper award. Selecting this option in the review form is visible to the other (meta-)reviewers as part of your review, but it is not disclosed to the authors. Reviewers should recognize papers that best illustrate the highest standards of computing education research, taking into account the quality of its questions asked, methodology, analysis, writing, and contribution to the field. This includes papers that meet all of the review criteria in exemplary ways (e.g., research that was particularly well designed, executed, and communicated), or papers that meet specific review criteria in exemplary ways (e.g., discoveries that are particularly significant for the community). The meta-review form for each paper includes an option to officially nominate a paper to the Awards Committee for the best-paper award. Reviewers may flag papers for award consideration during review, but meta-reviewers are ultimately responsible for nominating papers for the best paper award. Each meta-reviewer may nominate at most two papers for the best paper award. Nominated papers may or may not have been nominated by one or more reviewers. Nominations should be recorded in HotCRP and be accompanied by a paragraph outlining the rationale for nomination. NOTE: Whether a paper has been nominated and the accompanying rationale are not disclosed to the authors as part of the meta-review.
Meta-reviewers are encouraged to review and finalize their nominations at the conclusion of the SPC meeting to allow for possible calibration. Once paper decisions have been sent, the submission chair will make PDFs and the corresponding rationales for all nominated papers available to the Awards Chair. Additionally, a list of all meta-reviewers that have handled any nominated paper or have one or more conflicts of interest with any nominated paper will be disclosed to the Awards Chair, as those members are not eligible to serve on the Awards Committee.

Possible Plagiarism, Misrepresentation, and Falsification

If after reading a submission, you suspect that it has in some way plagiarized from some other source, do the following:

• Read the ACM guidelines on Plagiarism, Misrepresentation, and Falsification.
• If you think there is a potential issue, write the PC chairs at pc-chairs@icer.acm.org to escalate the potential violation, and share any information you have about the case. Authors are required to disclose any potentially overlapping work to the PC chairs upon submission.

The chairs will investigate and decide as necessary prior to the acceptance notification deadline. You should not mark the paper for rejection based on suspected plagiarism. Mark it based on the paper as it stands, while the PC chairs investigate.

Thank you very much for reading this document and thank you very much for being part of the ICER reviewing process. Do not hesitate to email the Program Co-Chairs at pc-chairs@icer.acm.org if you have any questions.

Policy on use of AI in writing reviews and meta-reviews

The use of AI to check or improve spelling, grammar or within-sentence phrasing is permitted in writing ICER reviews and meta-reviews. The use of AI to generate multi-sentence portions of text in reviews or meta-reviews is not allowed. Reviewers and meta-reviewers are urged to flag possible uses of AI in other reviews or meta-reviews, either to the meta-reviewer or to the program chairs. Reviews judged to have been written using AI will be removed.

This document is a living document intended to capture the reviewing policies of the ICER community. Please email the Program Co-Chairs at pc-chairs@icer.acm.org with comments or questions; all will be taken into account when updating this document for next year’s ICER.

Based on the ICER 2020/2021 Reviewing Guidelines (Amy Ko & Anthony Robins & Jan Vahrenhold) as well as the ICSE 2022 Reviewing Guidelines (Daniela Damian & Andreas Zeller). We are thankful for the input on these earlier documents provided by members of the ICER community.

Table of Contents

1. Goals of the ICER Reviewing Process
2. Action Items
3. Submission System
4. Roles in the Review Process
5. Principles Behind ICER Reviewing
6. Conflicts of Interest
7. The Reviewing Process
8. Review Criteria
9. Award Recommendations
10. Possible Plagiarism, Misrepresentation, and Falsification
11. Practical Suggestions for Writing Reviews

1. Goals of the ICER Reviewing Process

The ICER Reviewing Process as outlined in this document is designed to support reaching the following goals:

• Accept high quality papers
• Give clear feedback to papers of insufficient quality
• Evaluate papers consistently
• Provide transparency in the review process
• Embrace diversity of perspectives, but work in an inclusive, safe, collegial environment
• Drive decisions by consensus among reviewers
• Strive for manageable workload for PC members

2. Action Items

Prior to continuing to read this document, please do the following:

• Read the call for papers at https://icer2025.acm.org/track/icer-2025-papers. This is the ground truth for scope and submission requirements. We expect you to account for these in your reviews.
• Read the author guidelines at https://icer2025.acm.org/track/icer-2025-papers#Author-Guidelines. We expect your reviews and meta-reviews to be consistent with these guidelines. After having read this document, please block off a number of time slots in your calendar:

PC/Reviewers

Key dates:

• Prior to March 14, 2025: Familiarize yourself with the ICER 2024 Reviewing Guidelines
• March 14 – 21, 2025: Bid on papers and declare conflicts
• March 21 – April 15, 2025: Review roughly 6 papers, depending on submission volume
• April 15 – May 1, 2025: Asynchronously discuss papers with other reviewers and the Senior PC members assigned to your papers.

SPC/Meta-Reviewers

Key dates:

• Prior to March 14, 2025: Familiarize yourself with the ICER 2024 Reviewing Guidelines
• March 14 – 21, 2025: Bid on papers and declare conflicts
• March 21 – April 15, 2025: Monitor the reviewing of 9-11 papers, depending on submission volume
• April 15 – May 1, 2025: Asynchronously discuss papers with the reviewers assigned to those papers, prepare meta-review and recommendation.
• May 1, 2025: Complete submission of meta-reviews for all papers
• May 1 – May 6, 2025: Skim the paper and read reviews for 1-2 submissions (which you did not handle) that will be discussed at the Senior PC meeting.
• May 6 – May 9, 2025: Participate in synchronous, online SPC meetings to discuss and decide on papers and finalize your meta-reviews. Based on last year’s experiences, we are currently planning for up to four such virtual meetings to accommodate different time zones. As last year, we also expect that your participation in two of these meetings would be necessary and sufficient. Overall, expect it to be a four-hours commitment during this week, plus time to review your 1-2 assigned discussion papers.
• May 29 – June 5, 2025: Re-check (a hopefully very, very small number of) minor revisions against the requests made in “conditional accept” decisions.

If you are new to reviewing in the Computing Education Research community, the following ITiCSE Working Group Report may serve as an introduction:

• Petre M, Sanders K, McCartney R, Ahmadzadeh M, Connolly C, Hamouda S, Harrington B, Lumbroso J, Maguire J, Malmi L, McGill MM, Vahrenhold J. 2020. “Mapping the Landscape of Peer Review in Computing Education Research.” In ITiCSE-WGR ’20: Proceedings of the Working Group Reports on Innovation and Technology in Computer Science Education, edited by Rφίling G, Krogstie B, 173-209. New York, NY: ACM Press. doi: 10.1145/3437800.3439207.

3. Submission System

ICER 2025 uses the HotCRP platform for its reviewing process. If you are unfamiliar with this, you will find a basic tutorial below. But first, make sure you can sign in, then bookmark it: https://icer2025.hotcrp.com If you have trouble signing in, or you need help with anything, contact Juho Leinonen juho.2.leinonen@aalto.fi and David Smith dhsmith2@illinois.edu , the ICER 2025 submission chairs, for help. Make sure that you can log in to HotCRP and that your name and other metadata is correct. Check that emails from HotCRP are not marked as spam and that HotCRP email notifications are enabled.

6. Conflicts of Interest

ICER takes conflicts of interest, both real and perceived, quite seriously. The conference adheres to the ACM conflict of interest policy (https://www.acm.org/publications/policies/conflict-of-interest) as well as the SIGCSE conflict of interest policy (https://sigcse.org/policies/COI.html). These state that a paper submitted to the ICER conference is a conflict of interest for an individual if at least one of the following is true:

• The individual is a co-author of the paper
• A student of the individual is a co-author of the paper
• The individual identifies the paper as a conflict of interest, i.e., that the individual does not believe that he or she can provide an impartial evaluation of the paper.

The following policies apply to conference organizers:

• The chairs of any track are not allowed to submit to that track.
• Program chairs are not allow to submit to any track.
• All other conference organizers are allowed to submit to any track.
• All reviewers (PC members) and meta-reviewers (SPC members) are allowed to submit to any track.

No reviewer, meta-reviewer, or chair with a conflict of interest in the paper will be included in any evaluation, discussion, or decision about the paper. It is the responsibility of the reviewers, meta-reviewers, and chairs to declare their conflicts of interest throughout the process. The corresponding actions are outlined below for each relevant step of the reviewing process. It is the responsibility of the chairs to ensure that no reviewer or meta-reviewer is assigned a role in the review process for any paper for which they have a conflict of interest.

7. The Reviewing Process

Step 1: Authors Submit Abstracts

Authors will submit a title and abstract one week prior to assigning papers. Authors are allowed to revise their title and abstract before the full paper submission deadline.

Step 2: Reviewers and Meta-Reviewers Bid for Papers

Reviewers and meta-reviewers will be asked to bid on papers for which they have sufficient expertise (in both phenomena and methods) and then the PC chairs will assign papers based on these bids. The purpose of bidding is not to express interest in papers you want to read. It is to express your expertise and eligibility for fairly evaluating the work. These are subtly but importantly different purposes.

• Specify all of your conflicts of interest.
• Bid on all of the papers you believe you have sufficient expertise to review. Sufficient expertise includes knowledge of research methods used and prior research on the phenomena. Practical knowledge of a topic is helpful, but insufficient.
• Do not bid on papers about topics, techniques, or methods that you strongly oppose. That precludes authors from being fairly reviewed by authors with negative bias; see below for positive biases and how to control for them.

Step 3: Authors Submit Papers

Submissions are due one week after the abstracts are due. As you read in the submission instructions (https://icer2025.acm.org/track/icer-2025-papers#Submission-Instructions), submissions are supposed to be sufficiently anonymous that a reader cannot determine the identity or affiliation of the authors. The main purpose of ICER’s anonymous reviewing process is to reduce the influence of potential (positive or negative) biases on reviewers’ assessments. You should be able to review the work without knowing the authors or their affiliations. Do not try to find out the identity of authors. (Most guesses will be wrong anyway.) See the submission instructions for what constitutes sufficient anonymization. When in doubt, write to the PC chairs for clarity at pc-chairs@icer.acm.org.

Step 4: PC Chairs Decide on Desk-Rejects

The PC chairs, with the help of the submissions chairs, will review each submission for papers that violate anonymization requirements, length restrictions, or plagiarism policies. Authors of desk rejected papers will be notified immediately. The PC chairs may not catch every issue. If you see something during review that you believe should be desk rejected, contact the chairs before you write a review; the PC chairs will make the final judgement about whether something is a violation, and give you guidance on whether and if so how to write a review.

Managing Conflicts of Interest

PC chairs with conflicts are excluded from deciding on desk rejected papers, leaving the decision to the other program chair. If both PC chairs are conflicted on a paper, an experienced member of the senior program committee will be designated to make the decision.

Step 5: PC Chairs Assign Reviewers

Based on the bids and their judgement, the PC chairs will collaboratively assign at least three reviewers (PC members) and one meta-reviewer (SPC member) for each submission. The PC chairs will be advised by HotCRP’s assignment algorithm, which depends on all bids being high quality. Remember, for these assignments to be fair and good, your bids should only be based on your expertise and eligibility. Interest alone is not sufficient for bidding on a paper. The chairs will review the algorithm’s assignments to identify potential misalignments with expertise.

Step 6a: Reviewers Review Papers

Assigned reviewers submit their anonymous reviews through HotCRP by the review deadline, evaluating each of their papers against the review criteria (see Review Criteria). The time allocated for reviews is four weeks in which the reviews need to be written. Due to the internal and external (publication) deadlines, there cannot be any extensions.

Step 6b: Meta-Reviewers and PC Chairs Monitor Progress

Meta-reviewers and PC chairs will periodically check in to ensure that progress is being made.

Step 7: Reviewers and Meta-Reviewers Discuss Reviews

After the reviewing period, the assigned meta-reviewer asks the reviewers to read the other reviewers’ reviews and begin a discussion about any disagreements that arise. If there is such a disagreement, reviewers are asked to do the following:

• Engage in a discussion about sources of disagreement.
• Use the review criteria to guide your discussions.
• Be polite, friendly, and constructive at all times.
• Be responsive and react as soon as new information comes in.
• Remain open to other reviewers shifting your judgements.

If your judgement does shift, update your review to reflect your new views. There is no need to indicate to the authors that you changed your review but do leave a comment for the other reviewers and the meta-reviewer indicating what you changed and why (HotCRP does not track changes). Discussing a paper is not about who wins or who is right. It is about how, in the light of all information, a group of reviewers can find the best decision on a paper. All reviewers (and the authors!) have their unique perspective and competence. It is perfectly normal that they may have seen things you have not, just as you may have seen things they have not. The important thing is to accept that the group will see more than the individual. Therefore, you can always (and are encouraged to!) shift your stance in light of the extra knowledge.

If you have configured HotCRP notifications correctly, you will be notified as soon as new information (another review or a new discussion item) about your paper comes in. It is important that you react to these, and as soon as possible. Do not let your colleagues wait for days when all that is needed is some short statement from your side.

Step 8: Meta-Reviewers Write Meta-Reviews

After the discussion phase, meta-reviewers use the reviews, the discussion, and their own evaluation of the work to write a meta-review and recommendation. A meta-review should summarize the key strengths and weaknesses of the paper, in light of the review criteria, and explain how these led to the decision. The summary and explanation should help the authors in revising their work where appropriate. A generic meta-review (“After long discussion, the reviewers decided that the paper is not up to ICER standards, and therefore rejected the paper”) is not sufficient. There are four possible meta-review recommendations: reject, discuss, conditional accept, and accept. The recommendation needs to be entered in the meta-review. The PC chairs will make the final decision based on the meta-review rationale; if necessary, papers will be discussed at the SPC meeting.

• Reject. Ensure that the meta-review constructively summarizes the reviews and the rationale for rejection. The PC chairs will review all meta-reviews to ensure that reviews are constructive, and may request meta-reviewers to revise their meta-reviews as necessary.
• Discuss. Ensure that the meta-review summarizes the open questions that need to be resolved at the SPC meeting discussion, where the paper will either be recommended as reject, conditional accept, or accept. Papers marked discussed will be scheduled for discussion at the SPC meeting. All papers for which the opinion of the meta-reviewer and the majority of reviewer recommendations do not align should be marked “discuss” as well.
• Conditional Accept. Ensure that the meta-review explicitly and clearly states the conditions that must be met with minor revisions before the paper can be accepted. To accept with conditions, the conditions must be feasible to make within the one-week revision period, so they must be minor. The PC chairs will make the final decision on whether the requested revisions are minor enough to warrant conditional acceptance.
• Accept. These papers will be accepted, assuming authors deanonymize the paper and meet the final version deadline.

Step 9: PC Chairs and Meta-Reviewers Discuss Papers

The PC chairs will host synchronous SPC meetings with all available meta-reviewers (SPC members) to discuss and decide on all “discuss”, “conditional accept”, and “accept” papers. Before this meeting, a second meta-reviewer will be assigned to each such paper, ensuring that there are at least two meta-reviewers to facilitate discussion. Each meta-reviewer assigned to a paper should come prepared to present the paper, its reviews, and the HotCRP discussion. Each meta-reviewer’s job is to present their recommendation, and/or if they requested discussion, present the uncertainty that prevents them from making one. All meta-reviewers who are available to attend a SPC meeting session should, at a minimum, skim each of the papers to be discussed and their reviews (excluding those for which they are conflicted), so they are familiar with the papers and their reviews prior to the discussions. At the meeting, the goal is to collectively reach consensus, rather than relying on the PC chairs alone to make final decisions. Papers may move from “discuss” to either “reject”, “conditional accept”, or “accept”; if there are conditions, they must be approved by a majority of the non-conflicted SPC and PC chairs at the discussion. After a decision is made in each case, the original SPC member will add a summary of the discussion at the end of their meta-review, explaining the rationale for the final decision, as well as any conditions for acceptance, and updating the recommendation tag in HotCRP.

Step 10: PC Chair Review

Non-conflicted PC chairs will review meta-reviews, meta-review recommendations, and finalize paper decisions.

Step 11: Notifications

After the SPC meeting, the PC chairs will notify all authors of the decisions about their papers; these notifications will be via email through HotCRP. Papers that are (unconditionally) accepted will be encouraged to make any changes that may have been suggested but not required; papers that are conditionally accepted will be reminded of the revision evaluation deadline.

Step 12: Authors of Conditionally Accepted Papers Revise their Papers

Authors of conditionally accepted papers have one week to incorporate the requested revisions and to submit their final versions for review by the assigned meta-reviewer.

Step 13: Meta-Reviewers Check Revised Papers

Meta-reviewers will check the revised papers against the required revisions. Based on the outcome of this, they will change their recommendation to either “accept” or “reject” and will update their meta-reviews to reflect this.

Step 14: Notifications

PC chairs will sanity-check all comments on those papers for which revisions were submitted. Conditionally accepted papers for which no revisions were received will be marked as “reject”. PC chairs then finalize decisions. After this review, all recommendations will be converted to official accept or reject decisions in HotCRP and authors will be notified of these final decisions via email sent through HotCRP. Authors will then submit to ACM TAPS for final publication.

Program Committee (PC) Chairs

Each year there are two program committee co-chairs. The PC chairs are solicited by the ICER steering committee and appointed by the SIGCSE board to serve a two-year term. One new appointment is made each year so that in any given year there is always a continuing program chair from the prior year and a new program chair. Appointment criteria include prior attendance and publication at ICER, past service on the ICER Program Committee, research excellence in Computing Education, collaborative and organizational skills to share oversight of the program selection process. The ICER Steering Committee solicits and selects candidates for future PC chairs.

Program Committee (PC) Members / Reviewers

PC members write reviews of submissions, evaluating them against the review criteria. The PC chairs invite and appoint the reviewers. Each reviewer will serve a one-year term, with no limits on reappointment. Appointment criteria include expertise in relevant areas of computing education research and past reviewing experience in computing education research venues. Together, all reviewers constitute the program committee (PC). The PC chairs are responsible for inviting returning and new members of the PC, keeping in mind the various forms of diversity that are present at ICER.

Senior Program Committee Members (SPC) / Meta-Reviewers

SPC members review the PC members’ reviews, ensuring that the review content is constructive and aligned with the review criteria, as well as summarizing reviews and making recommendations for a paper’s acceptance and rejection. They also moderate discussions about each paper and provide feedback on reviews if necessary, asking reviewers to improve the quality of reviews. Finally, they participate in a synchronous SPC meeting to make final recommendations about each paper, and review authors’ minor revisions. The PC chairs invite and appoint Senior PC members. Each Senior PC member can be appointed for up to four years in a row; after a hiatus of at least one year, preferably two years, re-appointment is possible.