Report on the Technical Track of ICSE 2015

The goal of this report is to inform the community about how we, the program chairs, organized and managed the technical track of ICSE 2015. The technical track, the most prestigious of our flagship conference, is continuously evolving in an effort to make better decisions, give better feedback to the authors, and more wisely involve the community resources. This year we adjusted the reviewing process to use a model consisting of a traditional Program Committee (PC) and a Reviewing Committee (RC) to get more reviewing expertise and balance the reviewing load, provided early rejection notices, made the process more transparent to the community, monitored more closely the quality of the reviews, used online discussions and summaries to improve the communication between reviewers and with authors, and assigned as many reviewers per submission as possible to make better decisions. The report provides details on how the technical track was organized (Section 1) and performed based on data collected through the conference management tool (Section 2), a survey of reviewers and authors (Section 3), and our personal experiences (Section 4). • 452 submissions by more than 1250 authors • Reviewing Committees – 48 PC + 33 RC members from 25 countries – 23% new to ICSE technical track reviewing committees • Reviewing Process – ∼ 100 reviewing days – 1137 reviews + summaries + online discussions (∼100 ICSE submissions) – +15% of reviews were revised for anti-patterns found by program chairs • Reviewing Expertise and Quality – Reviewing expertise was higher than previous ICSEs (as reported by authors and reviewers) – Authors assessment show slight improvements over last ICSE in terms of reviewing quality, but about 20% of authors still disliked their reviews – Authors of accepted papers had a much more favorable view of their reviews – 87% of the PC/RC agreed that the final decisions were balanced and justified • Reviewing Load – Max 18 reviewed submissions for PC members and 9 for RC members – 90% agreed that the load and schedule was manageable • Program includes 84 papers (18.5% acceptance) and 6 ACM-Sigsoft D. Awards 2 Brief Reflections from the Chairs (expanded in Section 4). • Refine: Process. It received improved grades for reviewing expertise, slightly improved for quality, and controlled the reviewers' workload as planned. But there is still a room for improving the quality and the delivery of the feedback. • Drop: Panic about scalability of …