The CS&IT Annual Conference is coming up and I am getting more excited all the time. As a computer science teacher, it’s always been the best professional development I attend, as every session has something designed for teaching computing. This event is thought-provoking, useful, and always interesting.
In the last few years, the model of creating the program has completely changed, and with the help of Program Chair Duncan Buell, I wanted to crack open the lid and let you see some of the magic.
First, multiple reviewers read each submission and rated each on several criteria including quality and relevance to the CS&IT audience. Each reviewer was assigned a random selection of submissions, so each submission was read by different reviewers, with overlap to improve inter-rater reliability. This is how many conferences handle reviews.
Second Duncan went through the top 35 proposals, looking for anomalies, such as cases where all reviews were high but one which brought down the average, to verify that the numbers were reasonable. He also tried to notice if particular reviewers had been uniformly harsh or uniformly easy in an attempt to reduce the effect of “the luck of the draw” of which reviewers reviewed which proposals.
Then he started working to figure out which of the top submissions would be in the final program: “In my experience, the first 6 to 10 of 20 would be fairly obvious. The next five or so might be reasonably easy to pencil in, and then it gets tough.”
The goal is to have a diverse set of offerings from a diverse set of presenters. For example, two of the top 35 proposals were about the new AP Principles course. Given that we only have 20 sessions, the choice was made to offer only the top-rated proposal about AP Principles rather than having two sessions about the new AP and miss out on a presentation about something else. Some proposers submitted multiple ideas, and often only one was chosen, particularly when what looked like the same submission came in as a one-hour and three-hour option.
Finally the committee organized the top sessions into the program, whittling it down further to make sure that each time slot has a diverse set of offerings likely to appeal to different attendee populations.
I think they’ve done a wonderful job and I hope you will agree. If you haven’t signed up yet, hurry up and do so! I look forward to seeing you in New York this summer.
Michelle Hutton
CSTA President