Assessing Impact in Computing Education

I spend much of my time thinking about reforming computer science education in Los Angeles. My goal is to make computer science courses accessible and engaging for students who have not traditionally participated in computing. To this end, I have been part of a dynamic team (the Computer Science Equity Alliance) that has developed a new course which introduces students to the foundational knowledge of computer science. In the professional development which is coupled with this course, a dynamic community of teachers, expert practitioners, and university faculty come together to build individual and collective knowledge about computing topics and the instructional strategies needed to engage diverse students in computer science. We piloted this course the first year in seven schools and enrolled 300 students. This fall, the course will be offered in 20 schools in the Los Angeles Unified school district.
However, measuring the impact of this effort has different meanings for different national and local stakeholders. Thought the mere existence of this course when there were no courses before is a measure of success itself, the computer science education community wants to know more about the impact of this course on high school students.
When informed about this effort or other K-12 initiatives in computing, many leaders of computer science education often seek measures of longitudinal effectiveness:
* Do these students take other computing courses?
* Do the students pursue a major in computing?
* Pursue advanced degrees?
* Work in the computing industry?
Other STEM educators believe the way to measure the impact of a foundational computing course is to measure mathematics and science achievement skills of students participating in the course, and compare these scores to non-computer science takers. They want to know, does learning about foundational knowledge in computing raise test scores in related subjects?
While these questions are important, I resist the urge to rely on this type of data to measure the success of our mission to broaden participation in high school computing. Our goal is for all students to develop an understanding of the computing discipline, not to train them to enter the pipeline and become computer scientists. Just as reforming algebra education does not set its goal as more math majors, computing education at K-12 should not be judged on higher education enrollment patterns. There are just too many confounding variables in play in decision-making at the college level. And while we anticipate that developed computational thinking skills might transfer to tackling problems in other STEM subjects, focusing on test scores in math and science reflects an unfortunate belief that computing is only important for its positive impact on achievement disciplines, rather than a discipline itself.
Instead, I believe the best data will come from looking at enrollment patterns over time (increases by gender, race, English language learner status); how many students continue to more advanced courses when offered at their school, interviewing teachers about their experiences teaching computing to Los Angeles students, and collecting pre- and post-class survey data from computing students about their perception of the importance of computer science, their interest in the subject, and their motivation to pursue further study. For us, this triangulation of data will most truly assess the effectiveness of a foundational course for broadening participation in high school computer science.
A student response elucidates this perspective: “I’m still paving my path to become a professional musician, but now I can use what I’ve learned from this computer science class to further that career, using codes for websites, banners, playlists, etc.” Though not pursuing computer science as a profession, the knowledge of computing will influence this young person’s life goals. For me, a course that offers such opportunities is the goal in itself.
Joanna Goode
CSTA Board of Directors

Leave a Reply