A joint blog post by Chris Stephenson of CSTA, Alfred Thompson of Microsoft, and Mark Guzdial of Georgia Tech
As much as we believe and try to make the case that studying computer science is good for all students, there is a profound lack of research to actually support this contention. With the movement to data driven decision making in every area of education, our inability to advocate for more and better computer science education in K-12 is severely curtailed by our inability to support our own observations and claims.
There are some things we do know which may help us make a more effective argument for K-12 computer science education, or at least make us better K-12 computer science educators.
We know that even pre-teen students have serious misconceptions about what computer science is and that this fundamental lack of understanding makes it very difficult to engage and retain students. Research has shown us that many students believe that computer science is simply using applications well. In one study, after six weeks of learning Scratch, Alice, Pico Crickets, and similar tools, and with Mike Hewner (a PhD student in CS education at Georgia Tech) lecturing them on CS topics, students still came away with the belief (for example) that “Someone who does Photoshop really well is a great computer scientist.” They probably think that programmers work in locked window-less rooms and never shower too!
We know that *not* having a CS background can be a serious detriment in a wide variety of professions. In 2005, Mary Shaw, Chris Saffidi, and Brad Myers presented a research paper focusing on the gap between professionals who program as part of their jobs and the number of people actually trained to do this work. These researchers estimated that by 2012 there will be 3 million professional software developers and 13 million people who program as part of their jobs but aren’t software developers. Brian Dorn’s just-completed dissertation shows why this is a significant problem. In his study of graphics designers who are self-taught programmers, Dorn found that in order to understand code fragments, the designers do things like search for a variable name — not knowing that that’s completely arbitrary and not useful. One of Brian’s subjects who was working in JavaScript, for example, stumbled onto a Java web page, and spent 30 minutes poring over language details that were irrelevant for his task
We still don’t know, however, whether learning computer science helps with anything else in the curriculum. . We have results showing that learning a visual language *does* transfer knowledge to textual programming later. Chris Hundhausen just did a careful HCI study showing that learners could get started more quickly with a visual programming language (like Scratch, Alice, or Kodu), and that parts of that knowledge did transfer to textual programming. That’s a big deal, because it says that Scratch and Alice really are useful for learning CS that will be useful later in life.
There are, however, no recent, scientifically-valid studies that show that students are able to transfer key concepts that they learn in computer science to other learning or that students who study computer science perform better on high-stakes testing in other subject areas (specifically math and science). The last major review of the research in this space (by David Palumbo in 1990) showed little evidence that programming impacted problem-solving in other domains. Neither are there recent studies (the most recent was Taylor and Mountfield in 1991) that determine whether students who study computer science in high school perform better in any area of post-secondary study including computer science. Sharon Carver’s dissertation work in 1988 showed that one *could* teach Logo so that it improved how elementary students solved problems in other areas (e.g., debugging instructions on maps), but little research has followed up on that result.
This lack of research-supported evidence is particularly troubling in light of the current discussions about the importance of “Computational Thinking”. While there is strong support for CT in many parts of the community including the National Science Foundation, without a strong and agreed-upon definition and effective assessment measures for students at various learning levels, we don’t have hard evidence there that CT is useful let alone necessary for every student.
We do know that we need to do a better job of convincing students that computer science is worth their interest and we might actually be making some progress on this front. For example, many teachers are working hard to help students see the connections between the current technologies that students are interested in (social networking, mobile applications, etc.) and the issues that they care about (the ways that medical agencies use computers to track and control epidemics or how relief agencies depend on computerized logistical systems to get the right sort of aid to the right places at the right time in an emergency). But once again, we have not established scientifically whether these connections motivate students who would not otherwise be interested in computer science.
There are some things we do know and some we can even prove scientifically but the bottom line is that we need more research. We need research that is long-term, broad reaching, and scientifically valid. We need to know what our students are learning and why it matters to them. We need to know how to help them learn better. And we need to know how to do a better job of engaging, inspiring, and retaining them. It is time for computer science education to grow up and prove its value, just as all of the other core disciplines are now having to do.
Chris Stephenson, CSTA
Mark Guzdial, Georgia Tech
Alfred Thompson, Microsoft
The relationship between learning mathematics and success as a computer programmer may be fertile ground for investigation and research.
Let’s keep a forward-leaning perspective. Learning computer science is good for all students who plan to thrive in technological society, because it confers a sense of command and power over software-based capabilities and their evolution.
The current generation of k12 students will be surrounded with smart devices and high-speed net infrastructure their entire lives. The question needing to be faced by educators and policymakers is: Will these kids occupy a position of dominion over automata, or coexist nervously with systems over which they have little control or say? If through educational neglect, these future citizens feel powerless to control technological artifacts, we’ll have inadequately prepared them for the world of today, let alone the more advanced future they will occupy.
Let’s accept the fact that K12 CS is up against institutional ossification. Bureaucratic institutions like K12 are not “wired up” to an external environment that demands keeping up-to-date with a changing world. K12 funding follows a “factory model”, i.e. a business with no R&D, just production. Any tech business run under this model is doomed to obsolescence-failure in the next 3-5 years.
The “standards-based results” movement further imperils the K12 system’s ability to modernize content, by focussing the very definition of educational progress on raising test scores. This wouldn’t be disastrous if the testing outfits like ETS and College Board had the vision and discipline to regularly merge new subject content like CS, digital electronics, nano-materials science into the standardized tests. Were they to do this, K12 schools would adapt to changing requirements (or face declining test scores). The key question is, what is the process for new educational content to infiltrate the “core” curriculum? Does anybody know people in College Board / ETS? Why aren’t there any CS problems on the SAT? What has to take place in order for this change? Does the assessment industry even know if it is supposed to be leading or following the educational mission?
What does the K12 system produce? There are any number of outputs we can discuss, but the one that makes sense to focus on is adequately-prepared college freshmen. This is still too vague. Secy. Duncan’s “Career or college-ready” criteria is to focussed on the individual student, and neglects the macroeconomic needs of our economy. Do we need more Chemistry majors? BLS says “no”. Do we need more Biology majors? Again, “no”. Do we need for CS majors? “Yes, by a factor of 2-3X”.
So, the two themes where we can push hard:
College Board / ETS: Negotiate a time-frame for including Computational Science questions on the SAT. If they refuse, demand to know why they should be taken seriously as measuring what students need to know in the 21st century? Make them explain why their assessments are defining CS “out of the core”. And demand to know, who is responsible for keeping SAT up-to-date with evolving knowledge content?
Dept of Ed, NSF, Congress, States: Get people focussed on defining the K12 “deliverable” in terms of quantities of college freshmen, having the right numbers and the right discipline mix, as derived from BLS estimates of the nation’s skillbase needs.
Clearly, anyone who is looking at this metric would have to conclude the K12 is failing rapidly at what would seem to be its global purpose.
We should be outspoken against lesser definitions of the job of K12. If it isn’t about filling the college pipeline with the human resources estimates dictated by our future economy, then what is it’s job?
You answered all the questions correctly. Great job in explaining your observations without getting into too many details, but providing enough information that would probably allow another analyst to validate your findings. Striking this balance is hard, and I think you did this well.