Where's the Science
On Friday Feb. 19th, students in Bob Waters’ Objects and Design class were given the chance to attend a guest lecture by Greg Wilson, an Assistant Professor of Computer Science at the University of Toronto. While guest lectures are by no means an uncommon occurrence in the College of Computing, Wilson’s talk raised a question applicable to a larger-than-normal percentage of College of Computing students: if we’re computer scientists, where’s all the science?
Walk through the Commons any day of the week, and odds are good that there will be at least one ongoing argument over what language, IDE, or operating system is the “best.” The problem is, how much of the evidence in those arguments would pass muster for a scientific study and how much of it is just hot air? In medicine, before a company can make any kind of claim about a new drug, it has to go through months of rigorous double-blind studies, but no such requirement exists for many areas of computer science, particularly in software engineering. Often, all it takes to convince an audience of a wild claim is to preface it with, “My friend Jim said,” or, “Everyone knows that,” and the speaker’s word is taken as law. After his talk, THE FIREWALL caught up with Wilson and asked him a bit more about his ideas and how they affected students.
FOR STUDENTS WHO DIDN’T ATTEND THE TALK, IN YOUR OWN WORDS, WHAT WAS YOUR TALK ABOUT?
The need for more scientific rigor in software engineering. All too often,people in both academia and industry claim that X is better than Y based solely on some personal experience and a couple of anecdotes. We wouldn’t accept that as proof in medicine or business (well, a lot of people would, but they shouldn’t); those of us who build software for a living should have equally high standards.
YOU TALKED A GREAT DEAL ABOUT THE LACK OF BASIC SCIENCE IN MUCH OF COMPUTER SCIENCE. WHAT DO YOU MEAN BY THIS?
I don’t know about Computer Science as a whole, but there’s certainly not enough “science” in software engineering. Take UML, for example: where is the evidence that drawing class diagrams and sequence diagrams actually results in better or cheaper software? Or arguments over Java versus Python versus whatever your favorite language is today: we’ve all heard people yell about this, but where’s the evidence? Note that I’m not saying different languages don’t matter---I’m saying that if people as smart and as talented as Larry Wall (Perl), Joshua Bloch (Java), and Simon Peyton Jones (Haskell) disagree, it’s time for us to stop yelling and start being systematic.
SINCE IT WILL LIKELY BE YEARS BEFORE STUDENTS AND NEW GRADS ARE IN A POSITION TO MAKE BIG DECISIONS ON HOW TO CHANGE THIS --WHAT CAN WE DO NOW AND AFTER WE GRADUATE TO NUDGE THINGS IN THE RIGHT DIRECTION?
You can start by reading Robert Glass’s excellent little book, Facts and Fallacies of Software Engineering, then get the second edition of McConnell’s equally excellent (but much larger) Code Complete. Between them, they’ll tell you most of what we actually know about building software. More importantly, they’ll introduce you to the studies that have been done, so that the next time someone says, “X is better than Y,” you’ll know to ask them for a citation.
PROBLEMS AFFECT OUR INDUSTRY? HOW WILL THEY AFFECT OUR INDUSTRY IN THE FUTURE?
You can expect to live 25-30 years longer than your great-grandparents. Most of that gain is a result of us learning to treat medicine as a branch of applied science. I think we’ll reap similar benefits from saying, “Prove it!” more often in software engineering.
WHAT ARE PEOPLE IN ACADEMIA AND INDUSTRY DOING TO REMEDY THIS PROBLEM, AND HOW MUCH OF AN EFFECT IS IT HAVING?
A lot of software engineering researchers are now doing this kind of empirical work. It hasn’t had a lot of impact yet, but I think we’ll see that change dramatically in the next 3-5 years.
COULD YOU TALK MORE ABOUT WHAT YOUR BOOKS ARE ABOUT AND HO
I’m editing a new collection, which O’Reilly will publish this summer, that brings together a lot of the primary research in evidence-based software engineering. We’re still trying to decide on a title, but you can follow our progress at http://third-bit.com.
