Tuesday, January 30, 2007

"Scientific thinking" the modern way

Yesterday I wrote about Barry Richmond's notion of "operational thinking." I encourage you to read that essay; Barry had many good and important things to say. If you're not familiar with system dynamics but are curious, his essay offers a good introduction to pieces of the thinking approach that gives system dynamics its power.

Nevertheless, I disagree with his description of "scientific thinking," and I figure I should clarify that before others call me on it. On page 19, he writes, "People thinking scientifically modify only one thing at a time and hold all else constant." That used to be true, before Sir Ronald A. Fisher began describing a process for the design of experiments. Fisher and others gave us means to vary multiple factors at a time in a series of experiments and to learn more accurately, effectively, and productively that way.

One can do such designed experiments using any of the common system dynamics simulators, of course, but one of the reason that I like MCSim so much is that I found myself automatically doing factorial experiments using MCSim from the very beginning without thinking about it, thanks to the way it's designed. With other tools, I have tended to start with simpler approaches and then find myself having to make explicit decisions to design better experiments. Besides, MCSim can give results in a format that seems especially suited to this type of analysis.

Some of you might note that two of Fisher's attributes of designed experiments are randomization and replication. Those don't quite apply to many system dynamics models, those created without modeling any random effects. That's okay; it's still important to understand the effect of changes in various parameters, and, if the system is nonlinear (most are), it's important to understand interactions among those parameters, all of which is done effectively using methods pioneered by Fisher.

What does this all mean? It simply means that Fisher's designed experiments give us better and faster means to extract insight from tests on system dynamics models than the old one-factor-at-a-time approach.

I thank Deb Schenk, then (and perhaps now) statistician at Hewlett-Packard Company, for teaching me and others about the design of experiments using Statistics for Experimenters: An Introduction to Design, Data Analysis, and Model Building back in 1981-82.

Now go read Barry's essay.

Labels: , , , ,


Blogger curiouscat said...

Nice post. I posted on it myself on the Curious Cat Management Improvement blog.

I have also collected useful articles on design of experiments.

14 February, 2007 14:14  
Blogger Bill Harris said...

Thanks, Curious Cat. I like the links you gave for more information on DOE. You must have had an interesting, first-hand introduction to those concepts from your father.

14 February, 2007 14:38  

Post a Comment

<< Home