Data: fundamental premises
I rediscovered it recently. Here are its five points:
- One should only take data for a specific purpose; the quantity of data necessary for maintaining historical perspective and a report card is far less than we presently take;
- The value of the flow of information is epsilon less than the value of the flow of products, and the same attention should be paid to making both flows simple, easy to understand, and defect-free;
- Nothing beats talking to people for basic communications, but limited data helps to expand the capability of people to analyze a situation;
- Data collection is almost never free, although the costs are often well hidden;
- Manual data collection may be more valuable than computerized data collection (much as we have learned that manual, Kanban-oriented shop floor control may be preferred to computerized systems); for one thing, it is arguably easier to verify the accuracy of many kinds of data when manually collected and plotted.
While the original was an unnumbered list, I've added numbers to make commenting easier.
How might I modify those premises today?
Seemingly contrary to what I wrote in points 3 and 5, I do understand that automated data collection can be valuable, and I do understand that data helps us avoid subjective biases (even as talking with people helps us avoid missing important insights). I've described elsewhere a case in which people on a production line failed to report the most common problem they saw; when the problem was pointed out to them because it was evident in recorded data, they said, "Oh, that's not a problem; it happens all the time." Triangulation is important, as is paying serious attention to the data, not just letting a computer draw a few conclusions and accepting those conclusions without further thought.
I still stand by point 2 and the related point 4. Most of the organizational systems in which we work can be understood as feedback systems, and information feedback is a key determinant of system behavior in such systems. I would suggest that system dynamics can be a tool to help determine what data is important. That data feedback necessary to make the system dynamics model work well may be just the data needed to make the real system work well.
I'd largely stand by point 1, too. It's tempting to squirrel away all the data we can take and then have it just in case we need it. The problem comes in point 4; it costs time and money to ensure we're getting the data we think we're getting. If we don't need particular data, we're tempted to not worry about its accuracy as much. Then, later, if we do decide we need it, it may be hard to determine what it really means or how accurate it really is, and we may make bad and costly decisions by relying on data we only think we have.
What are your fundamental premises regarding data and its use in organizations?