116
16) before the experiment is completed, begin preliminary data reduction and analysis.
- Allow time to think about what you are observing, regardless of how busy you are just collecting data.
- Rough, first-order corrections and analysis, including back-of-envelope plots, are acceptable at this stage.
- Determine whether or not you are generating unreliable data.
- Seek clues to needed improvements (e.g., finding a major unremoved variable). While avoiding minor changes, consider the advisability of restarting the experiment with a substantial improvement.
- Beware of potential bias to subsequent results caused by expectations from the preliminary analysis.
- Do not let these preliminary analyses substitute for post-experiment, systematic data reduction and analysis.
- Some experimenters or their managers find it fruitful to write progress reports regularly during the experiment.
17) handle calculations scientifically:
- Omit meaningless digits. Usually the final result will have no more significant digits than the least-accurate variable in the calculation. Carrying one superfluous digit is preferable to omitting a meaningful digit. A propagation-of-errors analysis is even better.
- Average raw data rather than final processed data, to save steps.
- Check your calculations. If using a calculator, use a different keying sequence than for the initial calculation, to avoid making the same mistake twice. If using a computer, check results with a calculator for one or two of the samples. Computers usually make no mistake or make the same mistake for every sample, if they are correctly interpreting the input format of all of the data. However, exceptions exist (e.g., calculations that work OK for data values greater than zero but not for data less than zero).
- Ask yourself whether or not the result looks reasonable. In the old slide-rule days, quickand-dirty estimation was essential; now, this skill is rare.
Subsequent experimental steps are less relevant to the subject of experimental design and can be left to other chapters. These include: analyzing data, interpreting the experimental results, drawing conclusions, comparing these conclusions to those of other studies, and designing a modified experiment to test the conclusions.
***
Pitfalls of Experimental Design
“Faulty execution of a winning combination has lost many a [chess] game on the very brink of victory. In such cases a player sees the winning idea, plays the winning
