Role of Insight in Science
Hypothesis and observation, or theory and empiricism, are only two of the three essential ingredients of modern scientific method. The third pillar of wisdom is insight -- the sudden transcendence of obstacles by a new perspective, like the river transforming to vapor and crossing the desert. Insight changes the counterpoint of hypothesis and data into the upward spiral of hypothesis, data, insight, new hypothesis, different data, . . .
Other terms are used synonymously with ‘insight’: illumination, intuition, serendipity, scientific hunch, revelation, inspiration, enlightenment, sudden comprehension, guess, and discovery. Most of these terms have such a heavy connotation of either religious, psychological, or everyday secular meaning, however, that they are somewhat distracting to use in the current scientific discussion.
Insight brings joy to science. Without this thrill, many of us would not be scientists.
In their excellent and still timely article on the role of ‘scientific hunch’ in research, Platt and Baker [1931] define a scientific hunch as “a unifying or clarifying idea which springs into consciousness suddenly as a solution to a problem in which we are intensely interested. . . A hunch springs from a wide knowledge of facts but is essentially a leap of the imagination, in that it goes beyond a mere necessary conclusion that any reasonable man must draw from the data at hand. It is a process of creative thought.” This is not deduction, but induction -- and sometimes induction totally unwarranted from the available evidence. Sometimes it is a solution to a minor technical problem, and sometimes an insight so fundamental that we can never again see the world in the old way.
Helmholtz [1903], Wallas [1926], Platt and Baker [1931], Sindermann [1987] and others concisely describe scientific method as consisting of four stages: preparation, incubation, illumination, and verification. I agree that these four stages are real, and we will come back to them soon in considering how insight can be encouraged or hampered. These four terms betray a strong bias, however, toward casting illumination as the central and most important aspect of science, with the other stages serving only a supporting role. Such a view is by no means universal. Indeed, when 232 scientists replied to a questionnaire concerning insight in science, 17% said that scientific revelations or hunches never help them find a solution to their problems [Platt and Baker, 1931]. I suspect that they rely on insight as much as I do, but they dislike the connotations of the words ‘revelation’ and ‘hunch’, and they prefer to think of science as more rational than those terms imply. Possibly also, they shy from insight's ‘nonscientific’ characteristics: it is nonreproducible, nonquantifiable, unpredictable, unreliable, and sometimes almost mystical.
Scientists’ reliance on insight is incredibly diverse, partly because of variations in ability but also largely because of value judgments concerning rational data-gathering versus irrational insight. Some get ideas and experiment to test their ideas, some prefer to test others’ hypotheses, and some try to gather data until an answer emerges as virtually proved. A few types of research claim to thrive on minimal insight. For example, C.F. Chandler said that one could solve any problem in chemical research by following two simple rules: “To vary one thing at a time, and to make a note of all you do” [Platt and Baker, 1931]. To many scientists, such an approach is either infeasible or boring.
The four stages of research occupy unequal proportions of our research time. We might wish that insight were 25% of the job volumetrically as well as conceptually, but Thomas Edison’s generalization is probably more accurate: “science is 99% perspiration and 1% inspiration.” At least that is what a former advisor told me when, as a new graduate student, I showed little enthusiasm for spending countless hours doing routine measurements for his project.
