Charles Spearman

Introduction

Charles Edward Spearman (1863–1945) was the British psychologist who gave intelligence research its central object — g, the general factor — and statistics one of its most consequential tools, factor analysis. Both were the same discovery approached from two ends: he worked out how to look for structure in a matrix of correlations, and what he found when he looked was a single common factor running through every mental test he could measure.

He was also, almost from the beginning, wrong in ways that mattered and right in ways that have outlasted the arguments. The positive manifold he described — the fact that all cognitive tests correlate positively with one another — is one of the most robust findings in the whole of differential psychology. His inference that it must therefore reflect a single underlying ability was contested within five years of its publication, and is still contested.

Spearman is also the figure most responsible for the fact that intelligence research became a quantitative field. His methods for correcting correlations for measurement error became the basis of classical test theory, which is what makes reliability and validity estimable at all.

Early Life and Education

Spearman was born in London on 10 September 1863. His background was not academic; his family was a military one, and he spent fifteen years as an officer in the British Army, mostly in India, before turning to psychology. He resigned his commission in the 1890s to study, which meant that he came to the subject as a middle-aged man with no institutional standing, no patron and no research record.

He was in his thirties before he began formal study, and his training was entirely German — which at the time meant entirely experimental and psychophysical. He studied at Leipzig under Wilhelm Wundt, at Göttingen under G.E. Müller, and at Berlin under Carl Stumpf, the psychologist most responsible for the phenomenological tradition that Gestalt psychology grew out of. He completed his doctorate at Leipzig in 1906, at the age of forty-three.

The biography is relevant to the reception of his work. Spearman published the paper that made his name in 1904, two years before he had a doctorate and while he was still, in British terms, an outsider. Much of the acrimony of the following two decades — the disputes with Thomson, with Thorndike, and eventually with Thurstone — had the character of an argument about who had the standing to interpret a correlation matrix. It is also why he spent so much of his career defending a result rather than extending it: The Abilities of Man is a book by someone still answering 1909.

Two things about this training shaped everything that followed. The first was a conviction, learned from the German laboratories, that psychology could be quantitative and should be. The second was the specific technical problem he encountered there: measurement error. Correlation coefficients computed between two fallible measures systematically underestimate the true relationship between the underlying quantities. Spearman's doctoral work was substantially concerned with how to correct for this.

In 1907 he joined University College London, working under James Sully, and in 1911 was appointed to a chair in psychology — a position he held until his retirement in 1931. He was elected a Fellow of the Royal Society in 1924.

Academic and Professional Career

The paper that made his name was published in 1904, two years before he had a doctorate: "General Intelligence, Objectively Determined and Measured." It did three things at once. It set out the correction procedures for attenuation by measurement error. It applied factor analysis — a technique he was inventing as he went — to a correlation matrix of ability tests. And it proposed the two-factor theory.

His books developed and defended the position across the rest of his career: The Nature of Intelligence and the Principles of Cognition (1923), The Abilities of Man (1927), a substantial attempt to answer his critics and assemble the evidence for a single factor, and Psychology Down the Ages (1937). He also edited the British Journal of Psychology for many years, and used it as a platform for the argument.

Major Contributions to Psychology

The positive manifold and g

The observation is simple and has never been contradicted: people who do well on one cognitive test tend to do well on others, and the correlations are all positive. This is not a trivial result — there is no reason in advance why vocabulary, mental rotation, arithmetic, and memory span should be positively related at all.

Spearman's explanation was that performance on any mental test reflects two things: a general factor common to all of them, which he called g, and a specific factor peculiar to that test, which he called s. The variance shared among tests is g; the rest is s plus error.

He never claimed to know what g is. His own suggestion, in 1927, was that it might be something like mental energy — a proposal that has not aged well, and which he advanced with more caution than it is usually quoted with.

Factor analysis

The method mattered more than the conclusion. Spearman generalised the correction for attenuation into a general technique for decomposing a correlation matrix into latent sources of variance, and formulated a testable prediction: if a single general factor accounts for all the shared variance, then for any four tests the products of the opposite correlations should be equal. This is the law of tetrad differences — r₁₃·r₂₄ − r₂₃·r₁₄ = 0 — which he demonstrated in 1927.

It was the first time a substantive claim about the structure of the mind had been stated in a form that could be falsified with arithmetic. Factor analysis, in the hands of Louis Thurstone and his successors, became the standard tool for structural questions throughout psychology, and tetrad-style predictions remain a live technique in causal discovery.

Classical test theory

Spearman's work on correcting correlations for measurement error — the decomposition of an observed score into a true score and an error component, and the treatment of reliability as the ratio of true to observed variance — is the foundation of classical test theory. Every reliability coefficient and every correction for attenuation in use today descends from it.

He also introduced the rank correlation coefficient now known as Spearman's rho, for cases where the relationship is monotonic but the data are not suited to Pearson's coefficient.

Major Publications

"General Intelligence, Objectively Determined and Measured" (1904, American Journal of Psychology) — the founding paper: reports the positive manifold and proposes that it is explained by a single general factor.
"The Proof and Measurement of Association between Two Things" (1904, American Journal of Psychology) — introduces the rank correlation coefficient now known as Spearman's rho.
The Nature of Intelligence and the Principles of Cognition (1923) — the full statement of the two-factor account, in which any test score reflects a general factor g and a specific factor s.
The Abilities of Man: Its Nature and Measurement (1927) — the book in which he ventured that g might amount to something like mental energy, a proposal he advanced more cautiously than it is usually quoted.
Psychology Down the Ages (1937, two volumes) — his own survey of the discipline's history, written at the end of his active career.

Impact and Legacy

Every intelligence test in use is a descendant of Spearman's problem, and every one of them reports a composite score on the assumption that some general factor is worth measuring. The human intelligence literature, the g-loadedness literature, and the enormous body of work on the correlates of cognitive ability all rest on the positive manifold he first measured properly.

Criticism — and it began immediately. The first psychologist to raise problems was Cyril Burt, who pointed out in 1909 that Spearman's analysis had not accounted for mental effort, and who assembled a larger data set showing that a single general factor could not explain the correlations: the tetrad equation was not satisfied. Spearman knew about objections of this kind as early as 1906 and attempted to dismiss them by arguing that the offending tests were not meaningfully distinct — a defence that is difficult to evaluate, because the criterion for what counts as "meaningfully distinct" was supplied by the theory being tested.

Godfrey Thomson accepted Spearman's data and methods and offered a different interpretation: the mind consists of a very large number of independent bonds, and any given test samples some subset of them. On that account, positive correlations arise without any general factor at all, simply because tests that sample overlapping sets of bonds will correlate. This is now called sampling theory, and it remains the principal alternative explanation of the positive manifold. Edward Thorndike developed a related multi-factor account, arguing that different tests draw on overlapping but distinguishable cognitive processes.

The methodological objection is deeper than any particular counter-theory. Factors extracted from a correlation matrix can be rotated without changing how well they fit the data, which means the number of factors is underdetermined by the evidence unless additional criteria are imposed. Thurstone's simple-structure criterion was one such imposition, and it produced multiple factors from data Spearman had read as one. That the same matrix supports both readings is a fact about the method, not about the mind.

A modern version of the sampling argument has revived the debate. Process-overlap theory holds that cognitive tests draw on a small set of executive processes plus domain-specific ones, and that the positive manifold arises because the executive processes are shared rather than because a single general ability exists. The theory reproduces the main psychometric findings — including the positive manifold and the hierarchy of factors — without positing g, and it has been shown to fit large data sets at least as well as the g-centred models. Whether that means g has been explained away or merely re-described is exactly the disagreement Spearman and Thomson were having in 1910, conducted with better data.

There is also a loose end in his own results that he noticed and never resolved. The correlations among tests are not equally strong at every level of ability: the manifold appears to be weaker among higher-ability examinees, so that the structure looks more differentiated at the top. This is now called Spearman's law of diminishing returns or ability differentiation. Its existence is disputed, and where it is found it complicates the claim that a single factor of fixed importance runs through all cognitive performance.

There is also the unresolved question of what g would be, even if it exists as a statistical entity. A factor is a description of covariance structure, not a mechanism; the inference from "a single factor accounts for a large share of the variance" to "a single ability causes performance" is an interpretation, not a result. The debate that Gould's The Mismeasure of Man conducted in public had already been running in the psychometric literature for sixty years, and it has not closed.

Finally, Spearman's legacy is entangled with Burt's. Burt was his ally and successor in the British intelligence-testing tradition, and the posthumous discovery that Burt's twin data were fabricated — including, apparently, data reported jointly with Spearman's associates — cast a shadow over the whole research programme. There is no evidence that Spearman fabricated anything. But the episode demonstrates a hazard inherent in the subject: when the conclusions bear on disputed questions of heredity and policy, the field's evidentiary standards become as consequential as its findings.

Learn More

Louis Thurstone — who generalised factor analysis and argued for multiple abilities instead of one
Alfred Binet — who built the first practical intelligence scale, and warned against what it became
John C. Raven — who built the Progressive Matrices specifically to measure one component of Spearman's g
Intelligence Testing — the field Spearman's methods created

References

Spearman, C. (1904). "General intelligence," objectively determined and measured. American Journal of Psychology, 15(2), 201-292.