The Human Use of Human Beings: Cybernetics Pioneer Norbert Wiener on Communication, Control, and the Morality of Our Machines
By Maria Popova
“Information will never replace illumination,” Susan Sontag asserted in considering the conscience of words. “Words are events, they do things, change things,” Ursula K. Le Guin wrote in the same era in her exquisite meditation on the magic of real human communication. “They transform both speaker and hearer; they feed energy back and forth and amplify it. They feed understanding or emotion back and forth and amplify it.” But what happens when words are stripped of their humanity, fed into unfeeling machines, and used as currencies of information that no longer illuminates?
Half a century before the golden age of algorithms and two decades before the birth of the Internet, the mathematician and philosopher Norbert Wiener (November 26, 1894–March 18, 1964) tried to protect us from that then-hypothetical scenario in his immensely insightful and prescient 1950 book The Human Use of Human Beings: Cybernetics and Society (public library) — a book Wiener described as concerned with “the limits of communication within and among individuals,” which went on to influence generations of thinkers, creators, and entrepreneurs as wide-ranging as beloved author Kurt Vonnegut, anthropologist Mary Catherine Bateson, and virtual reality pioneer Jaron Lanier.
Wiener had coined the word cybernetics two years earlier, drawing on the Greek word for “steersman” — kubernētēs, from which the word “governor” is also derived — to describe “the scientific study of control and communication in the animal and the machine,” pioneering a new way of thinking about causal chains and how the feedback loop taking place within a system changes the system itself. (Today’s social media ecosystem is a superficial but highly illustrative example of this.)
In a complement to Hannah Arendt’s contemporaneous insight into how tyrants use isolation as a weapon of oppression and manipulation, Wiener explains why, under this model of information systems, communication and control are inexorably linked:
Information is a name for the content of what is exchanged with the outer world as we adjust to it, and make our adjustment felt upon it. The process of receiving and of using information is the process of our adjusting to the contingencies of the outer environment, and of our living effectively within that environment. The needs and the complexity of modern life make greater demands on this process of information than ever before, and our press, our museums, our scientific laboratories, our universities, our libraries and textbooks, are obliged to meet the needs of this process or fail in their purpose. To live effectively is to live with adequate information. Thus, communication and control belong to the essence of man’s inner life, even as they belong to his life in society.
A pillar of Wiener’s insight is the second law of thermodynamics and its central premise that entropy — the growing tendency toward disorder, chaos, and unpredictability — increases over time in any closed system. But even if we were to consider the universe itself a closed system — an assumption neglecting the possibility that our universe may be one of many universes — neither individual human beings nor the societies they form can be thought of as closed systems. Rather, they are pockets of attempted order and decreasing entropy amid the vast expanse of cosmic chaos — attempts encoded in our systems of organizing and communicating information. Wiener examines the parallel between organisms and machines in this regard — a radical notion in his day and plainly obvious, if still poorly understood, in ours:
If we wish to use the word “life” to cover all phenomena which locally swim upstream against the current of increasing entropy, we are at liberty to do so. However, we shall then include many astronomical phenomena which have only the shadiest resemblance to life as we ordinarily know it. It is in my opinion, therefore, best to avoid all question-begging epithets such as “life,” “soul,” “vitalism,” and the like, and say merely in connection with machines that there is no reason why they may not resemble human beings in representing pockets of decreasing entropy in a framework in which the large entropy tends to increase.
When I compare the living organism with such a machine, I do not for a moment mean that the specific physical, chemical, and spiritual processes of life as we ordinarily know it are the same as those of life-imitating machines. I mean simply that they both can exemplify locally anti-entropic processes, which perhaps may also be exemplified in many other ways which we should naturally term neither biological nor mechanical.
In a sentiment of astounding foresight, Wiener adds:
Society can only be understood through a study of the messages and the communication facilities which belong to it; and that in the future development of these messages and communication facilities, messages between man and machines, between machines and man, and between machine and machine, are destined to play an ever-increasing part.
In control and communication we are always fighting nature’s tendency to degrade the organized and to destroy the meaningful; the tendency… for entropy to increase.
In consonance with Neil Gaiman’s conception of stories as “genuinely symbiotic organisms that we live with, that allow human beings to advance,” Wiener considers how living organisms resemble and are aided by information systems:
Organism is opposed to chaos, to disintegration, to death, as message is to noise. To describe an organism, we do not try to specify each molecule in it, and catalogue it bit by bit, but rather to answer certain questions about it which reveal its pattern: a pattern which is more significant and less probable as the organism becomes, so to speak, more fully an organism.
We are not stuff that abides, but patterns that perpetuate themselves. A pattern is a message.
Messages are themselves a form of pattern and organization. Indeed, it is possible to treat sets of messages as having an entropy like sets of states of the external world. Just as entropy is a measure of disorganization, the information carried by a set of messages is a measure of organization. In fact, it is possible to interpret the information carried by a message as essentially the negative of its entropy, and the negative logarithm of its probability. That is, the more probable the message, the less information it gives.
Wiener illustrates this idea with an example that would have pleased Emily Dickinson:
Just as entropy tends to increase spontaneously in a closed system, so information tends to decrease; just as entropy is a measure of disorder, so information is a measure of order. Information and entropy are not conserved, and are equally unsuited to being commodities. Clichés, for example, are less illuminating than great poems.
The prevalence of cliches is no accident, but inherent in the nature of information. Property rights in information suffer from the necessary disadvantage that a piece of information, in order to contribute to the general information of the community, must say something substantially different from the community’s previous common stock of information. Even in the great classics of literature and art, much of the obvious informative value has gone out of them, merely by the fact that the public has become acquainted with their contents. Schoolboys do not like Shakespeare, because he seems to them nothing but a mass of familiar quotations. It is only when the study of such an author has penetrated to a layer deeper than that which has been absorbed into the superficial clichés of the time, that we can re-establish with him an informative rapport, and give him a new and fresh literary value.
From this follows a corollary made all the clearer by the technologies and media landscapes which Wiener never lived to see and with which we must and do live:
The idea that information can be stored in a changing world without an overwhelming depreciation in its value is false.
Information is more a matter of process than of storage… Information is important as a stage in the continuous process by which we observe the outer world, and act effectively upon it… To be alive is to participate in a continuous stream of influences from the outer world and acts on the outer world, in which we are merely the transitional stage. In the figurative sense, to be alive to what is happening in the world, means to participate in a continual development of knowledge and its unhampered exchange.
In a passage that calls to mind Zadie Smith’s lucid antidote to the illusion of universal progress and offers a sobering counterpoint to today’s strain of social scientists purveying feel-good versions of “progress” via the tranquilizing half-truths of highly selective statistics willfully ignorant of the for whom question, Wiener writes:
We are immersed in a life in which the world as a whole obeys the second law of thermodynamics: confusion increases and order decreases. Yet, as we have seen, the second law of thermodynamics, while it may be a valid statement about the whole of a closed system, is definitely not valid concerning a non-isolated part of it. There are local and temporary islands of decreasing entropy in a world in which the entropy as a whole tends to increase, and the existence of these islands enables some of us to assert the existence of progress.
Thus the question of whether to interpret the second law of thermodynamics pessimistically or not depends on the importance we give to the universe at large, on the one hand, and to the islands of locally decreasing entropy which we find in it, on the other. Remember that we ourselves constitute such an island of decreasing entropy, and that we live among other such islands. The result is that the normal prospective difference between the near and the remote leads us to give far greater importance to the regions of decreasing entropy and increasing order than to the universe at large.
Wiener considers the central flaw of the claim that the arrow of historical time is aligned with the arrow of “progress” in a universal sense:
Our worship of progress may be discussed from two points of view: a factual one and an ethical one — that is, one which furnishes standards for approval and disapproval. Factually, it asserts that the earlier advance of geographical discovery, whose inception corresponds to the beginning of modern times, is to be continued into an indefinite period of invention, of the discovery of new techniques for controlling the human environment. This, the believers in progress say, will go on and on without any visible termination in a future not too remote for human contemplation. Those who uphold the idea of progress as an ethical principle regard this unlimited and quasi-spontaneous process of change as a Good Thing, and as the basis on which they guarantee to future generations a Heaven on Earth. It is possible to believe in progress as a fact without believing in progress as an ethical principle; but in the catechism of many Americans, the one goes with the other.
With this, Wiener turns to the most gaping void in the narrative of progress — a recognition of the interconnectedness of existence across scales and species, which the pioneering naturalist John Muir so memorably captured a century earlier in his assertion that “when we try to pick out anything by itself, we find it hitched to everything else in the universe.” A decade before Rachel Carson awakened the modern environmental conscience, Wiener considers the larger planetary costs of humanity’s “progress”:
What many of us fail to realize is that the last four hundred years are a highly special period in the history of the world. The pace at which changes during these years have taken place is unexampled in earlier history, as is the very nature of these changes. This is partly the result of increased communication, but also of an increased mastery over nature which, on a limited planet like the earth, may prove in the long run to be an increased slavery to nature… We have modified our environment so radically that we must now modify ourselves in order to exist in this new environment. We can no longer live in the old one. Progress imposes not only new possibilities for the future but new restrictions… May we have the courage to face the eventual doom of our civilization as we have the courage to face the certainty of our personal doom. The simple faith in progress is not a conviction belonging to strength, but one belonging to acquiescence and hence to weakness.
The new industrial revolution is a two-edged sword… It may be used for the benefit of humanity, but only if humanity survives long enough to enter a period in which such a benefit is possible. It may also be used to destroy humanity, and if it is not used intelligently it can go very far in that direction.
Three decades later, the great physician, etymologist, poet, and essayist Lewis Thomas would articulate the flip side of the same sentiment in his beautiful meditation on the peril and possibility of progress: “We are in for one surprise after another if we keep at it and keep alive. We can build structures for human society never seen before, thoughts never thought before, music never heard before… Provided we do not kill ourselves off, and provided we can connect ourselves by the affection and respect for which I believe our genes are also coded, there is no end to what we might do on or off this planet.” Wiener’s most visionary point is that if we are to not only survive but thrive as a civilization and a species, we must encode these same values of affection and respect into our machines, our information systems, and our technologies of communication, so that “the new modalities are used for the benefit of man, for increasing his leisure and enriching his spiritual life, rather than merely for profits and the worship of the machine as a new brazen calf.”
More than a century after Mary Shelley raised these enduring questions of innovation and responsibility in Frankenstein, Wiener offers a sentiment of astonishing prescience and relevance to the artificial intelligence precipice on which we now stand, in an era when algorithms are deciding for us what we read, where we go, and how much of reality we see:
The machine’s danger to society is not from the machine itself but from what man makes of it.
The modern man, and especially the modern American, however much “know-how” he may have, has very little “know-what.” He will accept the superior dexterity of the machine-made decisions with out too much inquiry as to the motives and principles behind these… Any machine constructed for the purpose of making decisions, if it does not possess the power of learning, will be completely literalminded. Woe to us if we let it decide our conduct, unless we have previously examined the laws of its action, and know fully that its conduct will be carried out on principles acceptable to us! On the other hand, the machine [that] can learn and can make decisions on the basis of its learning, will in no way be obliged to make such decisions as we should have made, or will be acceptable to us. For the man who is not aware of this, to throw the problem of his responsibility on the machine, whether it can learn or not, is to cast his responsibility to the winds, and to find it coming back seated on the whirlwind.
At the heart of Wiener’s decades-old book is a point of great timelessness and great urgency, which ought to be inscribed on the mental motherboard of every coder, technologist, and entrepreneur. Eight years after the trailblazing philosopher Susanne Langer considered how the questions we ask shape the answers we give and the world we build, he writes:
When human atoms are knit into an organization in which they are used, not in their full right as responsible human beings, but as cogs and levers and rods, it matters little that their raw material is flesh and blood. What is used as an element in a machine, is in fact an element in the machine. Whether we entrust our decisions to machines of metal, or to those machines of flesh and blood which are bureaus and vast laboratories and armies and corporations, we shall never receive the right answers to our questions unless we ask the right questions.
Precisely because our existence is so improbable against the backdrop of a universe governed by entropy, it is imbued with a singular responsibility — a responsibility that is the source and succor of meaning in human life. In a sentiment which the Nobel-winning Polish poet Wisława Szymborska would later echo, Wiener writes:
It is quite conceivable that life belongs to a limited stretch of time; that before the earliest geological ages it did not exist, and that the time may well come when the earth is again a lifeless, burnt-out, or frozen planet. To those of us who are aware of the extremely limited range of physical conditions under which the chemical reactions necessary to life as we know it can take place, it is a foregone conclusion that the lucky accident which permits the continuation of life in any form on this earth, even without restricting life to something like human life, is bound to come to a complete and disastrous end. Yet we may succeed in framing our values so that this temporary accident of living existence, and this much more temporary accident of human existence, may be taken as all-important positive values, notwithstanding their fugitive character.
In a very real sense we are shipwrecked passengers on a doomed planet. Yet even in a shipwreck, human decencies and human values do not necessarily vanish, and we must make the most of them. We shall go down, but let it be in a manner to which we may look forward as worthy of our dignity.
Nearly a century later, The Human Use of Human Beings remains an immensely insightful and increasingly relevant read. Complement it with the great cellist Pablo Casals on making our world worthy of its children, then revisit Thomas Merton’s beautiful letter to Rachel Carson about technology, wisdom, and the difficult art of civilizational self-awareness.
Published June 15, 2018