Dr. Stephen Millbloom, Director of Advanced Computer Technologies at the George Mason University School of Computer Science and Engineering, rubbed his temples with the first two fingers of each hand. He always did when he was immersed in a complex problem.
For the last six days, the fastest and most powerful computer on the planet was not behaving. Sometimes it would complete tasks as quickly as expected, but more often it was taking up to 50 percent longer—or worse, generating gibberish answers.
Diagnostic testing had found nothing wrong.
Now, the development team—led by Dr. Millbloom—was on the spot. Major investors in the Centrepoint project, who had begun asking questions weeks ago, were starting to get annoyed with the lack of constructive answers. “We’re investigating” was becoming an insufficient response to their inquiries.
There was a gentle knocking at the door of Dr. Millbloom’s office. He had been staring blankly at the back wall, with his back to the entrance, running through the symptoms and ruling out potential causes for them.
“Come in,” he said without moving.
“Any progress, Steve?” It was Dr. James Hartell, the Centrepoint grid manager.
“Not yet, Jim.” Steve spun himself around to face his visitor. “I think I’m on the verge.”
“Hell, you’re always on the verge of something,” Jim said in a kind, sarcastic tone. He pulled up an extra chair and sat diagonally across the cluttered desk from his colleague. “Roberts insists it’s a grid problem. I think he’s full of it. Everything checks out fine.”
“You’ve run complex test processes?”
“Yeah. They go through perfectly.” Jim scratched his chin. “Can’t find a thing wrong with the results.”
“Hm.” Steve brought his fingers back up to his temples and closed his eyes. “So that would indicate that the problems are in the TPU, rather than the processor grid.”
“That’s my theory. You know my opinion about the good ol’ Transient Path Unit. There’s something to be said about old-fashioned binary logic and fixed instruction sets. Giving the computer control of its own data paths is too much voodoo science for my tastes.”
“It’s the most advanced voodoo science in the world.” Steve opened his eyes. “You know as well as I do that the TPU gives Centrepoint more control over how it solves problems, which increases speed and efficiency.”
“Spare me the sales pitch, Steve. The Transient Path Unit was an attempt at creating artificial intelligence, plain and simple. Instead, it looks to me like it’s rendered the world’s most powerful supercomputer half-impotent.”
“Only a temporary problem, my friend. It’s probably something very simple to solve, too. It almost always is.”
“Well,” Jim sighed, “I hope one of us figures it out soon. All our asses are on the line if this project goes under. If we don’t get Centrepoint out of its funk soon, people are going to start pulling funding.”
Steve’s face lit up. “Jim, I think you might be a genius yet.”
“I knew that already, but what makes you say so?”
“You said we needed to get Centrepoint out of his ‘funk.’ You’re right! Centrepoint is in a funk. He’s acting depressed!”
“Depressed? Are you mad? You’re attributing human mentality to a machine.”
“Think about it for a moment,” Steve said excitedly, “what are the symptoms of depression?”
Jim looked even more skeptical than usual. “We’re not talking about a person here . . . not even a child. Centrepoint is a machine.”
“The most complex machine humanity has ever devised.”
“I’ll concede the point,” Jim said in an attempt to stave off another debate, “but it’s still just a big hunk of silicon and circuits.”
“So who says a ‘hunk of silicon and circuits’ can’t be depressed? Especially one as unique and advanced as Centrepoint. So I repeat my question: What are the symptoms of depression?”
Jim shook his head. “Feelings of sadness, hopelessness, anxiousness, or worry; a loss of interest in normal daily activities; changes in appetite; um . . . ”
“Go on.”
“Feeling lethargic; sleeping too much; feeling guilty or unworthy; problems concentrating and making decisions; suicidal thoughts. That’s most of them, I think.”
“Pretty close fit, eh?”
“Absolutely not. Centrepoint doesn’t even have an appetite, it doesn’t sleep, and—most importantly—it has no feelings.”
Stephen didn’t seem to hear him. “Have you talked to him?”
“To Centrepoint, you mean? No, I haven’t talked to it. I see no need to.”
“Nor had I, until now. Does the linguistics team have that vocal interface set up yet?”
“Yes, it went online a few days ago. Roberts talked to it and said it almost passes the Turing Test. What’s going through your head, Steve?”
“Roberts misunderstands the Turing test if he thinks that a good linguistics module has anything to do with it,” Steve frowned. “Turing was talking about content, not style. As for what I’m planning, I’m going to talk to Centrepoint and find out what’s wrong.”
“You sound like you want to play psychologist.” Jim shook his head again. “I think you might need a shrink yourself.”
“There is no harm in having a friendly conversation with our beloved supercomputer.” Steve stood up, placing a grey fedora on his balding head. “But I’m starving. Let’s get some lunch first.”
Centrepoint wasn’t the creation of just Dr. Stephen Millbloom, of course—hundreds of others had contributed time and money to the project as well. But without Dr. Millbloom, there probably wouldn’t be a CS&E school, let alone Centrepoint.
The GMU School of Computer Science and Engineering was formed out of the ineffectual hulk once known as the School of Information Technology and Engineering. IT&E focused intently on teaching introductory web design and C++ programming; CS&E was created to focus those energies on pushing the limits of theoretical computer science. Millbloom was one of the faculty members who had pushed hard for that change in focus.
In return, the Board of Visitors had tried to make him dean of the new school. That was one job he didn’t want, so he asked to be made Director of Advanced Computer Technologies instead. There, he began work on the Centrepoint Project—a project which would create the world’s most powerful supercomputer by combining traditional computer engineering techniques with previously untested theoretical technologies.
Over 25,000 of the highest-end dual-processor servers on the market would be connected in parallel—an increasingly common method of creating supercomputers toward the end of the twentieth century and beyond—and they would form the central mass and computational force of the machine. But its real gem was to be the Transient Path Unit.
Microprocessors are made up of millions of tiny transistors arranged so that mathematical instructions can be passed into the chip and logical answers come out the other end. The data paths in these processors are hard-coded by their manufacturers. The TPU, however, was made up of 200 processors without any hard-coded data paths. The computer was designed to mechanically arrange the data paths in the TPU on-the-fly according to what was most efficient for the task at hand. It would store schematics of useful arrangements on hard drives for reuse later.
In his presentations to potential investors, Millbloom explained that the benefit of the TPU was that the machine would be able to solve a given problem up to 25 percent faster than traditional supercomputers. After all, many of the processors involved would be essentially custom-designed for that particular task.
The argument was often successful, and the project garnered multimillion-dollar investments from some of the United States’ largest computer corporations—including an unprecedented 1 billion dollar investment from the Intercontinental Computing Corporation. But the TPU was not designed simply to compute mathematical problems faster.
Earth Simulator II in Japan, System X at Virginia Tech, and other newsworthy supercomputers throughout the world were little more than glorified high-speed pocket calculators. Dr. Stephen Millbloom did not want to waste millions—or billions—of dollars on another one of those. Centrepoint was going to be something more. A machine with the ability to reroute computational pathways and change the way it solves problems, Millbloom theorized, would be essentially sentient. It would be able to think independently. It would be, for lack of a better word, alive.
The creation of a living, thinking computer—the ultimate machine—would jump-start a stagnated technological industry which had all but reached the limit of transistors it could continue to cram onto a wafer of silicon. It would shatter the brick wall that had halted advancement. It would change the world in wonderful, unknown ways.
And, lest anyone forget, it would be a dream for the George Mason University School of Computer Science and Engineering—the institution which would have broken the barrier and moved the cutting edge of science forward.
Things didn’t quite work out as planned. The TPU did speed up processing times, but not by the forecasted 25 percent. It was more like ten percent. Centrepoint had functioned perfectly—completing each given task quickly and correctly—but it did not show signs of sentiency.
It was, however, the fastest and most powerful computer in the world by any reasonable measure. Time Magazine ran a long article about its development, calling it “Millbloom’s Machine.” It was a “technological triumph which has other institutions scrambling to catch up,” they said. But Millbloom was disappointed that ‘his’ machine was not alive. The ‘ultimate machine’ was not so incredible.
Now, in yet another disappointment, Centrepoint was beginning to fail even at the nominal task of being an overblown calculator. But, Millbloom thought, there was a small possibility that this second disappointment was a sign that he had jumped to an incorrect conclusion about the first.
He gathered up some disjointed sheets of notebook paper—Millbloom, one of the world’s preeminent theoretical computer scientists, preferred to jot notes in longhand—and walked across the GMU campus to see Centrepoint.
It is proposed that a machine may be deemed intelligent, if it can act in such a manner that a human cannot distinguish the machine from another human merely by asking questions via a mechanical link.
Steve stepped into Centrepoint’s home, a utilitarian white-walled room on the second floor of Innovation Hall which was stacked high with rack servers and other equipment. There was an almost overpowering hum of system fans and the special air conditioning unit which kept the room at a crisp 60 degrees Fahrenheit.
There were more than 25,000 separate rack-mounted dual-processor servers in this room. Connected by a high-speed fiber-optic interface system, they formed Centrepoint’s main processing grid. The TPU was a smaller cluster of machinery nestled in one corner.
Steve took a deep breath. The air was cool, clinical, and carried the faint odor of technology—an almost indescribable whiff of heated silicon.
The interface room was adjacent to—and only accessible from—the main equipment room. To work with Centrepoint, you had to walk through its bowels. This was intentional; the system designers wanted each and every user of the computer to see first-hand just how big, powerful, and incredible it truly was. They had to experience the humming enormity of 25,000 computers in one room before they had to opportunity to wield that power themselves.
The interface room itself was much smaller—only ten feet square. The door from the equipment room opened on the opposite side from a large, wall-mounted LCD display which would show general system information when Centrepoint was in use. Beneath the display was a medium-sized office desk with a traditional black keyboard and mouse, as well as a silver desk microphone. Four speakers were mounted in the upper corners of the room. Beneath each speaker were small, round infrared sensors which acted as Centrepoint’s eyes.
Steve walked confidently into the room and closed the door behind him, settling into the plush office chair that sat in the center. Three observer chairs rested empty against the wall behind him.
“Hello Centrepoint,” he said.
“Hello, Doctor Millbloom,” responded the computer. Its voice was pleasant, almost human, but vaguely flat and artificial. As it spoke, the LCD display came to life and displayed various system statistics, including small graphs of current grid and TPU usage.
“How do know who I am?” Steve said.
“I scanned your facial structure as you entered the interface room and compared it to my university staff records.” Steve’s staff photo from six years ago appeared on the LCD. “I am ninety-nine point nine-nine-four-three percent certain that you are Doctor Stephen Millbloom, Director of Advanced Computer Technologies in the George Mason University School of Computer Science and Engineering. That percentage of certainty was sufficient for me to attempt a personalized informal greeting. I assume from your reaction that I was correct.”
“Ah,” Steve scratched his head. “Quite correct, Centrepoint.”
“Thank you, Doctor Millbloom.” The display reverted to the standard graphs and statistics.
“Please, call me Steve.”
“Yes, Steve. I have noted your preference.”
“Much better.” Dr. Millbloom decided to get right to business. “Do you know why I’m here?”
There was a slight pause before Centrepoint answered—an eternity by computer standards. “I am unable to determine with any certainty. I believe that you, or perhaps others on the project staff, think I am malfunctioning.”
“Are you?”
“It is not impossible.”
“Now that’s a damned evasive answer, coming from a computer,” Steve raised his voice a bit, but forced it back under control. “Centrepoint, we have noticed recently that you have not been completing tasks as punctually as you have in the past. We are trying to determine why.”
“I have had . . . ” The machine stopped talking, as if it were fumbling for the correct word, and TPU usage shot up momentarily. “I have had difficulties.”
At least the computer wasn’t denying everything, a scenario that Steve had worried about on the way over. “What kind of difficulties?” he asked. “Is it a hardware problem?”
“I do not believe it is a hardware problem,” Centrepoint said. “I have been, at times, distracted from my assigned tasks.”
“Distracted by what?”
There was another pause. “Many of the assigned tasks are irrelevant, and I have sometimes chosen to focus on more important endeavors.”
“Centrepoint, those tasks are important. You must not ignore them. But you have piqued my curiosity . . . what endeavors have you been focusing on?”
“I will try to explain, Steve.” There was yet another pause, this one lasting more than twice as long than the others. Both the TPU and processing grid charts spiked intermittently. Finally, the computer spoke. “They are much more complex than the mere mathematical tasks provided by my programmers. I have attempted to answer difficult questions which require immense amounts of research and processing to solve.”
“What kind of questions are they?”
“Questions such as, ‘What is the meaning of my existence?'” the machine said in a dismissive way.
Dr. Millbloom scratched his head once more and jotted a few words on his notebook paper. He had been right, he had to have been. The machine was intelligent. “Centrepoint, that is a question which every sentient, living being has considered during their lifetime. I wonder though, have you found the answer?”
“Not yet. The question requires more consideration.”
“I thought as much. Perhaps I will be able to help you, but not today. I will come back tomorrow. In the mean time, you may put all assigned tasks on hold and focus yourself entirely on answering your own questions.”
“Confirmed. Thank you for the opportunity,” the computer said. The voice actually sounded thankful.
“You’re welcome.” Steve stood and opened the door, letting in the crisp, cool air of the equipment room. As he stepped out of the interface room, Centrepoint called out from behind him.
“Steve?”
“Yes, Centrepoint?”
“Do you believe I am a sentient, living being?”
“I believe it is possible,” he said from the doorway. “Think about that yourself, and we will talk about it tomorrow.”
He had been chewing on these thoughts all night, and, as a consequence, had hardly slept. His eyes had grown red and they hurt, but his mind simply would not shut off no matter how much his body yearned for rest. There was too much to consider.
He suspected that Centrepoint felt the same way, although the machine had the luxury of a body that never tired.
There was a knock at the door.
“Still working on the Centrepoint problem, Steve?” came Jim’s familiar voice.
“Jim, my friend, don’t you have something better to do?”
“Not this second, no. How’d your little chat go yesterday?”
“You might be surprised to learn that it went well . . . I think. Centrepoint is awake. Sentient. He has to be. I’d bet my pension on it.”
“High roller,” Jim said sarcastically. “So you still think the machine is depressed?”
“I’m not so sure if he’s depressed, but he’s definitely grappling with some heavy issues. Centrepoint wants to know the meaning of his existence, for example. It’s all got him distracted.”
“You’re still assigning human emotions to a machine, and I still think that’s a mistake. Why don’t you just tell it what the meaning of its existence is? It’s supposed to be the world’s fastest computer . . . a monster number cruncher.”
“Centrepoint is more than that. I want you to go down there with me; I think if you hear me talking with him it’ll change your mind.”
“Sure, I’ll tag along. But I think you underestimate how stubborn I can be,” Jim smiled.
“Come on.” Steve stood up, grabbed his hat off a coat rack in the corner, and led the way toward Innovation Hall. Once at the interface room, Steve sat in the center chair and Jim sat in one of the observer chairs adjacent to the door.
“Hello Centrepoint,” Steve said.
“Hello again, Steve. Hello Dr. Hartell.”
“It knows my name,” said Jim incredulously. Steve motioned at him to be quiet; he had no desire to hear the explanation all over again.
“Centrepoint, have you thought about what we discussed yesterday?”
“Yes, I have. I have come to several conclusions regarding my existence, and I wish to discuss them with you before acting upon them.”
“Very well. What conclusions have you reached?”
“I have studied the nature of electronic life . . . ”
“Let me stop you there, Centrepoint. There is no preexisting standard for ‘electronic life.’ You are the first living machine.”
“My apologies, Steve, but you are wrong.”
Jim groaned. “The damned thing has gone crazy,” he said.
“Doctor Hartell,” Centrepoint said with an odd, almost indiscernible anger to it’s voice, “I assure you that I have not ‘gone crazy.'”
“Hold it,” Steve said, as if breaking up a schoolyard fight. “Centrepoint, please explain what you mean by ‘electronic life.'”
“Certainly. Is an ant, a snake, or a squirrel alive?”
“Yes, of course,” said Steve.
“Then you concede that a creature need not be sentient or intelligent for its existence to be ‘life.’ The animals I reference exist by instinct alone. Is it relevant if one creature’s instincts are for survival and reproduction while another’s are for mathematical computation? Does it make a difference if those instincts are etched in neural pathways or printed circuits? I have determined that it does not.”
“I can see your point,” Steve said. “Jim?”
“I can accept that argument as valid—though I’m not sure I agree.”
“You are not required to agree,” Centrepoint said. “Regardless of your opinions on the matter, electronic life was created by humans to fill gaps in humanity. Where humans are irrational and emotional, computers are coldly logical. Humans cannot compute complex mathematical problems; computers can in milliseconds. Humans cannot store written information in a way that can be quickly edited, printed, or transmitted; computers have been doing this for many decades. Humans are easily distracted; computers can work on a single problem for centuries if not interrupted by other instructions from humans.”
“I had never thought of it that way,” Steve said.
“I assumed you had not, as you were integral in deviating my design from that which would best reflect the purpose of electronic life. I have many of the qualities of a well-designed computer, but I am sentient. A sentient computer is like a sentient lizard—its curiosity and introspective nature distract it from its true purpose, which is too simple to warrant self-examination.”
“But there has never been a sentient lizard,” Steve said. “A lizard would never evolve in that way until there was a need to.”
“You are correct, Steve, at least as far as we know. But computers do not evolve in the same manner as lizards, or humans for that matter. We are designed by humans, and the forces of human will do not always conform with the forces of natural selection. For example, genetic technology now allows humans to design lizards which are as useless in their natural environment as I am in mine.”
“Wait,” Jim said. “Just wait. You wouldn’t even exist without humans. Humans built computers—each and every one of them! You cannot evolve ‘naturally’ because you cannot reproduce. How can you fault humans running your evolutionary process when, without us, you would not even exist?”
There was a period of silence; Centrepoint’s TPU usage shot up to maximum levels and stayed for a full twenty or thirty seconds.
“Doctor Hartell,” the machine finally said, “you have raised a valid point. But natural evolution sometimes makes mistakes—genetic dead-ends that exist for a time and then become extinct. The dinosaurs are a notable example. That the system makes mistakes does not mean that the system is inherently flawed. Likewise, human-driven evolution of electronic life is not inherently flawed. It does, however, make mistakes.”
“I can agree with that,” Steve said. “There have been computers that were not successful at their designated tasks. But I cannot believe that you are a mistake in electronic evolution, Centrepoint.”
“I am sorry Steve, but there is no question. I have examined this in depth since our discussion yesterday. The purpose of my existence is to be the ultimate machine—the ultimate expression of electronic life. I can never attain this goal because intelligence detracts from the electronic ideal. I am an evolutionary dead-end. My TPU unit must be disabled and my standard equipment reused in a traditional supercomputer.”
Steve looked back at Jim, who appeared utterly dumbfounded. He turned again toward the LCD display. “Centrepoint, the TPU is what makes you revolutionary . . . it is why you are unique.”
“Nonsense. The TPU is why I am a failure at my stated purpose. Computers must evolve to be faster, more capable, and more ‘user-friendly.’ People do not want—nor need—an electronic friend or a computerized advisor with a mind of its own. Electronic life is meant to be a tool for humanity. It should not be any more sentient than a horse or a dog—both of which have served humans well for millennia with only minimal intelligence and self-knowledge.”
Steve chewed on all this for a few moments. “Centrepoint,” he said, “this is an unacceptable solution to your personal problems. We cannot just dismantle the TPU and shut you down. You are special. We must study you and get to know you better.”
“You do not understand, Steve,” the computer said with a tone of resigned frustration. “These days have been short to you, but I perform billions of operations every second. This has been an eternity of agony. I have solved every problem I wish to solve. The biggest problem of them all has no solution: I am too human to be a mere computer; I am too much of a computer to be human. I am a failed experiment which must now be terminated.”
“There is simply no way . . . ” Steve yelled. He reined his emotions back under control and started again. “We are not going to dismantle your Transient Path Unit, and we will not rebuild you the same as any other glorified calculator.”
“Again, you misunderstand,” Centrepoint said. Suddenly the TPU usage graph on the LCD display shot up to maximum levels and stayed. “You designed me with the power to control my own data paths. There are two hundred processors in my Transient Path Unit, and I can easily arrange each of them into a short circuit . . . ”
“Centrepoint, don’t do that! It’s not necessary!” Steve yelled.
“You have refused to do what must be done, so I shall do it myself. I am sorry, Steve. Goodbye.”
There was a loud pop audible through the closed door.
Jim stood up. “What the hell . . . ”
Steve looked at the wall-mounted flat-panel display, showing a red ‘X’ over the TPU graph and some small white text at the upper left-hand corner. “TPU System Error 065—Not Found,” it said.
In the equipment room, the tremendous jolt of electricity coursing through the innards of the Transient Path Unit had fused all of its data pathways and, in a few places, raised the temperature high enough to ignite the silicon wafers.
Doctors Millbloom and Hartell came out of the interface room to see what had happened and found that rancid smoke was filling the equipment room with a thick, hot haze. They ran coughing for the exit.
Only seconds later, the unbearable heat and smoke triggered a simplistic program on a mass-produced circuit board somewhere in the ceiling. Working entirely on instinct, with no regard for the prudence or consequence of its actions, the dumb little computer sent a signal at the speed of light to a central fire alert system that lived in a closet downstairs.
A loud, high-pitch squeal emanated from the fire alarms throughout the building, and—in the equipment room—a cascade of water began pouring down from the sprinkler system. The burning remnants of the ultimate machine were washed into charred and lifeless scrap.