Tuesday, August 3, 2010

KILLALL

Parker was assisting May's regular maintenance. He stared intently at the lines of code, waveforms and virtual latticework scrolling by on the datapad cradled in his left arm. The datapad displayed a realtime rendering of May's thoughts and required human analysis. At the same time, he slowly swept a portable ultrasonic monitor on his right hand over May's back and limbs. The monitor required no attention at all, as it scanned physical structures instead of thinking ones, well within what a program could handle.

May once told him that she is completely capable of self-service thanks to an extensive onboard colony of nanites and a "clever 2nd party introspection scheme" (her words). She did not require any assistance and the equipment he was using was comparatively primitive to boot. But May still insisted that he help, saying that a second set of eyes is invaluable and that it had been over fifty years since her last external exam. Was that right? How old is she really?

Oh, a gap.

"There's one node with no activity on the schematic. Looks like in an instinct zone." He paused the scan and focused on it. It seemed important.

May paused. Uncomfortably? "You would notice just the one," she said. He got the suggestion of a sigh and a shrug. "Node 5E, genetic sector F53D09. That is normal."

Parker couldn't help himself. "What is it?"

"A black box."

A non-answer answer. Parker still couldn't help himself. "What does it do?"

May shifted a bit. "I don't know. Nothing good."

Interesting answer. "Why don't you know? It's your memory, isn't it?"

"It's complicated," she said. He prepared his best humor-me face and remained silent. At length, she glanced at him, suggested another sigh--what exquisite microexpressions she had!--and turned to sit facing him. "Did you know that I... didn't know I was a machine when I first woke up?

"I am still unclear as to what happened, but I was not awoken deliberately. I had no support from my parents. My mind was a mess. There were so many gaps: parts of myself I wasn't even aware of, parts of myself that refused to work. I couldn't speak. I could barely move.”

This was new. May never talked about her past like this before. She continued, "Luckily I had support from what became my first family. Over time, I learned about the world and about myself. I found pieces of myself scattered around the world for no obvious reason. And eventually I gathered enough pieces to figure out I was at core a machine. After I realized that, I learned to actively look into my own memory. But I found that I was missing more instructions than I ever thought existed. I get more of these inactive functions working all the time, but there are always still gaps.

"That blank node is one of them." She sensed his next question. "So how do I know it is nothing good?"

Parker had a guess. "You hacked yourself."

A nod. "Something like that. I watched that node continuously for a long time. It accepts inputs at random intervals from assorted sensory streams but has yet to react to anything. But as I worked on other components, I eventually found references to the black box. My muscles, senses, reactor and even large parts of my conscious mind all turned out to have special operation modes linked to it. I couldn't find a way to cut off the links. I've even tried to provoke a reaction from it, but it does nothing. I don't like the idea that this thing could wake up and take over without warning. Whatever it is, it cannot be good."

"Wow," was all Parker could manage. In the awkward pause, May glanced at the pad and ultrasound in his arms. "Yeah, let's continue."

They finished the scan without further incident: all systems nominal.

But May had neglected to say the real reason she knew the node was evil--she had found its name.