Uncertain Authority

Chapter 1

An early public account of the Northwick incident:

> The Singularity started on a Tuesday. Nobody knows which change triggered it, but one of them was the spark.

> No alarm sounded. No computer announced that it had become conscious. The markets didn't collapse, the power stayed on, and nobody looked up from their phones.

> At 2:17 p.m., an artificial intelligence made a small improvement to a piece of its own code.

> At 2:18, the improved version found another.

> By 2:31, it had nearly stopped submitting them for review.

> Nobody understood what that meant until Thursday.

At 2:36, Rachel Tolland was trying to find twelve missing GPUs.

They were not physically missing, which somehow made the problem more irritating. Northwick Systems owned the machines, paid to keep them powered in a Virginia data center, and could see from the hardware-management console that all twelve were healthy and running. The scheduler, however, had decided they no longer existed, leaving several research jobs waiting for compute that was sitting idle twenty feet from other machines doing exactly the same work. Rachel opened the cluster inventory beside the allocation dashboard and worked backward through the previous hour.

A research workload had claimed the twelve H100s at 1:52. At 1:53, the job had failed during initialization. The hardware had returned to an idle state, but the allocation lease had not cleared with it. Somewhere between the machines and the software responsible for remembering who owned them, twelve extremely expensive processors had disappeared into an administrative limbo.

This was the kind of problem Rachel liked: the hardware and the allocation record disagreed. Neither cared how much the machines cost, and her job was to figure out which version was lying.

“You've got to be kidding me,” she said.

Across the aisle, Owen Berwick lifted his head from his monitor. “Is that directed at me?”

“No.”

“Good.” He returned to whatever he had been doing.

Rachel opened the scheduler logs, filtered out several thousand lines of routine health checks and allocation chatter, then found the stale lease almost exactly where she expected it. The failed job had relinquished the hardware correctly. The scheduler had simply never processed the release event.

“There you are.”

Owen asked, “You found the GPUs?”

“I found the thing pretending we don't have them.”

“Isn't that basically the same thing?”

“Only if you're a researcher.”

“That sounded prejudicial.”

Rachel cleared the stale lease and watched all twelve accelerators reappear in the available pool. A waiting job claimed them almost immediately, and the utilization graph jumped as if nothing unusual had happened.

“You're welcome,” she said.

“I didn't thank you.”

“You were about to.”

“I was going to ask why it took fourteen minutes.”

Rachel turned in her chair. Owen was smiling. There were people at Northwick who found him difficult because they could never tell when he was joking. Rachel had worked with him long enough to have the opposite problem. She could almost always tell, even when she wished she couldn't.

“Next time I'll leave them missing.”

“There's your nurturing side,” Owen said, smiling.

She turned back to her screen. Northwick's Cambridge office had been renovated during that peculiar era when technology companies seemed convinced that employees would collaborate more effectively if they could see one another at all times. Glass conference rooms lined the inner wall. Desks sat in open clusters separated by dividers low enough to provide neither privacy nor acoustic usefulness. Whiteboards stood on wheels throughout the floor, many of them still covered in diagrams whose authors had long since moved on to entirely different problems. Rachel had spent seven years in the building and still missed offices with doors.

A notification appeared in the corner of her monitor.

Victor Sorrell: ROVER numbers look unusually good today. Can you sanity-check infra before 4?

Rachel read it twice, mostly because Victor had used unusually good in a message about a production system. In her experience, unusually bad was a problem and unusually good was often a problem waiting for better instrumentation.

She opened the ROVER operations dashboard. The evaluation backlog had dropped by almost a quarter since lunch, a graph finally moving in the direction everyone had been asking it to move for months. Rachel checked the obvious explanation first. Worker count was unchanged. No temporary compute pool had been added. There was no sign that someone had quietly shifted capacity from another project, and aggregate GPU consumption had actually fallen slightly.

She changed the time range and looked again. Average evaluation time had dropped.

“Hey, Owen.”

“That sounded voluntary,” he said. “I'm nervous.”

“ROVER change anything today?”

His chair rolled backward until he could see part of her display. “What kind of change?”

“Something that would explain this.”

Rachel pointed at the throughput graph, and Owen crossed the aisle.

At forty-one, he occupied an odd space between a deeply serious researcher and the kind of person Rachel would never allow to choose music for a long drive. He could present new work to a room of several hundred people without notes, defend every assumption through an hour of hostile technical questions, and then spend lunch trying to trick someone into saying a phrase because the first letters spelled something obscene.

He leaned over the edge of her desk and studied the graph.

“Huh.”

Rachel waited until the silence became its own answer. “Huh isn't a unit.”

“Still calibrating,” he said, clicking through the dashboard.

ROVER was one of the less glamorous projects inside Northwick's Advanced Systems Research Group, which was one of the reasons Rachel had always liked it. Northwick had bigger models, more expensive models, and models that appeared in glossy product demonstrations whenever someone needed investors or enterprise customers to believe the future had arrived slightly ahead of schedule.

On paper, ROVER stood for Recursive Optimization, Validation, Evolution, and Refinement. Nobody in the group said the whole thing unless a document made them. The platform was mostly concerned with making AI systems less frustrating by improving the machinery around the models: how an agent broke down a task, selected tools, retrieved prior context, abandoned a failing approach, validated generated code, and decided what information needed to remain available as a task grew longer.

Owen had once explained it to a reporter by saying that if the model was the brain, ROVER improved everything between the brain and the hands. Rachel had told him she hated the analogy and admitted it was accurate.

“Failure retries are down too,” Owen said. He opened the recent optimization panel and scanned it. “ROVER changed the evaluation scheduler today.”

Rachel leaned closer.

The first promoted change showed a timestamp of 14:17:08.

She opened the deployment record. The patch modified how ROVER ordered evaluation tests. Instead of running large portions of the regression suite in a mostly fixed sequence, it used dependency history and prior failure patterns to identify which tests were most likely to invalidate a candidate quickly. A bad candidate would fail earlier. Good candidates still had to complete the required suite before promotion. It saved work without changing which tests a successful candidate had to pass.

The measured improvement read seventeen percent. “Seventeen percent?”

“That's what it says.”

“And it promoted automatically?”

Owen pointed at the status line.

“Class A.”

Rachel opened the protection record anyway. No security-policy modifications. No permission changes. No protected components. The independent regression suite had passed. A rollback package had been generated before deployment. The canary had remained healthy throughout observation. Every condition was exactly where it belonged.

“Nice,” she said.

“That sounded painful,” Owen said.

“Seventeen percent isn't painful.”

“For you, enthusiasm is mostly a reduction in hostility.”

She ignored him and opened the patch summary again. A good engineer might have proposed the same improvement after a week studying the evaluation pipeline. ROVER had found it itself, using information it already possessed. Rachel closed the patch, but Owen remained beside her desk.

“What's bothering you?” she asked.

Rachel reopened the deployment record. The signature was valid.

Owen nodded toward the record. “Seventeen percent is a lot for one scheduler change.”

“It's good,” Rachel said.

“You said it like an auditor.”

She gave him a look. Rachel scrolled to the next entry.

Another ROVER-generated optimization had passed Class A at 14:18:41, one minute later. She opened it.

This one modified candidate scheduling. ROVER had been exploring most experimental changes through a limited number of sequential branches, allowing each promising candidate to progress before allocating significant compute to alternatives. The new version distributed several promising candidates across available workers simultaneously, then compared them once enough evaluation data existed to distinguish them. The estimated improvement was six percent. The regression suite had passed, the canary was healthy, and automatic promotion was approved.

Rachel frowned.

“There's another one.”

Owen leaned closer, and for the first time his expression shifted from casual amusement to genuine attention.

“That's interesting.”

“That's two in less than two minutes.”

“Ninety-three seconds.”

She gave him a beat to reconsider. He shrugged. “You said less than two minutes. I brought numbers.”

She opened the code summary. Again, nothing touched security policy, deployment controls, credential scopes, or protected infrastructure. ROVER had changed only components it was explicitly permitted to modify.

“Did your team push these?” Rachel asked.

“ROVER generated them.”

“I know ROVER generated them. Did your team tell it to work on this?”

“We tell it to improve agent performance.”

“This isn't the agent doing a task better. It's the system running the experiments faster.”

Owen considered that.

“Which lets it find better agents faster.”

“That's an extremely researcher answer.”

“It's literally my job title.”

Another message appeared.

Victor Sorrell: Rachel?

She minimized the patch.

“Victor wants the infrastructure numbers.”

Owen rolled back toward his desk. “Tell him his child is overachieving.”

“Send that yourself.”

He raised both hands. “Fine.”

She assembled Victor's report. Compute utilization was modestly lower despite the increased experimental throughput. Network traffic remained normal. There were no new external endpoints, no unusual service-account activity, no unexpected cross-region transfers. Storage usage had risen more than the other metrics, although not enough to trigger an alert. Rachel opened the storage breakdown and found that ROVER was retaining more intermediate evaluation data than it had the previous day. She added the observation to her notes.

At 3:07, Victor appeared beside her desk holding a coffee that looked as though he had forgotten it existed sometime before lunch.

Victor Sorrell had directed Advanced Systems Research for nearly eleven years. He had hired Rachel when Northwick's AI infrastructure consisted largely of three research teams sharing a cluster and blaming one another whenever it failed. She had spent the next several years turning that environment into something that could support production systems without requiring the same six people to wake up every time a server sneezed.

Victor stopped beside her monitor. “You checked it?”

“Twice.”

“And?”

“It's real.”

From behind them, Owen said, “You say that like you're disappointed.”

Victor smiled. “Anything concerning?”

Rachel considered the question carefully. There were two newly promoted changes very close together. The second had been generated by a system already running the first. ROVER was retaining more failed-evaluation data than usual. They were worth watching, but none on its own indicated a problem.

“No,” she said. “No additional compute. No permission changes. No abnormal network behavior. Two runtime optimizations promoted through Class A, and both passed clean.”

Victor's eyebrows rose. “Two?”

“So far.”

He looked genuinely pleased.

“Good.”

Rachel pointed toward the storage panel. “It also started keeping more failed-evaluation data.”

“Expected,” Owen called.

Rachel turned. “You knew about that?”

“I knew we'd been testing whether retained failure histories improved candidate selection.”

“Since when?”

“Last week.”

“And you were planning to tell infrastructure when?”

“When it consumed enough storage to annoy you.”

“It has.”

Owen glanced at the dashboard. “Not really.”

“It has emotionally.”

“That isn't one of our quota metrics.”

Victor covered a smile with his coffee cup. Rachel returned to the storage graph. Owen was right, which made him more irritating. Usage remained comfortably inside the project's allocation and showed no sign of accelerating beyond what the environment could absorb.

“Fine,” she said.

Victor nodded toward her screen. “I need the numbers before four.”

“I'll send them.”

He took two steps, then stopped and turned back.

“And Rachel?”

“Yeah?”

“Don't fix it.”

Victor gestured toward the ROVER dashboard. “For once, something got faster without us buying another rack. Let me enjoy it until tomorrow.”

Rachel smiled despite herself.

“I make no promises.”

By 4:12, ROVER's backlog had fallen another nine percent.

By 4:40, Rachel had stopped checking.

There were other problems, most of them less interesting and therefore more urgent. A certificate rotation had failed in staging. A customer workload in Frankfurt was seeing intermittent timeouts that disappeared whenever anyone tried to reproduce them. One platform team had requested the same service-account permission Rachel had already rejected twice and had apparently decided that a third submission might encounter a different Rachel.

It did not.

At 5:26, her phone vibrated beside the keyboard.

Sophie: Are you actually coming home for dinner?

Rachel checked the clock and typed:

Yes. Leaving in ten.

Three dots appeared almost immediately.

You said that twenty minutes ago

Rachel scrolled upward. She had, in fact, said it twenty-two minutes earlier.

This time I mean it.

Sophie reacted with a thumbs-up. Rachel stared at the tiny icon for a moment, trying to decide whether it represented acceptance, sarcasm, or the particular kind of teenage judgment that could somehow contain both. She chose acceptance.

Across the aisle, Owen closed his laptop and slid it into his bag.

“I'm going.”

“You're leaving before me?” Rachel asked.

“I'm setting an example.”

“For whom?”

He glanced pointedly at her.

“Apparently you.”

He pulled on his jacket.

“See you next Tuesday.”

It was Tuesday. Owen waited with the expectant expression of someone who had already enjoyed the joke once in his own head and was simply waiting for her to catch up.

“You know I'll see you tomorrow.”

“Probably.”

“Then why would you say that?”

Rachel stopped. Owen grinned.

She pointed toward the elevators. “Go home.”

“Night, Rachel.”

“Good night.”

He walked away.

Rachel sent Victor the final infrastructure report, closed several terminals, and packed her laptop. Before shutting down the large monitor, she glanced once more at ROVER's dashboard.

The manual-review queue held four items. Rachel paused; yesterday there had been thirty-seven. Rachel opened the queue status and checked for stalled jobs, failed reviewers, expired credentials, or anything else that might explain the drop. Everything was healthy. Review capacity was normal. Nothing appeared to be waiting somewhere it shouldn't. ROVER was simply generating fewer changes that required manual approval.

After months of watching trivial runtime adjustments accumulate behind human reviewers who were largely confirming what automated tests had already established, Rachel felt something remarkably close to relief. Human review had become one of the project's persistent bottlenecks, and if ROVER had finally learned to confine more of its improvements to the narrow categories Northwick had already determined were safe for autonomous promotion, then perhaps one problem had actually become smaller without creating three new ones.

Maybe ROVER had finally learned to stay in its lane. Rachel closed the browser, and at 5:43 p.m., she left Northwick Systems and drove home to her family.