Fishbowl (33 page)

All people could think about, it seemed, was money. And yet, Andrei told himself, that was the least important thing to him. He didn't understand how they just couldn't see that he was changing the world.

But he was wrong. Some people could.

37

FARMING 2.0 WAS
the second time Fishbowl had left the FBI red-faced. Andrew Buckett's ravings on the site prior to the Denver bombing, while not involving any actual planning of the attack, had highlighted the Bureau's failure to identify the threat posed by him and Hodgkin before they could act. This time, while less public, the effect of what Fishbowl had done on the Bureau's reputation inside the Beltway was potentially far more damaging. Despite having had Fishbowl on its watch list since the day Farming 1.0 had become known, the Cyber Division of the Bureau had failed utterly to realize or even suspect that Farming 2.0 was under development. Like the rest of the world, they only found out when rumours began to spread. But, once the cat was out of the bag, they knew they were dealing with something momentous.

Ed Garcia, a 34-year-old graduate of Duke University's Computer Science programme who had joined the Cyber Division seven years previously, was the agent with the deepest knowledge of Fishbowl. He immediately recognized the quantum leap in capability that had taken place if the rumours were true. He scanned the Grotto, got hold of the Spot the Bot app, spent some time on the network, and was soon convinced that some kind of artificial intelligence program was at work. He wrote a report that went via his boss to James Monk, the head of the Cyber Division. In it, Garcia didn't pull his punches about what he considered to be the full range of uses for such a program.

Monk reflected on the report for a couple of weeks, wondering if its conclusions were exaggerated. He got another couple of
agents in the division to validate it independently. By then, Andrei had posted his enigmatic statement in the Grotto that more or less confirmed that the rumours were on to something. The agents reported back to Monk that Garcia's conclusions, if anything, were conservative. The report went to the FBI director, and from him to the Department of Justice, where its conclusions were read by the attorney general. The attorney general had the legal issues investigated by the Office of Legal Counsel.

Six weeks after the furore erupted, when the first blizzard of public outrage was dying down and Andrei Koss and the Fishbowl team were imagining that the worst was past, Ed Garcia found himself with James Monk, the FBI Director and the attorney general in the Oval Office, meeting the president and his national security advisor.

The president listened as Monk outlined the issue. The frown on his face, which was familiar to the more senior people in the room, suggested that he was having a little difficulty grasping the nature of the computer program that they were talking about. One after another, the national security advisor, the attorney general and the FBI director chipped in.

‘You!' said the president, cutting across them and pointing at Ed Garcia. ‘Tell me what this is about.'

Garcia froze.

‘I thought you're the expert,' said the president irascibly. He had a cold that he had picked up on a G20 visit to Berlin, where he had been forced to spend a dinner sitting beside the German Chancellor, who had spent the whole evening sneezing into a disgustingly moist handkerchief. The president's fuse, always short, was already smouldering. ‘What's he doing here, Frank?' he demanded of the FBI director. ‘Didn't you say he was the expert?'

Garcia coughed. ‘Simply put, sir …'

‘Yes, put it simply,' said the president sharply. ‘And remember, when I went to school, we used things called pens and paper.'

‘Well, simply put, sir, it's a program that talks to you like it's a person.'

The president stared, his look of incomprehension turning to one of incredulity.

‘It can talk to millions and millions of people at once. Potentially, if they've built it right, it can make itself look different to different people, working out what kind of apparent person is most likely to gain an individual's trust. It's intelligent – what I mean by that is it can learn and evolve. From what you say, it can identify what you're interested in, what's bugging you, and it can respond to that in a way that will draw you in. You'll think you're talking to a person. But you're not. You're talking to a program that can figure out just which of your buttons to push to make you its friend.'

‘And it's used to sell stuff? Is that it?'

‘That's the obvious use, Mr President. You could set it up so it's looking for people who would be interested in a particular product and to find opportunities to recommend it. That's not our worry. What we're concerned about is that, in theory, if it can promote a product, it can promote anything – an opinion, an idea, an ideology. It could be used to radicalize people. It could be used to sway public opinion about an issue. It could be used to have an effect on an election.'

‘Ah …' The president got it now. He had no idea how such a thing could work, but he understood the implications.

‘That's if it exists.'

He looked back at Garcia. ‘We don't know if it exists?'

‘The CEO of Fishbowl has more or less admitted it, but not explicitly, and the company hasn't released any details about it. There are definitely patterns of chat on Fishbowl that are highly suggestive that something like this is in operation.'

‘On the other hand,' said Monk, ‘it's possible that he's consciously allowed the speculation to get ahead of the reality. Stoked it, if you will. The general expectation is that Fishbowl is getting ready for an IPO. If you want to boost your stock price before a possible IPO,
you might lay down the tracks that you've made this kind of advance – and make no mistake, Mr President, it would be a groundbreaking advance – while preserving deniability for later.'

‘Have we talked to this CEO to find out if it exists?' demanded the president.

‘Not yet,' said the attorney general. ‘We've been trying to determine first what the legal position is.'

‘And what is the legal position, Sue? Surely something like this can't be legal? Even if they just use it to sell stuff, surely it can't be legal.'

‘Actually, it is.'

‘We can't stop it?'

‘No, sir. Not under current law.'

‘So someone can just go out there and sell something without saying who they are?'

‘Our laws are generally about
what
you say – misrepresentation, fraud – and not who says it.'

‘So they can do this without saying who they are?'

‘Yes, sir. People do that all the time.'

‘And the people who use this Fishbowl thing, they don't mind that?'

The attorney general looked at Garcia for an answer.

‘Apparently not, sir.'

‘Jesus Christ,' murmured the president. ‘What kind of a world is this?'

No one chose to answer that question.

‘What if they did try to use it for political purposes?' asked the president. ‘That would have to be declared, right?'

‘Only if it's being funded,' said the attorney general.

‘So if this CEO decides he simply wants to put a message—?'

‘Nothing to stop him doing that.'

‘How many people does this thing talk to?'

‘One point four billion,' said Garcia.

‘Jesus
Christ
!' The president sat forward. ‘Is there any evidence that the Democratic Party funded the development of this program?'

Garcia looked at him quizzically. ‘No, sir.'

‘Check that out, Frank.'

‘As far as we understand,' said the FBI director, ‘this was developed for commercial purposes—'

‘Just check it out, Frank.'

‘Mr President, we would also have to check out whether the Republican Party was involved.'

The president was silent for a moment. ‘Better let me … have some conversations.'

The FBI director nodded.

‘Now, what if this thing fell into the wrong hands?' said the president. ‘What if someone steals it? Could someone steal it?'

Everyone looked at Garcia.

‘I don't think that's the issue,' said Garcia.

‘Why not?'

‘It's not a
thing
, Mr President. It's not like a gun you can walk in and steal. It's pages of code. Thousands and thousands and thousands of pages.'

‘You seem to know an awful lot about something when you don't even know if it exists.'

‘Well, it would be pages of code, sir.'

‘What if you stole the computer it's on?'

‘There's no—I mean, there wouldn't be any one computer it's on. It would be distributed and replicated across a bunch of computers that are linked together.'

‘What if you took control of the place where they are?'

‘They wouldn't be in one place. They'd be distributed. That's both for security and redundancy. There wouldn't be any one facility that, if someone took it over, they'd have control of this program.'

‘What if someone … hacks into it? Is that the term?'

‘Well, I would imagine that would be very hard to do. I would imagine that Fishbowl, with the resources it has available, would have about the best security out there.'

‘So does the Pentagon, theoretically,' muttered the president.
‘And every schoolboy and his dog seems to be able to hack into there.'

‘Potentially, in theory, yes, someone could hack into this program,' conceded Garcia.

‘And?'

‘They could mess with it, but probably only for a short period of time until the intrusion was discovered. What I mean is, they could probably disrupt it, but they couldn't take control of it. It would be very doubtful they could do that, certainly not in a way that wouldn't be immediately discovered.'

‘Well, if it can be disrupted, that sounds like that's something we should make sure we're able to do.'

Garcia and Monk glanced at each other.

‘So what you're saying,' said the national security advisor, ‘is that you can't steal it, and you can't hack it so that you can control it …'

‘But you could sell it,' said the FBI director. ‘Or license it. To an Islamist group, for an example.'

‘Jesus Christ, this thing's a fucking nightmare!' The president yanked a handful of tissues out of the box on the coffee table in front of him and blew his nose angrily, then dropped the used wad on the rug at his feet. ‘Can we restrict that? Can we restrict their ability to license it?'

‘If Fishbowl wants to license it to a foreign entity,' said the attorney general, ‘then theoretically we could control it by making it subject to export control, but there are difficulties with that. Export controls are largely for defence-related items for military use, and this program hardly falls into that category. At a pinch, we could try to make an argument that it's dual use – meaning that it could have both civilian and military applications – but whether a court would agree with that is another question. I mean, if Al Qaida uses PowerPoint for their propaganda, do we ban Microsoft from exporting it?'

‘What if they sell it to a US entity?' said the national security advisor.

‘Our laws aren't supposed to restrain trade, they're supposed to encourage it. So long as we have no legal grounds against the program, we have no grounds to restrict its distribution.'

‘So a foreign regime could set up a company here and buy this program and it wouldn't be subject to any kind of control.'

‘It starts to get quite complicated. If the purpose was to circumvent an export control, and we could show that the US entity was actually controlled by a foreign party, then I think that they wouldn't be able to do that – provided of course, that we can get an export control in the first place. But what if nothing is actually exported? What if the third party uses the program residing on servers in the United States? Have they exported something or not? We'd work on the assumption that it's an export, but it's not clear what a judge would rule. The export control system was set up to deal with transfers of physical goods – missiles, rocket parts, nuclear fuels, nuclear blueprints – not for a program that might never actually leave US soil. It's grey. It's very grey. And in terms of selling it to any bona fide US organization, we have no ability to control that. The legal situation—'

‘Hell, Sue,' said the president. ‘Bottom line is, this thing's not illegal, right?'

‘Not at present.' The attorney general shrugged. ‘It's not clear how we could make it illegal. Anything that applies to Fishbowl has to apply to everyone. It's not clear what that legislation would have to look like without making illegal just about every sales and advertising operation in the country. And that's before a first amendment challenge, which would certainly follow.'

‘So it's not illegal. OK. Let's put that aside.'

‘We're still looking at it but, for the moment, yeah, let's assume it's not.'

‘So tell me, if a political party here in the United States, for example, wanted to use this program in an election campaign, then this company would be free to license it and there would be nothing we could do until such time as they actually used it.'

‘I'm not sure there'd be anything we could do even if they did use it. Not unless it was used to say something defamatory or fraudulent.'

‘They'd be seeking to influence an election.'

‘Everyone seeks to influence elections, Mr President. If they declare, for instance, that they're from the Democratic Party, I don't see that it would make a difference if it's a program using a palotl – that's the name for the virtual personalities they create – or if it's a real person. If you have an animated figure in an advertisement, you don't need to state that it isn't human.'

‘People can see that it's not!'

‘A blind person can't. There's no law that states that you have to say, “I'm from such and such a party, and I'm a real human being.”' The attorney general smirked a little at what she had just said. She didn't know many politicians who would be able to pass that test.