How Music Got Free

VIKING

An imprint of Penguin Random House LLC

375 Hudson Street

New York, New York 10014

penguin.com

Copyright © 2015 by Stephen Richard Witt

Penguin supports copyright. Copyright fuels creativity, encourages diverse voices, promotes free speech, and creates a vibrant culture. Thank you for buying an authorized edition of this book and for complying with copyright laws by not reproducing, scanning, or distributing any part of it in any form without permission. You are supporting writers and allowing Penguin to continue to publish books for every reader.

LIBRARY OF CONGRESS CATALOGING-IN-PUBLICATION DATA

Witt, Stephen.

How music got free : the end of an industry, the turn of the century, and
the patient zero of piracy / Stephen Witt.

pages cm

Includes bibliographical references and index.

ISBN 978-0-698-15252-6

1. Sound recording industry. 2. Music and the Internet.
3. Sound recordings—Pirated editions.
4. MP3 (Audio coding standard)—History. I. Title.

ML3790.W59 2015

381'.45780266—dc23 2015010568

To Leonard and Diana,
my loving parents

I
am a member of the pirate generation. When I arrived at college in 1997, I had never heard of an mp3. By the end of my first term I had filled my 2-gigabyte hard drive with hundreds of bootlegged songs. By graduation, I had six 20-gigabyte drives, all full. By 2005, when I moved to New York,
I had collected 1,500 gigabytes of music, nearly 15,000 albums worth. It took an hour just to queue up my library, and if you ordered the songs alphabetically by artist, you’d have to listen for a year and a half to get from ABBA to ZZ Top.

I pirated on an industrial scale, but told no one. It was an easy secret to keep. You never saw me at the record store and I didn’t DJ parties. The files were procured in chat channels, and through Napster and BitTorrent; I haven’t purchased an album with my own money since the turn of the millennium. The vinyl collectors of old had filled whole basements with dusty album jackets, but my digital collection could fit in a shoebox.

Most of this music I never listened to. I actually hated ABBA, and although I owned four ZZ Top albums, I couldn’t tell you the name of one. What was really driving me, I wonder? Curiosity played a role, but now, years later, I can see that what I really wanted was to belong to an elite and rarefied group. This was not a conscious impulse, and, had you suggested it to me, I would have denied it. But that was the perverse lure of the piracy underground, the point that almost everyone missed. It wasn’t just a way to get the music; it was its own subculture.

I was at the very forefront of the digital download trend. Had I
been just a couple of years older, I doubt I would have become so involved. My older friends regarded piracy with skepticism, and sometimes outright hostility. This was true even for those who loved music—in fact, it was especially true for them. Record collecting had been a subculture too, and, for that vanishing breed, finding albums proved to be an exhilarating challenge, one that involved scouring garage sales, sifting through bargain bins, joining mailing lists for bands, and Tuesday visits to the record store. But for me, and those younger, collecting was effortless: the music was simply there. The only hard part was figuring out what to listen to.

As I was browsing through my enormous list of albums one day a few years ago, a fundamental question struck me: where had all this music come from, anyway? I didn’t know the answer, and as I researched it, I realized that no one else did either. There had been heavy coverage of the mp3 phenomenon, of course, and of Apple and Napster and the Pirate Bay, but there had been little talk of the inventors, and almost none at all of those who actually pirated the files.

I became obsessed, and as I researched more, I began to find the most wonderful things. I found the manifesto from the original mp3 piracy clique, a document so old I needed an MS-DOS emulator just to view it. I found the cracked shareware demo for the original mp3 encoder, which even its inventors had considered lost. I found
a secret database that tracked thirty years of leaks—software, music, movies—from every major piracy crew, dating back to 1982. I found secret websites in Micronesia and the Congo, registered to shell corporations in Panama, the true proprietors being anyone’s guess. Buried in thousands of pages of court documents, I found wiretap transcripts and FBI surveillance logs and testimony from collaborators in which the details of insidious global conspiracies had been laid bare.

My assumption had been that music piracy was a crowdsourced phenomenon. That is, I believed the mp3s I’d downloaded had been sourced from scattered uploaders around the globe and that this diffuse network of rippers was not organized in any meaningful way.
This assumption was wrong. While some of the files were indeed untraceable artifacts from random denizens of the Internet, the vast majority of pirated mp3s came from just a few organized releasing groups. By
using forensic data analysis, it was often possible to trace those mp3s back to their place of primary origination. Combining the technical approach with classic investigative reporting, I found I could narrow this down even further. Many times it was possible not just to track the pirated file back to a general origin, but actually to a specific time and a specific person.

That was the real secret, of course: the Internet was made of people. Piracy was a social phenomenon, and once you knew where to look, you could begin to make out individuals in the crowd. Engineers, executives, employees, investigators, convicts, even burnouts—they all played a role.

I started in Germany, where a team of ignored inventors, in a blithe attempt to make a few thousand bucks from a struggling business venture, had accidently crippled a global industry. In so doing, they became extremely wealthy. In interviews, these men dissembled, and attempted to distance themselves from the chaos they had unleashed. Occasionally, they were even disingenuous, but it was impossible to begrudge them their success. After cloistering themselves for years in a listening lab, they had emerged with a technology that would conquer the world.

Then to New York, where I found a powerful music executive in his early 70s who had twice cornered the global market on rap. Nor was that his only achievement; as I researched more, I realized that this man
was
popular music. From Stevie Nicks to Taylor Swift, there had been almost no major act from the last four decades that he had not somehow touched. Facing an unprecedented onslaught of piracy, his business had suffered, but he had fought valiantly to protect the industry and the artists that he loved. To my eyes, it seemed unquestionable that he had outperformed all of his competitors; for his trouble, he’d become one of the most vilified executives in recent memory.

From the high-rises of midtown Manhattan I turned my attention to Scotland Yard and FBI headquarters, where dogged teams of investigators had been assigned the thankless task of tracking this digital samizdat back to its source, a process that often took years. Following their trail to a flat in northern England, I found a high-fidelity obsessive who had overseen a digital library that would have impressed even Borges. From there to Silicon Valley, where another entrepreneur had also designed a mind-bending technology, but one that he had utterly failed to monetize. Then to Iowa, then to Los Angeles, back to New York again, London, Sarasota, Oslo, Baltimore, Tokyo, and then, for a long time, a string of dead ends.

Until finally I found myself in the strangest place of all, a small town in western North Carolina that seemed as far from the global confluence of technology and music as could be. This was Shelby, a landscape of clapboard Baptist churches and faceless corporate franchises, where one man, acting in almost total isolation, had over a period of eight years cemented his reputation as the most fearsome digital pirate of all. Many of the files I had pirated—perhaps even a majority of them—had originated with him. He was the Patient Zero of Internet music piracy, but almost no one knew his name.

Over the course of more than three years I endeavored to gain his trust. Sitting in the living room of his sister’s ranch house, we often talked for hours. The things he told me were astonishing—at times they seemed almost beyond belief. But the details all checked out, and once, at the end of an interview, I was moved to ask:

“Dell, why haven’t you told anybody any of this before?”

“Man, no one ever asked.”

T
he death of the mp3 was announced in a conference room in Erlangen, Germany, in the spring of 1995. For the final time, a group of supposedly impartial experts snubbed the technology, favoring its eternal rival, the mp2. This was the end, and the mp3’s inventors knew it. They were running out of state funding, their corporate sponsors were abandoning them, and, after a four-year sales push, the technology had yet to secure a single long-term customer.

Attention in the conference room turned to Karlheinz Brandenburg, the driving intellectual force behind the technology and the leader of the mp3 team. Brandenburg’s work as a graduate student had pointed the way to the technology, and for the last eight years he had worked to commercialize his ideas. He was ambitious and intelligent, with a contagious vision for the future of music. Fifteen engineers worked under him, and he oversaw a million-dollar research budget. But with the latest announcement, it looked as if he had led his team into a graveyard.

Brandenburg did not possess a commanding physical presence. He was very tall, but he hunched, and his body language was erratic. He constantly rocked on his heels, lurching his gangly body forward and back, and when he talked, he nodded his head in gentle circles. His hair was dark and kept too long, and his nervous, perpetual smile exposed teeth that were uneven and small. His wire-frame glasses sat over dark, narrow eyes, and stray hairs protruded like whiskers from his scraggly beard.

He spoke quietly, in long, grammatically perfect sentences,
punctuated with short, sharp intakes of breath. He was polite, and overwhelmingly kind, and he always tried his best to put people at ease, but this only made things more awkward. When he talked, he tended to dwell on practical matters, and, perhaps sensing boredom on the part of the listener, he would occasionally pepper this rambling technical discourse with weakly delivered, unfunny jokes. In his personality were united two powerful antiseptic forces: the skepticism of the engineer, and the stuffy, nation-specific conservatism they called
typisch Deutsch
.

He was brilliant, though. His mathematical talent was surpassing, and he held his contemporaries in thrall. These were men who had excelled in difficult academic disciplines and who had spent their lives near the top of competitive fields. They were not, as a rule, given to intellectual modesty, but when they talked of Brandenburg, their arrogance subsided and they reverted to quiet, confessional tones. “
He’s very good at math,” said one. “He really is quite smart,” said another. “He solved a problem I could not,” said a third, and this, for an engineer, was the most terrible admission of all.

When challenged on a point, Brandenburg would pause, then squint, then subject the contrasting claim to a piercing scientific dismissal. In disagreement, his voice grew almost imperceptible, and in his responses he was guarded in the extreme, careful to never make an assertion without the data to back it up. In the conference room then, as he lodged his final objection to the committee, the mp3 went out with a whisper.

Defeat was always bitter, but this one was more so since, after 13 years of work, Brandenburg had solved one of the great open questions in the field of digital audio. The body of research the committee was dismissing went back decades, and engineers had been theorizing about something like the mp3 since the late 1970s. Now from this murky scientific backwater something beautiful had emerged, the refined product of a line of inquiry that went back three generations. Only the suits in the room didn’t care.

Brandenburg’s thesis adviser, a bald, stentorian computer engineer by the name of Dieter Seitzer, had started him down this path. Seitzer himself was indebted to his own thesis adviser, an obsessive investigator named Eberhard Zwicker, the father of an obscure discipline called “psychoacoustics”—the scientific study of the way humans perceive sound. Seitzer had been Zwicker’s protégé, his experimental audio subject, and, most important, his mortal opponent. For nearly a decade, the two had met every weekday after lunch for a game of table tennis, during which, over the course of an hour, Zwicker would school his pupil on the
liminal contours of human perception while blasting ping-pong balls at his head. Zwicker’s chief finding, accrued over decades of research with real-world test subjects, was that the human ear did not act like a microphone. Instead it was an adaptive organ, one that natural selection had determined should 1) hear and interpret language and 2) provide an early warning system against enormous carnivorous cats.

The ear was only as good as it needed to be to achieve these goals, and no better. Thus, it had inherited a legacy of anatomical imperfections, and Zwicker’s research had revealed the unsuspected breadth of these errors. For example, anyone could distinguish two simultaneous tones separated by a half note or more, but Zwicker had found that, by moving the tones closer together in pitch, he could trick people into hearing just one. This effect was especially true when the lower-pitched tone was louder than the higher one. Similarly, any listener could distinguish between two clicks spaced a half second apart, but Zwicker had found that, by shortening this interval to just a few milliseconds, he could trick the ear into combining them. Here, too, increasing the relative loudness of one of the clicks made the effect more pronounced. The aggregate effect of these “psychoacoustic masking” illusions meant that reality, as humans heard it, was something of a fiction.

With time, Seitzer began to outplay the master. Zwicker was an anatomist, and his insights were products of the analog era. Seitzer,
by contrast, was a computer scientist, and he anticipated the coming era of digitization. In particular, he suspected that, by exploiting Zwicker’s research into the ear’s inherent flaws, it might be possible to record high-fidelity music with very small amounts of data. This unique education gave him an unusual perspective. When the compact disc debuted in 1982, the engineering community celebrated it as one of the most important achievements in the history of the field. Seitzer, practically alone, saw it as a ridiculous exercise in overkill. Where the sales literature promised “
Perfect Sound Forever,” Seitzer saw a maximalist repository of irrelevant information, most of which was ignored by the human ear. He knew that most of the data from a compact disc could be discarded—the human auditory system was already doing it.

That same year, Seitzer applied for a patent for a digital jukebox. Under this more elegant model of distribution, consumers could dial into a centralized computer server, then use the keypad to request music over the new digital telephone lines that Germany was just beginning to install. Rather than pressing millions of discs into jewel cases and distributing them through stores, everything would be saved in a single electronic database and accessed as needed. A subscription-based service of this kind could skip the manifold inefficiencies of physical distribution by hooking the stereo directly to the phone.

The patent was rejected. The earliest digital phone lines were primitive affairs, and the enormous amount of audio data on the compact disc could never fit down such a narrow pipe. For Seitzer’s scheme to work, the files on the disc would have to be shrunk to
one-twelfth their original size, and no known approach to data compression would get you anywhere near this level. Seitzer battled with the patent examiner for a few years, citing the importance of Zwicker’s findings, but without a working implementation it was hopeless. Eventually, he withdrew his application.

Still, the idea stayed with him. If the limitations of the human ear had been mapped by Zwicker, then the remaining task was to quantify these limitations with math. Seitzer himself had never been able to solve this problem, nor had any of the many other researchers who had tried. But he directed his own protégé toward the problem with enthusiasm: the young electrical engineering student named Karlheinz Brandenburg was one of the smartest people he’d ever met.

Privately, Brandenburg wondered if a decade of table tennis with an eccentric otological experimenter had driven Seitzer insane. Information in the digital age was stored in binary units of zero or one, termed “bits,” and the goal of compression was to use as few of these bits as possible. CD audio used more than 1.4 million bits to store a single second of stereo sound. Seitzer wanted to do it with 128,000.

Brandenburg thought this goal was preposterous—it was like trying to build a car on a budget of two hundred dollars. But he also thought it was a worthy target for his own ambitions. He worked on the problem for the next three years, until in early 1986 he spotted an avenue of inquiry that had never been explored. Dubbing this insight “analysis by synthesis,” he spent the next few sleepless weeks writing a set of mathematical instructions for how those precious bits could be assigned.

He began by chopping the audio up. With a “sampler,” he divided the incoming sound into fractional slivers of a second. With a “filter bank,” he then further sorted the audio into different frequency partitions. (The filter bank worked on sound the way a prism worked on light.) The result was a grid of time and frequency, consisting of microscopic snippets of sound, sorted into narrow bands of pitch—the audio version of pixels.

Brandenburg then told the computer how to simplify these audio “pixels” using four of Zwicker’s psychoacoustic tricks:

First, Zwicker had shown that human hearing was best at a certain range of pitch frequencies, roughly corresponding to the tonal
range of the human voice. At registers beyond that, hearing degraded, particularly as you went higher on the scale. That meant you could assign fewer bits to the extreme ends of the spectrum.

Second, Zwicker had shown that tones that were close in pitch tended to cancel each other out. In particular, lower tones overrode higher ones, so if you were digitizing music with overlapping instrumentation—say a violin and a cello at the same time—you could assign fewer bits to the violin.

Third, Zwicker had shown that the auditory system canceled out noise following a loud click. So if you were digitizing music with, say, a cymbal crash every few measures, you could assign fewer bits to the first few milliseconds following the beat.

Fourth—and this is where it gets weird—Zwicker had shown that the auditory system also canceled out noise prior to a loud click. This was because it took a few milliseconds for the ear to actually process what it was sensing, and this processing could be disrupted by a sudden onrush of louder noise. So, going back to the cymbal crash, you could also assign fewer bits to the first few milliseconds
before
the beat.

Relying on decades of empirical auditory research, Brandenburg told the bits where to go. But this was just the first step. Brandenburg’s real achievement was figuring out that you could run this process iteratively. In other words, you could take the output of his bit-assignment algorithm, feed it back into the algorithm, and run it again. And you could do this as many times as you wished, each time reducing the number of bits you were spending, making the audio file as small as you liked. There was degradation of course: like a copy of a copy or a fourth-generation cassette dub, with each successive pass of the algorithm, audio quality got worse. In fact, if you ran the process a million times, you’d end up with nothing more than a single bit. But if you struck the right balance, it would be possible to both compress the audio and preserve fidelity, using only those bits you knew the human ear could actually hear.

Of course, not all musical work employed such complex
instrumentation. A violin concerto might have all sorts of psychoacoustic redundancies; a violin solo would not. Without cymbal crashes, or an overlapping cello, or high register information to be simplified, there was just a pure tone and nowhere to hide. What Brandenburg could do here, though, was dump the output bits from his compression method into a second, completely different one.

Termed “Huffman coding,” this approach had been developed by the pioneering computer scientist David Huffman at MIT in the 1950s. Working at the dawn of the Information Age, Huffman had observed that if you wanted to save on bits, you had to look for patterns, because patterns, by definition, repeated. Which meant that rather than assigning bits to the pattern every time it occurred, you just had to do it once, then refer back to those bits as needed. And from the perspective of information theory, that was all a violin solo was: a vibrating string, cutting predictable, repetitive patterns of sound in the air.

The two methods complemented each other perfectly: Brandenburg’s algorithm for complicated, overlapping noise; Huffman’s for pure, simple tones. The combined result united decades of research into acoustic physics and human anatomy with basic principles of information theory and complex higher math. By the middle of 1986, Brandenburg had even written a rudimentary computer program that provided a working demonstration of this approach. It was the signature achievement of his career: a proven method for capturing audio data that could stick to even the stingiest budget for bits. He was 31 years old.

He received his first patent before he’d even defended his thesis. For a graduate student, Brandenburg was unusually interested in the dynamic potential of the marketplace. With a mind like his, a tenure-track position was guaranteed, but academia held little interest for him. As a child he’d read biographies of the great inventors, and at an early age had internalized the importance of the hands-on approach. Brandenburg—like Bell, like Edison—was an inventor first.

These ambitions were encouraged. After escaping from Zwicker,
Dieter Seitzer had spent most of his own career at IBM, accruing basic patents and developing keen commercial instincts. He directed his graduate students to do likewise. When he saw the progress that Brandenburg was making in psychoacoustic research, he pushed him away from the university and toward the nearby Fraunhofer Institute for Integrated Circuits, the newly founded Bavarian technology incubator that Seitzer oversaw.