Spycraft Tools - Xkeyscore - How It Works

Xkeyscore - How It Works Step By Step

As mentioned, everything starts with open-source software. Anyone familiar with setting up servers has heard of a LAMP stack. That stands for a Linux operating system, Apache web server, MySQL database, and the PHP programming language. These core components run millions of servers around the world, both in the Cloud and on dedicated hardware. They are the heart and soul of innovation for hundreds of thousands of open-source projects.

Admins use completely standard utilities for regular tasks and maintenance: CRON jobs to slate tasks, NFS clustering for distributed computing and storage, and SSH to log into a boring-looking command prompt.

The agents and operatives, however, are treated to something a bit more sophisticated. They can use a browser-based interface to log into Xkeyscore from any field office around the globe, as well as any other authorized data redistribution points. From there they can enter the name of a person, place, or thing of interest and immediately start receiving known data on the subject, as well as start the process to access live information.

Live information is available thanks to a series of state-sponsored monitoring in countries all around the world, backdoors left in cell phones and other communications hardware, and undersea data cable tapping operations.

I wish I was joking. Here are the details.

Xkeyscore can collate the data brought in from webcam hacking projects like (the now-outdated and replaced many times over) Optic Nerve. It can bring in police and emergency services footage from traffic and security cameras. It can tap into mobile microphones thanks to the millions of mobile phones out there, via the Smurf spy suite. It can also pull in the GPS data from said phones, and request time on either government or private satellite cameras via the GeoInt agency and associated programs. And of course, PRISM and Project Tempora allow for the tapping of international Internet traffic all around the world. That data can be pulled in raw, or pre-processed and filtered for certain file types, keywords, or phrases.

And those methods are just the tip of the iceberg. For subjects of interest using E-Mail providers that the NSA or the Five Eyes nations have an 'in' with, instant E-Mail access is possible, and in some cases live chat monitoring. With a little prep and help from field operatives, any additional data from ad hoc sources (bugs, shotgun microphones, and the like) can be fed into one of Xkeyscore's hundreds of local nodes and made available across the entire network. Facial recognition software is everywhere of course, and can easily be incorporated into these systems as needed.

To be honest, the framework is fairly primitive. Even clunky. It's the sheer amount of data they have available at their fingertips that's frightening. Each node of Xkeyscore, normally located in (or just outside) major metropolitan areas or in edge networking setups around the world, can pull down over 20 terabytes of data daily. Operatives and machine learning programs tag pieces of interest for permanent storage, and the rest of the storage space is recycled regularly.

Metadata is fed up through the Marina system for indexing. Pinwale takes care of E-Mail access and indexing. And the cycle of mass surveillance goes on.

Is Xkeyscore Legal?

As with many secret U.S. government programs, attempts to find out if the process and actions being taken are above-board have been stonewalled. In 2016, the Privacy and Civil Liberties Oversight Board asked for details on Xkeyscore operations. All they got was a 13-page memo from the NSA's legal team. In 2021, a classified report was filed, with additional unclassified comments detailing the lack of oversight that the government has over Xkeyscore; how it works, who has access, and how it has been enhanced since the 2013 Snowden leak is all a mystery, even to members of the U.S. government outside of the NSA.

To know if Xkeyscore is legal if they're properly getting warrants for cases that need them, and if they're respecting the difference in rules between international and domestic spying… one needs access. One needs to be able to examine the system that they're supposed to be providing oversight for. If the NSA won't give auditors the access they need, and the government won't shut them down for violating these requests for information and access, nobody outside of the NSA has any idea of the system's legality.

How Is Xkeyscore Being Used Today?

In late 2020, the Danish government was exposed as using Xkeyscore under the RAMPART-A agreement. The government's data on happenings within the country was being fed into the U.S. intelligence system for use by Five Eyes and their extended government partners.

This story is important for two reasons: Firstly, we know that Xkeyscore hasn't been retired and is still in active use under much the same function as it had in the past. And secondly, the NSA is still actively recruiting other governments to expand Xkeyscore use, despite it having no government oversight.

The Danes helped the U.S. spy on other European leaders. This all took place under the umbrella of Operation Dunhammer. That's a hell of a way to treat your allies.

Storage and computing power for the system and other mass surveillance programs have required expansion into the Cloud. Various private contracts have been awarded, and some challenged in court because of the oversight issue possibly impacting the bidding process, to the tune of billions of dollars. How much of that is dedicated to Xkeyscore-related systems is unknown… once again underscoring the importance of oversight.

With Biden in charge now, there are hopes that some form of review board can be re-established. But nobody has high hopes for that.

And That's That

So now you know about Xkeyscore… how it works, why we know as much as we know about the invasive system, and how it is being expanded all over the globe with absolutely no checks and balances. There's no rainbow at the end of this article, no note of hope. As far as we can tell, this is business as usual for the NSA and Five Eyes, and there's almost no hope of gaining any insight as to the legality or full extent of their activities any time soon.

Stopping or even examining a system like this requires political will. And since the source of Xkeyscore and the heart of its operations is in the United States, it has to be U.S. politicians and citizens leading the charge.

But as of Q1 2022, that's unlikely to happen. Privacy is dying a very public death, but a pervading feeling of helplessness has swept across the nation. Don't expect anything to change on the mass surveillance scene any time soon.