Sorry, that's classified: What you need to know about how government secrets are made

Classified information is a vast shadow world of data within the federal government. It provides secrecy for much of the information handled by agencies such as the State Department, the Pentagon and the CIA. Yet in the interest of open government, classified records are not meant to stay that way forever. Here's an overview of how the government handles its secrets.

What is classification?

The system of classifying records as confidential, secret and top secret (requiring the highest level of clearance) aims to prevent the release of information that could harm national security. Once a record has been classified, it's a federal crime to disclose it to the public by anyone other than the president.

How much information does the government classify each year?

In fiscal year 2016, the federal government created 39,240 new secrets, called original classification decisions. The number of new secrets has dropped substantially in recent years. It's unclear why there was such a huge spike in original classification decisions in the mid-1980s, but the Information Security Oversight Office of the National Archives concluded that some agencies, especially the Navy, may have submitted inflated numbers during this period and instituted better counting methods in later years.

However, lots of other new government records, from reports to memos to e-mails, may contain previously classified secrets and are therefore also classified. These are known as derivative classifications, and the government created more than 55 million records that incorporated existing secrets in fiscal year 2016.

Who has access to classified information?

About 2,200 federal officials in the United States have the power to make original classification decisions. As of 2015, more than 4 million public and private sector workers had security clearances, meaning they have access to classified information. More than 1.3 million of those individuals had top-secret clearance.

How much does all of this cost?

Keeping all those secrets is expensive. The federal government spent a record $16.89 billion on its classification system in fiscal year 2016, and private contractors spent an additional $1.27 billion.

Can secrets be declassified?

With exceptions for nuclear weapons data, no classified record is supposed to stay that way forever. President Barack Obama signed an executive order in 2009 creating a new system for declassifying records, with a presumption that records will be released after 25 years.

Agencies can decide on their own, or after a request, to declassify a record. It gets more complicated if more than one agency is involved – each one has veto power over declassification. The National Declassification Center, based at the National Archives, coordinates the process when records have been turned over for archiving and multiple agencies are involved.

So, how is it working?

Obama's executive order also set up systems for declassifying records more efficiently and allowing appeals by people who have been denied classified records. Nevertheless, researchers who specialize in declassified records say the process is too slow, and allows "securocrats" at each agency extraordinary power to withhold records that should be public.

The House Oversight Committee last year concluded that anywhere from 50 to 90 percent of records are improperly classified, meaning they have more protection than is really needed. The 9/11 Commission concluded that excessive secrecy by intelligence agencies was one factor in the failure to stop the 2001 terror attacks.

Do you have any absurd examples of overclassification?

Last December, Tom Blanton of the National Security Archive, a research project based at George Washington University, presented this document to the House Oversight Committee as an example of wasteful secrecy.

In 1986, the Joint Chiefs of Staff stamped "top secret" on a three-page memo intended for President Ronald Reagan's national security advisor. It was a general discussion of U.S. military objectives towards the Soviet Union in the 1980s.

In 2005, the National Security Archive filed a Freedom of Information request to the Department of Defense for the record.

The government decided that three offices had to sign off on releasing the record. The Office of the Secretary of Defense gave the okay. So did the National Security Council. But Mark Patrick, chief of the information management division at the Joint Staff, redacted all but one paragraph of the three-page document.

He did so under the authority of laws that protect confidential sources, war plans and sensitive information about foreign governments, even though the Soviet Union ceased to exist in 1991. The mostly blacked-out document arrived at the National Security Archive in November, 11 years after the FOIA was filed.

A staffer at the National Security Archive thought the document looked familiar. So he looked in the files, and found the same record had been released separately, with no redactions, in 2010 by the Department of Defense.

Blanton told the House committee: "And what's in there? Just the Joint Chiefs' comments on a draft presidential directive for our mid-1980s strategy against the Soviet Union. No weapons systems design. No intelligence assets. It's a waste of the Committee's time even to read this out loud."

This article has been updated after the release of 2016 figures by the ISOO.