Robert Steele has been a prescient thinker in the fields of search and content processing for decades. Best known for his work in open source intelligence, Mr. Steele has published widely on what I call “politico-info issues.”
Robert Steele, open intelligence source visionary
One April 2, 2013 Mr. Steele and I continued our discussion of online information which which appeared in Beyond Search in May of 2008. Most recently, The full text of my discussion with him appears below:
What is your background in intelligence?
I first started dealing with Top Secret Sensitive Compartmented information (TS/SCI) in 1976. I was selected to be the S-1/Adjutant for Battalion Landing Team 3/4 (3rd Battalion, 4th Marines, heavily reinforced) and found myself as a first-tour lieutenant in a Chief Warrant Officer job. In this position, I was responsible for the personnel management and security, including the security of all classified materials, for over 1,500 Marine embarked on six ships visiting six countries over six months.
In 1979 I was competitively selected to join the Central Intelligence Agency’s clandestine (spy) service, and was released from the Marine Corps to accept that appointment. I spent nine years with the CIA, doing three tours overseas as a case officer recruiting and handling agents, and then returned to Washington to do three desk tours, one in counterintelligence, one in advanced information technology, and one in futures planning and programming for technical systems (satellites).
In 1986, due to a lack of intelligence support from the national or other intelligence services, the Marine Corps invited me to help write the Marine Corps Master Intelligence Plan (MCMIP) as well as a plan for a Marine Corps Intelligence Center (MCIC). In 1988 I resigned from the CIA to become the second-ranking civilian in Marine Corps intelligence, and the senior civilian responsible for creating the MCIC from scratch, Planning and Programming Factors for Expeditionary Operations in the Third World. Published in 1990, this included the creation of an original analytic model with 144 factors across military, civilian, and natural-geographic domains, each with three to five degrees of difficulty defined by the warfighters themselves. This had never been done before and I do not believe it has been done since. This is precisely what the United Nations (UN) and the North Atlantic Treaty Organization (NATO) each need today. My Foreword to the first book just published by NATO on public intelligence, Internet-Based Intelligence in Public Health Emergencies – Early Detection and Response in Disease Outbreak Crises is a concise articulation of the challenges and the benefits to NATO.
In 1992 I became the champion for Open Source Intelligence (OSINT), and against some considerable opposition from the CIA, I received permission from the Marine Corps to organize an international conference. To my astonishment, we had over 620 attendees. This scared both CIA and the Marine Corps lawyers, so I was forbidden to run a second conference. I resigned after 18 years of service, giving up any possibility of a pension, in order to pursue what I believed to be the essential foundation for the future of intelligence. Today, 31 March 2013, marks precisely 20 years of “walk-about” as the CEO of Open Source Solutions Network, Inc. and more recently the non-profit, Earth Intelligence Network. Everything – over 30,000 pages – from all these years is free online at Phi Beta Iota Public Intelligence Blog.
It certainly sounds like through your almost 40 years of employment you have experienced virtually every aspect of the intelligence world from public to private. What three lessons remain with you from those 40 years?
My first lesson was that, for all the money we spend on it, the secret world is not really providing the return on investment taxpayers should expect. Intelligence – decision support – is simply not being provided to everyone that needs it. From my first tour handling secrets, I remember most clearly shredding it all each morning – nothing being sent to my Commanding Officer (CO) was the slightest bit relevant to our mission and our next location. In 1988 the Marine Corps spent 20 million dollars on an information system that could receive all secrets from all agencies, and we were stunned to find out that there was nothing in the secret databases relevant to our needs. Years later, General Tony Zinni, then the Commanding General of the US Central Command (USCENTCOM), would go on record saying that he got, “at best” 4% of what he needed each day from secret sources and methods. Everything else had to be found from human, print, and online open sources.
My second lesson, is that 95% or more of what we need to create ethical evidence-based decision support is available inexpensively and openly from academics, civil society, commerce, governments, law enforcement organizations, the media, all militaries, and non-governmental/non-profit organizations. It has been a real sadness for me to see the US secret world grow from $20 billion a year to $80 billion a year over the past 20 years, and still not produce anything useful. For just $3 billion a year, an Open Source Agency (OSA) could meet 90% of our needs with the added advantage of being able to meet the needs at all levels of government–including Congress, and also public needs across society from school rooms to board rooms.
My third lesson, is that all of our checks and balances are broken. I was invited to be the keynote speaker at a forthcoming conference in Wales on the past, present, and future of intelligence. In the course of preparing my remarks, I had the opportunity to plot all of the reform articles, books, and testimony, with spikes from 1992-1994, and then after 9/11, from 2002-2004. What really jumped out at me is the lack of congressional oversight. Traditional academics have not done broad research and ignore all sources beyond their limited “approved” circle. Additionally, the media simply parrots the party line and does no investigative journalism in the intelligence arena. Everything I would want to do today if I were the Director of National Intelligence (DNI) was clearly articulated between 1989-1994, but ignored. The Aspin-Brown Commission recommendations in 1996 were quite well-developed and none have been implemented. The 9/11 Commission produced little of value because it did not ask the tough questions. 9/11 was a policy and integrity failure at the political level, not an intelligence failure at the professional level. However, the 9/11 Commission was helpful in recommending the creation of an Open Source Agency (OSA) that is co-equal to the CIA. Unfortunately, as with all the other commissions over history, the bureaucracy, the industrial complex, and Congress, all remain static.
Where do you believe open source intelligence fits into all of this?
In order to appreciate the importance of open source intelligence it helps to emphasize that intelligence is synonymous with decision-support – the output of a very robust process of requirements definition, collection management, source discovery and validation, multi-source fusion, historically- and culturally-informed analytics, and the sharp visualization that answers an important question for a particular decision-making considering a particular decision challenge.
Few realize that most of what is produced by the secret world is not intelligence at all. Rather, it is secret information that is generic in nature and often not useful to decision-makers. We are at the very end of the Industrial Era and, due to grand legalized fraud by the banks, our economies are imploding. We are at the end of the Era where 50% of every government dollar could be documented waste, and the federal government could get away with borrowing one trillion dollars a year to feed all the earmarks that yield a 5% kickback to Congressional reelection campaigns. We need to begin making data-driven decisions, decisions that are both whole systems in nature, understanding cause and effect, and also deeply rooted in true cost economics, with a full appreciation for how much water, fuel, toxins, and child labor as well as tax avoidance are represented by any given product, service or behavior.
After I realized that the secret world had little substance to offer in support of Marine Corps strategy, policy, acquisition, or most operations, I moved into the open source world. I also inventoried open sources, open softwares, and open services around the world – sources and methods that were active and accessible in 183 languages, in comparison to the six or seven that are common in the secret world, where most analysts work in English and have very little foreign, historical, or cultural experience.
When Lt Gen Dr Brent Scowcroft, USAF (Ret) and the other members of the UN High-Level Panel on Threats Challenges, and Change identified and prioritized the ten high level threats to humanity, in 2004, I was struck by both their coherence, and the degree to which open sources rather than secret sources were the primary means of addressing each. I was so inspired by this that I funded the creation of the Earth Intelligence Network, a 501c3, and pulled together 23 others to design the World Brain and Global Game, with the intent of enabling participatory democracy and consumer-driven commerce. I did a study of the utility of open source against these ten threats, my findings were published in 2007, and the bottom line is that open sources are 95-99% of the answer for the non-military, non-terrorist, non-criminal threats, and no less than 75-85% of the answer for war, proliferation, genocide, and crime. For decades we have been looking through the wrong end of the telescope, trying to drill down to a few secrets, and ignoring the wealth of insight available in open sources.
In the 1990’s I served on the Information Handling Committee (IHC) and also the Advanced Information Processing and Analysis Steering Group (AIPASG) for the US IC. In retrospect I realize that all of the different secret compartments fragmented how we spent $1.5 trillion over 20 years, and that is why we lack a comprehensive Whole of Government cloud today, a multi-lingual automated warning system, or an all-source fusion analytic workstation such as we knew we needed in 1986 – it was called CATALYST for Computer Aided Tools for the Analysis of Science & Technology. Today, at the US Special Operations Command (USSOCOM), for many years the top OSINT producer (unlike the CIA’s Open Source Center (OSC), with produces Open Source Information (OSIF) that they then classify secret), they are still using over 25 softwares none of which share data effectively, all non-interoperable, all requiring fat-fingering.
Excellent points, how does open as a concept match up with the need for secrecy?
This is a very important question, because most people do not understand that open source software is vastly more secure than proprietary software. In my experience, secrecy is used by governments and corporations primarily to avoid accountability. Transparency, truth, and trust are the common currency for the 21st Century, in my view, and I am quite certain that going “all in” on Open Source Everything (OSE), a story I tell in THE OPEN SOURCE EVERYTHING MANIFESTO: Transparency, Truth, & Trust (North Atlantic Books / Evolver Editions, 2012) is the only affordable, scalable means of creating a prosperous world at peace that works for all.