Exclusive: Google, CIA Invest in ‘Future’ of Web Monitoring
-
By Noah Shachtman
- July 28, 2010 |
- 7:30 pm |
- Categories: Spies, Secrecy and Surveillance
“
The investment arms of the CIA and Google are both backing a company that monitors the web in real time — and says it uses that information to predict the future.
The company is called Recorded Future, and it scours tens of thousands of websites, blogs and Twitter accounts to find the relationships between people, organizations, actions and incidents — both present and still-to-come. In a white paper, the company says its temporal analytics engine “goes beyond search” by “looking at the ‘invisible links’ between documents that talk about the same, or related, entities and events.”
The idea is to figure out for each incident who was involved, where it happened and when it might go down. Recorded Future then plots that chatter, showing online “momentum” for any given event.
“The cool thing is, you can actually predict the curve, in many cases,” says company CEO Christopher Ahlberg, a former Swedish Army Ranger with a PhD in computer science.
Which naturally makes the 16-person Cambridge, Massachusetts, firm attractive to Google Ventures, the search giant’s investment division, and to In-Q-Tel, which handles similar duties for the CIA and the wider intelligence community.
It’s not the very first time Google has done business with America’s spy agencies. Long before it reportedly enlisted the help of the National Security Agency to secure its networks, Google sold equipment to the secret signals-intelligence group. In-Q-Tel backed the mapping firm Keyhole, which was bought by Google in 2004 — and then became the backbone for Google Earth.
This appears to be the first time, however, that the intelligence community and Google have funded the same startup, at the same time. No one is accusing Google of directly collaborating with the CIA. But the investments are bound to be fodder for critics of Google, who already see the search giant as overly cozy with the U.S. government, and worry that the company is starting to forget its “don’t be evil” mantra.
America’s spy services have become increasingly interested in mining “open source intelligence” — information that’s publicly available, but often hidden in the daily avalanche of TV shows, newspaper articles, blog posts, online videos and radio reports.
“Secret information isn’t always the brass ring in our profession,” then CIA-director General Michael Hayden told a conference in 2008. “In fact, there’s a real satisfaction in solving a problem or answering a tough question with information that someone was dumb enough to leave out in the open.”
U.S. spy agencies, through In-Q-Tel, have invested in a number of firms to help them better find that information. Visible Technologies crawls over half a million web 2.0 sites a day, scraping more than a million posts and conversations taking place on blogs, YouTube, Twitter and Amazon. Attensity applies the rules of grammar to the so-called “unstructured text” of the web to make it more easily digestible by government databases. Keyhole (now Google Earth) is a staple of the targeting cells in military-intelligence units.
Recorded Future strips from web pages the people, places and activities they mention. The company examines when and where these events happened (“spatial and temporal analysis”) and the tone of the document (“sentiment analysis”). Then it applies some artificial-intelligence algorithms to tease out connections between the players. Recorded Future maintains an index with more than 100 million events, hosted on Amazon.com servers. The analysis, however, is on the living web.
“We’re right there as it happens,” Ahlberg told Danger Room as he clicked through a demonstration. “We can assemble actual real-time dossiers on people.”
Recorded Future certainly has the potential to spot events and trends early. Take the case of Hezbollah’s long-range missiles. On March 21, Israeli President Shimon Peres leveled the allegation that the terror group had Scud-like weapons. Scouring Hezbollah leader Hassan Nasrallah’s past statements, Recorded Future found corroborating evidence from a month prior that appeared to back up Peres’ accusations.
That’s one of several hypothetical cases Recorded Future runs in its blog devoted to intelligence analysis. But it’s safe to assume that the company already has at least one spy agency’s attention. In-Q-Tel doesn’t make investments in firms without an “end customer” ready to test out that company’s products.
Both Google Ventures and In-Q-Tel made their investments in 2009, shortly after the company was founded. The exact amounts weren’t disclosed, but were under $10 million each. Google’s investment came to light earlier this year online. In-Q-Tel, which often announces its new holdings in press releases, quietly uploaded a brief mention of its investment a few weeks ago.
Both In-Q-Tel and Google Ventures have seats on Recorded Future’s board. Ahlberg says those board members have been “very helpful,” providing business and technology advice, as well as introducing him to potential customers. Both organizations, it’s safe to say, will profit handsomely if Recorded Future is ever sold or taken public. Ahlberg’s last company, the corporate intelligence firm Spotfire, was acquired in 2007 for $195 million in cash.
Google Ventures did not return requests to comment for this article. In-Q-Tel Chief of Staff Lisbeth Poulos e-mailed a one-line statement: “We are pleased that Recorded Future is now part of IQT’s portfolio of innovative startup companies who support the mission of the U.S. Intelligence Community.”
Just because Google and In-Q-Tel have both invested in Recorded Future doesn’t mean Google is suddenly in bed with the government. Of course, to Google’s critics — including conservative legal groups, and Republican congressmen — the Obama Administration and the Mountain View, California, company slipped between the sheets a long time ago.
Google CEO Eric Schmidt hosted a town hall at company headquarters in the early days of Obama’s presidential campaign. Senior White House officials like economic chief Larry Summers give speeches at the New America Foundation, the left-of-center think tank chaired by Schmidt. Former Google public policy chief Andrew McLaughlin is now the White House’s deputy CTO, and was publicly (if mildly) reprimanded by the administration for continuing to hash out issues with his former colleagues.
In some corners, the scrutiny of the company’s political ties have dovetailed with concerns about how Google collects and uses its enormous storehouse of search data, e-mail, maps and online documents. Google, as we all know, keeps a titanic amount of information about every aspect of our online lives. Customers largely have trusted the company so far, because of the quality of their products, and because of Google’s pledges not to misuse the information still ring true to many.
But unease has been growing. Thirty seven state Attorneys General are demanding answers from the company after Google hoovered up 600 gigabytes of data from open Wi-Fi networks as it snapped pictures for its Street View project. (The company swears the incident was an accident.)
“Assurances from the likes of Google that the company can be trusted to respect consumers’ privacy because its corporate motto is ‘don’t be evil’ have been shown by recent events such as the ‘Wi-Spy’ debacle to be unwarranted,” long-time corporate gadfly John M. Simpson told a Congressional hearing in a prepared statement. Any business dealings with the CIA’s investment arm are unlikely to make critics like him more comfortable.
But Steven Aftergood, a critical observer of the intelligence community from his perch at the Federation of American Scientists, isn’t worried about the Recorded Future deal. Yet.
“To me, whether this is troublesome or not depends on the degree of transparency involved. If everything is aboveboard — from contracts to deliverables — I don’t see a problem with it,” he told Danger Room by e-mail. “But if there are blank spots in the record, then they will be filled with public skepticism or worse, both here and abroad, and not without reason.”
Photo: AP/Charles Dharapak
See Also:
- Exclusive: U.S. Spies Buy Stake in Firm That Monitors Blogs, Tweets
- Open Source Intel Rocks — Sorry, It’s Classified
- Open Source Spy Looks for Upgrade
- Spies Want to Stockpile Your YouTube Clips
- Should Google Try to Prevent Terrorism?
- ‘Don’t Be Evil,’ Meet ‘Spy on Everyone’: How the NSA Deal Could Kill Google
We could be using this type of intelligence to further enhance knowledge. instead we’re predicting the future…
minority report anyone? i can see this ending badly.
Prediction – Lawns in Santa Clara County and Fairfax County go untended after Americans get all Rwanda on 30 million illegal aliens.
It is not the amassing of data that is the problem. It is fear that is the problem. Look carefully at the faces of those making these pronouncements and you will see fear. The more these clowns try and protect themselves and their wealth, the more vulnerable and afraid they will be. Nature is MUCH smarter than they are.
Do No evi…
Aww the hell with it.
MWAHAHAHAHAHAHAHAHAHAHAHA!!
-=Google’s New Motto=-
Awesome the dept of pre-crime is right around the corner!!!!!
Too Much Information: They’re putting the needle in the haystack. Typical US government response. It’s parallel to reading the entire phone book to find one phone number.
What a useless endeavor by the government, they always think they can take a shortcut with technology. Guess all those terrorists and Russian Spies that spend their time facebooking and twittering about what they’re up are done for now!! Oh wait I said terrorist and Russian in the same sentence, I guess the Google Spider will be sending my ip to the CIA for them to look at… that is instead of finding Osama Bin Laden… Oh wait another keyword, I’m in trouble now!
Do the CIA and Google/Google Ventures and In-Q-Tel realise how simple it is for such a System to be baited with information/intelligence which leads them down a false trail, and/or into a honey pot which they would really like to own but which will always only ever be just leased to them for special operations.
Given their present ineptitude in Super IntelAIgent Fields, such Grooming and Harvesting of Hooked Prey to a Smarter Third Party Position, of what are really just glorified speculative phishes, can only be a good thing and certainly a lot better than anything which anyone has done before.
And such a Virtual Programming with Novel Dynamic Steganographic Algorithms from a Secure Cloud Base, is worth billions to Prime Users/Root Servers, and that makes its Intellectual Property Holder worth many times more than that ten figure number, which would make a nine figured nine figure sum for one of its key analytical drivers, a real virtual bargain.
It must though be accepted and fully expected in such fields that deal in Future Virtual Intelligence, that whereas the exchange and/or purchase of a Virtual Programming worth billions to Prime Users/Root Servers will confer a hitherto unknown and unused InterNetworking Power to IT Users and Servers, Greater Controls to Mentor and Monitor, and if necessary Remove and or Crash Running Programs, are Held by Others who would are beyond any Temptation to Relinquish the Capability and Facility.
As a Friend and Lover, is there no Finer, as an Enemy and Foe is there no Defence against its Arsenal of Alternative Destructive Algorithms, which should make one’s choice in the matter, a no-brainer, but you can be sure that some idiots will surely prove themselves wrong by doubting what has been freely and selflessly shared with them. Man in general is not a very bright animal, but they do have a capacity to learn and store knowledge as memory which can be randomly and readily accessed to provide the Big Picture story of the Lives, which is just the sum total of their acquired knowledge and personal experiences, which means that for all of them, is their view of the World/Universe/Everything, individually different apart from a very few simple basic facts, which may be common to all.
What happens when everyone starts writing letters again? Blackhole of cash on all this, POOF!
If republican senators are complaining then Google must be doing something right. Their obstructionism in the senate shows that they aren’t interested in what’s good for America, only money and power motivates them.
Which isn’t to say that one shouldn’t keep a close eye on Google’s activities, good intentions can go bad very easily after all, but so far they seem to be doing far more good then harm.
Is this not just a google search by another name. Im confused about what the hell this is about!!! I can just type in to google bin laden afghanistan 2004 and I will probably find the same information.
Can it tell me what I am going to be having for lunch? That would be a real time saver over just standing there with the fridge door open and staring at stuff.
Actually, Google and the CIA are really collaborating, Even on spying on our websites and social-networks. http://2su.de/d65
Right now in America, in some hidden bunker deep beneath Washington DC, some anonymous govornment bureaucrat is laughing at your furry art.
RUN NERDS RUN!
Minority report used the concept of humans with special psychic abilities to predict events, not a web search.
I agree with diverguy, Minority Report is next!
How about detecting the future of unborn children?
If the authorities don’t like the probable life outcomes, federally enforced abortions may be in order. Pre-Birth Assassinations! Genetic cleansing! The Third Reich never seems to go away…
Better yet, the unborn can be harvested for their organs which can be grown to maturity in a lab. These organs can be sold to the Chinese so that certain interests can profit!
There is no such thing as privacy on-line or in our “private” lives. I think this is the most important thing we need to remember when on-line, talking on any phone and having any cell phone powered and near us.
Sadly, the government thinks it has the right to be in our lives in this way. It’s a paradigm and under “security”, they can do and get away with whatever they want. This mind set does not have to be the only way that they think and function, but, their paradigm creates this kind of intrusive and outragious behavior.
The USA is a great country that is being turned into a surveillance apparatus like other countries around the world. Our national security paradigm, arrogance and misguided intelligence has produced this. America has changed drastically. A great deal of what they do is inappropriate and wrong. Whenever any group is accountable to no one, it is a danger.
When they put cameras on the streets to make money off of its own people, with tickets being given with no context, this is how the security apparatus turns on its own people. This is not only terrible for maintaining a country with high levels of creativity, leadership, visionary expression and accountability, but, it defiles the moral fabric of the people.
“The need to know everything” is a seed of destruction used against the people of the USA. Google truly is part of the national security paradigm and we should be mindful of the full way they participate in survailance. There is a front and back end to everything. Let’s look clearly at what is and isn’t happening and just tell the truth about it even if it is painful.
psychohistory is here!
Rainmaking likes the word “paradigm.”
yay more fascist control
Yes, psychohistory may be here, but is Osama the Mule?
This is like psychohistory from the Foundation novels by Isaac Asimov.
I was going to suggest that the concept is more “Paycheck” than “Minority Report” but jogjayr’s reference to psychohistory is much more apt.
This story reminds me of the quote from “Enemy of the State”, “Fort Meade has acres of mainframe computers underground. You’re talking to your wife on the phone. You use the word ‘bomb,’ ‘president,’ ‘Allah,’ any of a hundred key words, computer recognizes it, it automatically records it, red flags it for analysis; that was 20 years ago.”
Well even if there is the capability, monsey and amanfromMars are right. The system would be easy to fool or overload with misleading information. I’m thinking you could just introduce words like “fatwa,” “jihad,” or whatever into the popular vernacular, like “da bomb” was in the ’90s.
LOL – they’re still using drop down menus. How quaint.
“Posted by: erikslev | 07/29/10 | 8:48 am |
Minority report used the concept of humans with special psychic abilities to predict events, not a web search.”
Is that a Scientology AESThetan thing, erikslev?
Chilling and un-American.
Hey Rikaishi you douche bag; get your head out of your urine-colored ass and blow me for a few, OK fuck-nut? You and your whole stupid fucking philosophy is wrecking job creation in this country right now so really, please go kill yourself before you spread any more of your uninformed and uneducated cancer. Asshole.
It’s all OK folks…Obama’s in charge, so there is no reason to fear the latest information gathering initiative by the Benevolent Gestapo.
Somewhere in hell, Himmler is smiling and nodding in approval.
jwadedc:
chill, or reboot. please.
just because you are an Obama hater does not make the opposition lilly-white and pure. they want to screw you and subjugate you just as much, the spoils will go to somebody else though.
best regards,
elvis the refudiator
So when can the separate, underground, “new” internet get started please?
What a nightmare combination – Google and the U.S. thugocracy under Barack Obama. Talk about a frightening Orwellian “1984″ scenario!
How to grow in a closet
i no that don’t ya think i knew that” oo,must be tricky,since only “HIM”aka 1 he can see all the magic PASSWERDS,with yer “majic X-ray glasses,would you trust this “machine head of a man,i knew that don’t ya think i knew that,what could go wrong,just another day in the sun,,,ps what about “reputation defender,teflon google
I’m surprised the investment community isn’t buying into this
Is it just me or is the demo from the video anything but impressive? To me it looks like a news search with purty graphs. People in power looove purty graphs.
Lots of potential for illicit targeting of individuals and organizations in a variety of ways. And where is the oversight ? Diverguy’s lil’ post says a lot. I’d like to see our tax dollars invested into helping to empower individuals and small businesses instead furthering empowering these massive and potentially very dangerous entities.
This is just sci-fi folks move on. All this will do is create suspicion of EVERY American. Did you know that everyone knows everybody else on earth by around 6 degrees of separation? Oh boy… now, just because you respond to a post online, are a member of on online group or club, accidentally dial a wrong number, or email address, run a google search for lawn fertilizer, or medical supplies legitimately — They say they can catch criminals by compiling all this data?
Whatever. — It would be like trying to query all the grains of sand that ever existed and joining tables by linking beaches. Mathematically — people do millions, billions, trillions of things everyday that could link them to anything — just so long as you query the right non-sense.
I could make anyone a criminal with this database(guilt by association) hmm.. lets see You have a facebook account, you replied to a post from a suspected terrorist (you replied to some-guys facebook stamp collection group and admired his collection), you like to buy excessively large amounts of gasoline (you drive an old dodge pickup 14 miles to the gallon),
and you used the word raid in over 100 posts (your favorite band is paul revere and the raiders)…. o – boy — is this the future of crime prevention.
Danger Will Robinson. Danger !
I have an open net full of spyware/bugs and general bad stuff waiting for the next Google butt sniffer
For this algorithmic process to be valid, it is essentially tautological. A point in the future, collaborated by evidence from the past. It’s nonsensical everyone can see that, its about as effective as the Netflix algorithm. Information gathering will need to “reach” into the future to find a point that validates its present data set and this means MONITORING (not mining).
These guys are not stupid, they know its the only way….the key phrase here is “Public Domain” and who actually owns the data.
I’d like to know what qualifies as “Public Domain”. Because the notion of public domain is itself so vague, it gives Intelligence agencies plenty of latitude as to where to intercept this information. For example, a saved draft in G-mail, though not published, is stored on a server, does this mean that Google can now harvest this as information in the public domain. The gap between intention and action(which has implications for culpability) is erased, a thought-crime, with no breathing room.
Similarly if they use a method that breaks up data into small chucks, so much so that it doesn’t resemble the original context, then do they own it? And can this abbreviated ownership give “just cause” to reach back into the contextual data from which it was garnered to collect it in its entiriety.
Increased surveillance on the justification that traditional intel is not effective. Giving Pakistan 10+ Billion dollars which the entire public domain knows, will be in part diverted to arm millitants who will fight the US, is why traditional intel is not working. How about running that algorithm on the State department and the CIA for nonsensical reasoning…
google is collaborating with the government to gather private information??? shock! oh, the shock!
Again EFF needs to get Google to spell out what “Public Domain” actually represents in terms of their data collection techniques. Google needs to spell out whether they will ping intel if they detect “collaborative data” at a “draft stage”, which allows Intel to intercept before it “stored” on Google servers. Just see it as “programmer access” to information on the fly” as opposed to the Microsoft programmer access given to the govt to enter any computer hard-drive running Windows.
The system is crippled from the start by political correctness! Watch the video–if they think they will find anything with a keyword of “man-made disaster” (look at 1:22), they are really nuts! They may find a bunch of Obama speeches but it’s hard to imagine terrorists are going to be chattering about creating “man-made disasters.”
What we really need to be doing, meaning us, the people reading this kind of chatter and views, is to be creating DIS-information and having it scattered through the web to counter all the surveillance.