Can Web 2.0 penetrate the Intelligence Community?

Clive Thompson’s article, Open Source Spying, in this Sunday’s New York Times Magazine was a fascinating look at how Web 2.0 is creeping its way into the least-open IT environment imaginable – the US Intelligence community.

The article starts by describing problems well-known to anyone who has tried to provide technical solutions to the major TLA agencies: outmoded technologies fomented by technical and policy walls prevent any meaningful sharing of information. 

I spent time in 2003-2004 delivering analytic technology solutions to the Intelligence community.  While advanced technologies had a few advocates in some high places (among others former Navy Admiral and “Total Information Awareness” sponsor John Poindexter was a big fan of our technology), there were hurdles in place that were too high to clear. 

Probably the greatest hurdle was the lack of sophistication of the actual agency employees combined with the greed and arrogance of the systems integrators who “served” them.  The defense and intelligence communities have been outsourced to private contractors for the past 25 years.  This was started by Reagan, who believed you could downsize government by cutting a $40k per year civil servant and replacing them with an $80k per year private contractor (often the same person but now on a different payroll).  Over time, the government lost its ability to recruit strong IT minds, and those who did join would quickly shift to the private sector where they could get a huge raise for performing the same job.

Since the agencies were left with little IT expertise, they became highly dependent upon contractors, large systems integrators like Lockheed, Northrop Grumman, SAIC and others.  Unfortunately, efficiency and public safety are lower priorities than revenue growth for these firms.  During my time in Washington, I found that the systems integrators put up walls to keep “off-the-shelf” software out, even while Corporate America had embraced COTS solutions as opposed to custom software.  COTS software is less expensive, easier to support, easy to integrate and has a much lower level of implementation risk than a custom solution, but systems integrators can make more money having a team of 50 people spend two years building something than having a team of 10 implement something that already exists.

The Times article describes recent efforts to change the system, led by Dale Meyerrose, CIO of the new Director of National Intelligence.  Meyerrose instituted a mandate of using COTS solutions (mainly for compatibility reasons) and has also begun to tackle some of the cultural obstacles.

At the same time, the DNI took over a CIA program that explored new methods of gathering and sharing intelligence.  Among the first methods considered were use of wikis and blogs.  Could decentralized tools like wikis and blogs survive in the most centralized of all IT environments?

Tests are still underway, but early results are compelling.  As an example of how this could work, the article describes how a wiki was used to capture and update information about the crash of a private commuter plane into a Manhattan apartment building.  Over the course of about two hours, that page was updated 80 times by analysts from nine different agencies.  They were quickly able to reach the conclusion that this was not a terrorist-related incident.  How long might that have taken in the traditional model, where analysts at different agencies were unable to share information?

Will Web 2.0 applications solve the many woes of the intelligence community?  I think that it will take more than a few innovative programs to weaken the multi-billion dollar grip the contractor community has over intelligence.  But, with the right leadership (and perhaps some committee hearings on war profiteering), it’s possible that more user-generated content initiatives will displace the failed knowledge management projects of the past.  And if these efforts can begin to take hold in the command-and-control centric world of the intelligence community, just think what impact they might have in a more flexible organization like your own.

Comments

[Scott Parry]

The industrial military complex just put you on their "watch list".

Expect suspicious interuptions and interference in your

[BREAK lak sjdh k kjhkjj jsdhkjhsk djhksjdhfkjs dfs

s
dfsjdf sdf
sdf skdfjhs df
sdsdsdflj kjsdh jsdfkjsdfs

sd Northrup fuaq 934y sjdhf i sdfjh s9pe8ry q234r anerase

werwef
w3r
23r
w
tr
wre

789789897 Shut him down 890908
19283hjuahe f90823h4kj wer
232342398y2 h3ji4

234kjh2 3498723kj4
2o34uyb923847b234jasdkfjhs dfjnn
UNBREAK]

communication devices.

If you get this, email back so I know you are still with us.

Scott