The National Security Agency may not be the first organization that comes to mind when you think of contributors to open source software projects. But over the last few years, as we reported last month, the agency created and open sourced an rather interesting software platform known as Apache Accumulo. Basically, it's a "NoSQL database" for handling massive data sets securely.
As we mentioned last month, Sqrrl -- a company spun out from the NSA to commercialize Accumulo -- recently received seed funding from two major venture capital firms, and today, the fledgling outfit announced that those two firms were Atlas Venture and Matrix Partners, which made a total investment of $2 million.
Accumulo is based on a research paper published by Google in 2006 describing its own massive database, BigTable. The NSA wasn't the only organization to try to clone BigTable. Other similar projects include Apache Cassandra (which was created by Facebook and incorporates ideas from both the BigTable research paper and from a paper published by Amazon.com on its Dynamo data storage system) and HBase (which was created by Powerset before it was acquired by Microsoft and is also used by Facebook).
One of the biggest differences between Accumulo and other BigTable clones is the level of security the NSA built in. Referred to as "cell level security," it allows granular security to be applied to every piece of data included in the database, even if when that data is distributed throughout a large cluster of servers.
Sqrrl CEO and co-founder Oren J. Falkowitz -- who worked on Accumulo for the NSA -- says the agency couldn't have just contributed additional features to an existing project. "Back in 2008 when the project was started there just wasn't a viable project to latch on to," he says. "HBase existed, but it was a very different project." He also says that the security features found in Accumulo couldn't just be bolted on to Hbase or Cassandra. "It would require practically an entire re-write of Hbase."
Falkowitz argues that the NSA's use of Accumulo proves its scalability and readiness for the enterprise. He couldn't comment on how the software is being used at the NSA, but given the capabilities of the agency's data centers, we can deduce the organization has some pretty serious data storage requirements.
And it's not just the NSA using the software. For example, he says End Game Systems, a portfolio management company, recently switched from Cassandra to Accumulo. He says Sqrrl is seeing a lot of interest from the financial sector and other companies with high security needs.
Sqrrl will compete with DataStax, a company that commercializes Cassandra, and Cloudera, which provides enterprise support and tools for Hbase and Hadop.
