ICIJ is using Talend’s open source integration software to load more than 2.6TB of unstructured data into a Neo4j graph database, which uses the Linkurious graph visualization platform to organize and access the information. The data includes relational database files, emails, and various types of documents concerning the 215,000 offshore bank accounts and shell companies that the law firm and its predecessors created for thousands of individuals between 1977 and 2015.
“The Panama Papers is a data analysis project that is significant on many levels and the kind of work requiring intuitive big data technology,” said Laurent Bride, CTO, Talend. “We are very pleased that our solution was used to support this project and in conjunction with our long-standing partner Neo4j.”
The major leak of documents was originally obtained by German newspaper S?ddeutsche Zeitung, which it then shared with Washington-headquartered ICIJ, a network of independent reporting teams around the world and more than 100 media partners. Later today, the ICIJ will release a searchable database with information on more than 200,000 offshore entities that are part of the Panama Papers investigation, as well as information concerning over 100,000 additional companies that were part of the 2013 ICIJ Offshore Leaks investigation. Using Talend software, ICIJ will use the database to rapidly ‘connect the dots’ between the corporate information for secret offshore companies and the people behind them.
