Blog

33c3 – Syrian Archive

On of the most interesting and important projects reported on at the 33c3 was the Syrian Archive project. This is an immensely important project that is impart documenting the Syrian conflict, including the human cost, but is also trying to help work towards a lasting peace in Syria. A major component of this work involves the curation of documentary evidence.

This includes evidence gathering and documentation of incidents; the acknowledgement that war crimes and human rights violations have been committed by all sides; the identification of perpetrators to end the cycle of impunity and the development of a process of justice and reconciliation.

— syrianarchive.org

The project which started in 2014 collects data, often in the form of images or video, from citizen journalists on the ground in Syria. The goal being to create an evidence based tool that can be used by journalists, HRDs and lawyers. The collected data is then securely stored on backed up servers, reducing the potential for loss of evidence. The project also builds meta-data for the evidence, which is often lost (particularly if the video is uploaded to social media services which often strip out the meta-data). Meta-data is often extremely important for the verification of the evidence as it helps to locate an incident temporally and spatially.

They also work to ensure the integrity of the data, including by producing a hash code of the data at the point of upload. This ensures that the evidence cannot be tampered with at some later point. All this is done through a range of simple tools. The result is a verifiable, searchable, and secure data repository that is accessible to anyone. The archive also allows for evidence to be cross referenced across multiple sources, and multiple platforms, helping to verify the claims.

This work is of great value as often in wars all sides seek to hide the full extent of their impact on the civilian population. The database has already proved instrumental in determining the facts around an air strike that wrongly hit a Mosque in Syria. Claims and counter claims cast doubt of the real events, with the Russian ministry of defence claiming that the Mosque was still intact, but witnesses claiming it had been destroyed. The data set allowed investigators to verify that a Mosque had been hit, and only that the name of the Mosque was incorrectly reported, leading to the confusion. Both the actual incident, and the claimed incident, can both be recorded in the database. The archive also allows the use of tactics or weapons to be tracked across multiple events, such as the use of chemical weapons.

The openness is key to this project, and links with some of my own research. We live in a world where different interested parties will make claims and counter claims about news or events. This makes it hard to determine which claim is best supported by the evidence on the ground. What this archive, and others like it, do is allow anyone to make an assessment of the evidence available, perhaps enabling them to understand the events in question better.

The talk was presented by Jeff Deutch and Hadi Al-Khatib, thanks to them for letting look at the slides again for reference. The videoed talk is linked below.

Panama Revisted

The people over at The International Consortium of Investigative Journalists have updated the released panama data. Its not clear to me if that is more data than they had already released, or that this time it is a ready made Neo4J database. They provide two versions of the database, Windows and Mac. Its easy to get it to work in Linux, just copy the graph.db file from out of the archive into the databases directory of your Neo4J install.

I made a quick query to look for officers with the same address. Seems there some, it would need something more sophisticated to did any deeper.

MATCH (n:Officer)–(a:Address)–(m:Officer) RETURN n,a,m LIMIT 25

graph

 

 

 

 

 

The ‘System’ vs Donald Trump

I have been thinking about what Donald Trump means for the ‘system’, by system I am thinking about the complex system that is the US Government Machine and its associated parts. Part of my thinking is that systems of Government have learnt how to persist, they have adaptive to promote and maintain their own existence. The internal system dynamics and relationships are an evolved, and are an emergent property of the ‘system’. The people in the system don’t even really know they are part of it. You could make the argument that this is plausible as a Government needs to be transparent enough to its citizens so that they don’t revolt, but not too transparent that they revolt. This adaption is a buffer between the people and the Government, and allows the system to persist (not statically, its a quasi-stable state). Its not just the Government that as adapted to this way of persisting, its the media, the elite etc etc. The laws, process, norms and culture make it hard to radically alter the system quickly without a massive shock or tremendous effort.

It does bring us to the question of Trump. Trump is a major upset to this entire system, he is a massive shock. So the question is, is the ‘system’ just going to grind him down into nothing. Has the US state ‘system’ seen enough Trumplike behaviour that there are emergent process that will just kick in and slowly squeeze him until he disappears. Maybe the position of stable state will move a bit, its always moving, but not much.

The other possibility is that he is enough of a disruption or shock that the whole system just breaks apart, and self-organises into something completely new… That could produce anything.

I am not sure which is worse.

US Election – The First Cyberwar?

The presidential election is starting to look like the front line in a cyberwar. With different national and global actors waging a war of information (or disinformation), the main weapon being deployed seems to be the sowing the seeds of doubt in the minds of the US electorate. Its hard to say where it all started, but let us start with the emails, hacking, Trump, Wikileaks and the Russians.

So the emails. Wikileaks has got hold of a large number of (I should point out that it is at least plausible that visits to Wikileaks are recorded…) emails from inside the Democratic party, they seemed to think that they were enough of a smoking gun to bring an end to Hillary’s campaign. Maybe during a normal election, there is however nothing normal about this election. That said the constant drip of damaging stories has proved to be, well damaging, just so far not terminal. Where did the emails come from? Well, some people think Russian hackers got hold of them and then handed them over to Wikileaks. There is a degree of credibility to this hypothesis, when exactly the Russians decided to hack into the servers, and what Russians it is however are unknown. Truth be told we don’t know if it was Russians, Wikileaks may not even know it was Russians however they say it wasn’t. So how might this cyberwar be playing out?

Its the Russians and Trump

Russian narrative is useful to the Democrats and they can link Trump (possibly quiet legitimately) to Russians, he likes Putin because Putin said nice things about him. They can also link Trump to the leak/hack, he encouraged the Russians to hack the democratic servers. Maybe, just maybe, Trump’s intervention was enough to get the Russians interested? The timing of that probably doesn’t make sense, but its an interesting thought. Trump therefore could have fired shots in this cyberwar.

Its Russia doing it for themselves

It is possible that this is Russian mischief making, state backed or not, and they did perhaps just for the lolz. Or, to potentially push America into a state of political chaos which they would ultimately benefit from. Keep America fighting itself so they can push on with their own geopolitical agenda. However wins the election next week could very well be instantly bogged down in in this mess, leaving less time for dealing with global issues.

Wikileaks, where do they fit in?

Wikileaks could very well just have received this early Christmas present in their submission system. They have then used it to cause maximum damage by slowly dripping information out, they would likely say this is to increase the exposure of them and the function they perform in global society. Exposing what should be in the public domain. I have sympathy for this ambition, however they are also editorialising this story to cause maximum damage. Not that any other media company would do any different; do we call them a media company now? As far as I have seen there is no evidence that Wikileaks solicited the leak in any way. Therefore it will be up to the public to decide if the emails are in their interest, and potentially important enough to end Clinton’s political ambitions. I think one downside is that many people just read the headlines and not the content.

Where does Trump really fit in?

Is Trump a puppet of Putin that is being supported by Russian hackers? Its not totally out of the question. What the Russians are doing for Trump is well at least two things. One, if they did the hack, is provide a rich source of damaging headlines for Clinton. Trump should be nowhere in this election, but yet he hangs on somehow, helped by Clintons skeletons. The other is that the leaks provide a useful deflection from Trump’s own skeletons, not least the series of women that claim to have been sexually assaulted by him. Claims that have a degree of credibility. The thing that surprises me most about Trump is that anyone finds him at all compelling or even coherent. His speeches are often bizarre, drifting, rants that are difficult to follow. There is almost no policy, other than a few bits about walls and swamps. His suggestion that the election is rigged (unless he wins) just makes it more difficult to believe anything that comes out of this election.

So where does this leave us? Well, knowing very little for sure. This whole thing is the perfect disinformation operation, and it might not even have an orchestrator. The whole election campaign is now post truth, because the truth has become so hard to find in amongst the lies and half-truths. There are so many views, opposing views, and facts on both sides. So many agendas and seeds of doubt being sown, how does anyone make sense of what has been said, or assess its credibility? Cyberwar isn’t just about turning off the lights, it can be more subtle, and in this cyberwar its any sense of the facts and important issues that have been lost into the darkness.

Java Panama Papers Neo4J Network Generator

Further to the first attempt at importing the Panama Papers network data into Neo4J I did a very quick Java program that greats an embedded Neo4J database. It needs a bit of checking as it finds nodes that have the same node_id. Which I assume is some sort of mistake in the program or the data, it also looks like there is some duplicate relationships.

This program generates relationships of the different types. Such as ‘officer_of’, rather than the hack used to get Cypher to import the data (see earlier post).

The code can be found in my new github.

Below is Blairmore, Ian Cameron, the intermediary, and loads of other companies that use the same intermediary.

Fig2

 

 

 

 

 

 

 

Not many directly links to Blairmore.

Panama Papers: Import Data to Neo4J using Cypher

I downloaded the panama paper network data, I was hoping it would be all the data, sadly not. Its it still interesting however. The import process is not to tricky. The following Cypher commands will get the data into a running Neo4J database. Note there is a \” in the Addresses file that will break the import. Search for it an replace with \ “. Data can be downloaded from here.

To get the relationships in we have to do a bit of hack as you cannot generate a relationship type on the fly from a CSV file with Cypher. I will do this properly with a bit of Java soon.

Change the paths! This is for the Addresses:

USING PERIODIC COMMIT LOAD CSV WITH HEADERS FROM 'file:/path/Addresses.csv' AS line CREATE (:Addresses { address: line.address, icij_id: line.icij_id, valid_until: line.valid_until, country_codes: line.country_codes, countries: line.countries, node_id: toInt(line.node_id), sourceID: line.sourceID})

For the Intermediaries:

USING PERIODIC COMMIT LOAD CSV WITH HEADERS FROM 'file:/path/Intermediaries.csv' AS line CREATE (:Intermediaries { name: line.name, internal_id: line.internal_id, address: line.address, valid_until: line.valid_until, country_codes: line.country_codes, countries: line.countries, status: line.status, node_id: toInt(line.node_id), sourceID: line.sourceID})

Officers:

USING PERIODIC COMMIT LOAD CSV WITH HEADERS FROM 'file:/path/Officers.csv' AS line CREATE (:Officers { name: line.name, icij_id: line.icij_id, valid_until: line.valid_until, country_codes: line.country_codes, countries: line.countries, node_id: toInt(line.node_id), sourceID: line.sourceID})

Entities:

USING PERIODIC COMMIT LOAD CSV WITH HEADERS FROM 'file:/path/Entities.csv' AS line CREATE (:Entities { name: line.name, original_name: line.original_name, former_name: line.former_name, jurisdiction: line.jurisdiction, jurisdiction_description: line.jurisdiction_description, company_type: line.company_type, address: line.address, internal_id: line.internal_id, incorporation_date: line.incorporation_date, inactivation_date: line.inactivation_date, struck_off_date: line.struck_off_date, dorm_date: line.dorm_date, status: line.status, service_provider: line.service_provider, ibcRUC: toInt(line.ibcRUC) , country_codes: line.country_codes, countries: line.countries, note: line.note, valid_until: line.valid_until, node_id: toInt(line.node_id), sourceID: line.sourceID})

Finally the relationships, or edges. Note the hack, all relationships are of type ACCOC. This isn’t a big problem but offends me a little bit. I will post you Java code that generates the graph dir from the files.

USING PERIODIC COMMIT LOAD CSV WITH HEADERS FROM 'file:/path/all_edges.csv' AS csvLine
MATCH (n1 { id: toInt(csvLine.node_1)}),(n2 { id: toInt(csvLine.node_2)})
CREATE (n1)-[:ACCOC {role: csvLine.rel_type}]->(n2)