Public access to microdata through the Integrated Data Infrastructure

Oct 16
Author:Che Tibby
Share on LinkedInTweet about this on TwitterGoogle+Share on FacebookEmail to someone

“Privacy By Design”, “Social Licence”. If you say them often enough they become mantra by which an agency can start to embed a culture of data safety among its people.

The world of public sector ICT is littered with the corpses of high-spend projects that died from a thousand cuts of outrage because no-one asked the simple question, “Is this going to make anyone feel creeped out?”

Privacy by design is pretty straightforward – you need to think through how to ensure constant observation of privacy in the early stages of your build. Not so much thinking that a preoccupation with security stops development, but enough so that privacy and security aren’t something someone outside your business needs to point out to you. If you find yourself on the front page of a city’s major newspaper because an ICT-savvy citizen uncovered the tax records of a senior political figure, then finding your next job could be challenging.

Social licence is essential to any project that uses public data. Without properly understanding your social licence, it’s possible for your carefully managed and curated product to find itself being launched into the stone-cold glare of an unaccepting public.

At Statistics New Zealand, we’ve seen the rapid growth of the Integrated Data Infrastructure (IDI) since 2011. Starting as a small integration project, the IDI now encompasses all major government administrative datasets, and offers microdata to hundreds of researchers in dozens of research domains. If a research project can be demonstrated to be in the public interest (a strict requirement of our enabling legislation), a research team can be provided with a select set within a suite of potential data sources. Interested in knowing if trainee doctors head overseas when qualified? You can find out. Interested in knowing how much that expensive degree will earn you? You can find out.

Maintaining trust in Statistics NZ

The quid pro quo attached to this unprecedented level of access to microdata is that we expect users to observe appropriate levels of security, and maintain the privacy of New Zealanders. Why? Because research to date demonstrates that the New Zealand public trusts Statistics NZ to protect its data. This trust has been built up over decades, and the maintenance of that trust is perhaps the only reason we’re able to bring together as much data as we have.

But it’s a balancing act. Pressure to open as much data as possible is constant, especially where the potential value the data offers is substantial. This means that we need to remain responsive to our users, our customers, while actively managing and maintaining public trust. In large part we do so by proactively communicating the real-world benefits integrated data offers, in an effort to move people from the naïve trust built up over time to an informed trust based on direct communication.

Is it working? Well, to date we’ve had no significant data breaches (although we have had some difficult conversations with researchers), demand is increasing, and we have the support of the Privacy Commission.

Guess we must be doing something right.

Che Tibby will be speaking on ‘Public access to microdata through the Integrated Data Infrastructure’ at the Data Analytics for Effective Decision Making conference in Sydney this November. Book your place by November 4th to save $100 on ticket prices!

Data analytics 2016

Submitted by Che Tibby

Che Tibby

Che Tibby is the lead analyst on the Integrated Data Infrastructure at Statistics New Zealand, and has helped carry a prototype system forward to a fully-fledged all-of-government IT solution. Che’s skillset is built on having studied political philosophy at the University of Melbourne, working as a taxi dispatcher in Auckland, running a pizza shop in Wellington, and washing dishes for a café in Carlton. He thinks the things he learned make him pretty well the ideal candidate to lead a team of highly-skilled systems analysts.

Leave a Comment

Your email address will not be published. Required fields are marked *

Other blog posts you may enjoy: