Open Data: The Hard Lessons

    NAGW Navigator: Volume 1 • Issue 2 • Fall 2017

    By Ron Pringle

    The City of Boulder has been running an Open Data program for about four years now. In that time, we’ve had ups and downs, pursued things that ended up being dead ends and in some cases found paths to success. Based on our experience I’d like to highlight a few things that have and haven’t worked for us.

    Write a policy – We started without a policy and thought that if we built up enough momentum and shared successful case studies, the advantages of publishing open data would be magically obvious to everyone. They weren't. We eventually ran into roadblocks of the “you can’t make us” and the “we’re too busy” variety. Make open data a part of your culture by adopting a policy. Policies have political support. You need that.

    Build it and they will come – It’s 2017, you can’t just publish data and call it a day. While you might get lucky and have a vibrant community who instantly uses your data in wonderful, meaningful ways, chances are you won’t. Either way, you need to provide inspiration. We’ve done that by adding Areas of Inquiry which provide some possible starting points for working with the datasets in our catalog.

    Don’t overlook automation – The more you can automate the intake and publication of datasets the more sustainable your program will be. Don’t underestimate how hard Extract/Transform/Load (ETL) work can be and make sure you have the capacity to do the work. In our case, we piloted various ETL strategies but have still run into bottlenecks due to staff turnover or lack of availability due to other projects.

    Create an open data handbook – Our handbook is geared towards our departmental data stewards who are responsible for inventorying and submitting data for publication. The handbook walks them through that entire process. Like all documentation, don’t assume they’ll read it or refer to it. Reinforce the handbook with group sessions and individual hands-on training if needed. It can be labor-intensive but having trained, knowledgeable staff closer to the departments doing this work is invaluable.

    We continue to learn and refine our processes and approach to open data at the City of Boulder. If you want further insights or just need someone to discuss open data with, contact me!