Big Data in Government

[Originally posted on NextGov.com: Follow Philly's Lead and Dive into the Big Data Future]

Big data is valuable data in government
Chief Data Officer Mark Headd, City of Philadelphia

Big data” has become one of the new buzzwords and it is quickly making its way into conversations in government. However, it is difficult for government officials to identify what the big data discussion means for their own organizations, what the challenges are, how they can create additional capacity taking on a job that does not necessarily connect to the core mission of their agency and how they have to tackle the issue to respond to requests from the public.

The big data discussion hits government from two different sides: First, big data is created by citizens in their daily online interactions using social media either directly with government or talking among themselves about issues related to government. As the recently released first guidance for social media metrics for federal agencies shows, government is just now getting into the groove of measuring, interpreting and acting on insights they can potentially gain from their interactions with citizens. The other trend has started a few years ago with the newly initiated conversations around open government and the launch of the federal data sharing site data.gov, a public website that hosts hundreds of data sets produced by federal agencies.

Originally, the big data discussion started outside of government, but has direct implications for government as more and more agencies, politicians and citizens are using social media to interact with each other. Social networking platforms, such as Facebook or Twitter, allow citizens to directly connect to government agencies, share their immediate sentiments via comments in their own news feed  In doing so, they create hundreds of new data points, that increase the data volume far beyond a single phone call with a citizen request. As a matter of fact, the conversations can go back and forth between government and citizens, but also among citizens. Social media conversations might not even directly involve government, but they are related to ongoing hot-button issues, upcoming policy changes or the cut of a government program.

Keeping track of potentially thousands of externally created data points published by citizens on a daily basis has become an unmanageable problem that is slowly being addressed in the public sector. As a response, some agencies have shut down the possibility to leave comments on their Facebook pages reducing the cost to respond and track, others actively pull in citizen input or moved on to other ideation platforms that focus the conversation on a specific problem statement and streamline the solicitation of targeted responses and input from the public (see for example Challenge.gov).

The second trend that government agencies are facing is the mandate of the Open Government Initiative to release government data sets in machine-readable format for public consumption.  The flagship initiative data.gov has paved the way for state and local governments to respond in a similar fashion. Most recently, NY state has released its own data portal, a website that hosts about 6,500 data sets from state, city and local government sources in NY state.

The challenge for public managers is manifold: they have to identify appropriate data sets, clean them, potentially merge them from different databases, and make sure that they do not contain any individual or personal information that cannot be released to the public by law. Independent of each agency’s individual response, given the multitude of citizen interactions and ongoing conversations in combination with the top-down mandates, additional resources, increased capacity and new positions with a specific skill sets are necessary to appropriately respond. Beyond the internal organizational challenges to manage information streams, big data is much more: Government agencies also need to understand how they can open themselves up for third parties who are reusing the data.

Mark Headd, the newly appointed first Chief Data Officer of the City of Philadelphia, recently spoke to my social media class at the Maxwell School and shared his first-hand insights into the world of Big Data in government.  Mr. Headd was appointed through an executive order of Mayor Nutter in Philadelphia and organizationally embedded within the ICT unit and directly reports to the CIO and mayor. Mayor Nutter made it a political priority to understand and organically implement elements of the open government movement – an advantage that other cities might not have, where Chief Information or Chief Data Officers still need to battle political fights before they can implement change.

Mark Headd

He describes himself as a data evangelist and an embedded technologist who has the task to discover government data, think about ways to make it available to the public and find a match between the data and external stakeholders who can potentially use the data to create public value. Internally, he is focused on cultural change more than on data analysis issues or technological problems: He aims to convince public managers to see the potential value the data can have for the public, start discussions about the reasons to release data and the way government officials view themselves, but most importantly inform them about changing expectations and citizen needs. Mr. Headd then facilitates connections between data sources and potential data users outside of government.

As one of the first Code for America cities in the U.S., Philadelphia’s local tech community of civic hackers has an immense motivation to reuse public information and create valuable applications. As opposed to data.gov, where data sets are mostly available for so-called “elite access” – a small group of highly trained computer specialists and data analysts – the approach in Philadelphia focuses on data that is not highly specialized, already publicly available, such as transit data, day care centers, information about flu shot locations, etc. Most people will consume the existing data through web browsers, either on their desktops or mobile phones. Mark Headd describes Philadelphia’s approach to open data as a focus on the “last mile”. By that he means, that the city invites civic hackers who recombine the existing disconnected data sets in a mindful way to go beyond mere display of data sets, as it is done on data.gov. The city wants to increase value to go beyond merely pushing out data as the main objective, instead they collaboratively want to build new mobile phone applications by recombining data.

Events such as “Code for Philly” in collaboration with Code for America combines members of government collaborating with the local technology community to use data and build new projects that have the potential to create a civic good. Again, Philadelphia comes with a unique advantage: The existing culture, that is similar to citizen such as Boston, Baltimore, is geographically close to NYC has a very active civic technology community with programmers who are passionate about the city, feel a sense of belonging and community, which other cities such as San Francisco which doesn’t have home-grown technologist.  Mr. Headd’s goal is therefore to capitalize on the people’s love of their city.

One example, for Mr. Headd’s success are applications such as CityGoRound.org, which is a clearing house for applications around transit data. Local transit applications are built to help citizens catch their train. In addition, the application and code are also made available for reuse in other cities, by simply plugging in local transit data. Transit authorities agreed to a standard that makes sharing of already existing applications easy – work products don’t need to be reinvented or recreated around the country. As a result, the city and its technology stakeholders are collaboratively building an entire eco-system around government data use. All cities can use the same infrastructure and format to use the data.

One of the challenges Mr. Headd sees is convincing citizens to reuse the data and make use of the applications. One approach Philadelphia has chosen is to advertise the newly created third-party products on public buses (see for example ‘Where I my SEPTA?’). However, the question of endorsing and publicly sponsoring products that were built outside of government is still an unresolved issue.

Another challenge is the cultural change necessary to change existing bureaucratic governance procedures. For Mark Headd the solution is a conversation about effectiveness and efficiency of the current use of government data. He shows public managers he interacts with how they can reduce inefficiencies in responding to a steady stream of Freedom of Information Act-requests (FOIA) to release data to individual citizens or journalists. Every request takes time, is oftentimes burdensome to the unit and labor-intensive to research and respond to. Mr. Headd works together with public managers to look at the top-5 data requests, collaboratively tries to find ways to release the data and at the same time unload the administrative burden off the unit to respond to each request. Responders can simply point requestors to the publicly available data set and save time, resources, and money to research, vet, and formulate responses.

As an example, the Department of Licenses and Inspections receives multiple requests to release data about the number of locations of vacant houses as well as code violations. By releasing the data on a public website, the city allowed developers to create mobile applications and in turn significantly reduces the number of written requests and phone calls. The research activities for similar types of requests are minimized by simply pointing requestor to the new app. Government staff can turn their attention to the core mission, instead of being derailed by FOIA requests. A direct benefit to the release of government data.

Similarly efficient is the reuse of the data on the citizen side: during hackathon events civic hackers are building a service on top of government data sets and are therefore helping themselves, instead of having to reach out to government. A new form of co-delivering public services build on big data.

Mr. Headd shared a few insights on how other Chief Data Officers can tackle the issues in their own cities. He says “Nobody wants to be first, so point people to other success stories in other agencies.” He is constantly evangelizing about the value of big data, but is also informing local and city government and making his colleagues aware of what is going on around the U.S. (and the world), which helps them understand the benefits of releasing data. He suggests to show public managers tangible benefits, instead of talking about less tangible benefits such as openness or accountability which are very difficult to quantify, especially in budget-driven conversations.

Mark Headd sees limitless applications for the release of government data and the analysis and reuse of big data: Budgets, spending, crime or transit data enable people to see how well city employees are doing their jobs and help them make aware of the improvements or new focus area. The big data discussion can help here to talk about high performing government and all the things that work very well in local government. Most of the coverage government receive is unfortunately focused on things that are going wrong – big data can change the focus.

Lastly, social media and government data can then come together to create more personalized connections to citizens by communicating success stories. Citizen engagement will stay the major challenge: Similar to voting, Philadelphia has identified about 40 other processes in which citizen feedback is needed, engagement is low, and new experiments to increase feedback are needed. An application was recently launched to pull citizen opinions into the policy-making processes: Textizen.com allows citizens to send in their feedback by cellphone – without needing an expensive smart phone to actively participate in the policy-making process. By institutionalizing easy to use tools to which every citizen independent of their age group, income or technological literacy has access to, tools like Textizen can become part of a government’s future planning process and can automatically generate input without hosting town hall meetings at which limited numbers of people are participating.

The example of Philadelphia’s success is certainly an outlier: The city is blessed with a unique combination of advantages that other local governments might not have:

  • a political mandate that supports and mandates reuse of public information,
  • a technologist who understands managerial as well as technological and cultural issues in government, and
  • a unique tech community who loves its city and pushes the envelope to innovate.

However, all cities around the U.S. are invited to simply reuse existing applications without reinventing the wheel on a daily basis. Get going with resources that are already freely available and dive into the future of big data in government.

New article: Social media adoption and resulting tactics in the U.S. federal government (GIQ)

Cover by inesmergel
Cover, a photo by inesmergel on Flickr.

Government Information Quarterly just published one of my articles on the adoption decisions of federal departments to use social media and how the decision making processes lead to online tactics.

Abstract:
In 2009, the departments in the executive branch of the U.S. federal government received the presidential marching order to “harness new technologies” in order to become more transparent, collaborative and participatory. Given this mandate, this article sets out to provide insights from qualitative interviews with social media directors to understand the factors that influence internal adoption decisions to use social media applications, such as Facebook, Twitter, or blogs. Three distinct factors influence the adoption decisions of social media directors: information about best practices in their informal network of peers, passive observations of perceived best practices in the public and private sector, and “market-driven” citizen behavior. The resulting adoption tactics include: (1) representation, (2) engagement, and (3) networking. The findings point to the need for higher degrees of formalized knowledge sharing when it comes to disruptive technology innovations such as social media use in highly bureaucratic communication environments. Recommendations based on the lessons learned are provided for practitioners and social media researchers to develop social media tactics for different organizational purposes in government.

Full reference: Mergel, I., Social media adoption and resulting tactics in the U.S. federal government, Government Information Quarterly (2013), http://dx.doi.org/10.1016/j.giq.2012.12.004

Please email me if your library does not have a subscription for the article.

Social media and the 2012 election: Class syllabus online

My new class “Social media and the 2012 election” starts next week and I wanted to post my syllabus for public comment.

This is a class that I taught for the first time in 2008 during the Obama campaign. After the election the president was praised for his Internet strategy that complemented his traditional campaigning and many scholars have pointed out that he was able to motivate non-voters via social media to go to the polling booths.

Between 2009 and 2012 I spent a lot of time trying to understand how the lessons learned during the successful presidential campaign can be used for day-to-day governing activities. While the Open Government mandate pushed a lot of efforts in the U.S. federal agencies forward to invest time and resources into harnessing new technologies, government agencies are also facing many challenges when using social media. For that purpose, I observed and interview social media directors in the U.S. federal government to understand their strategic, managerial, and administrative decision making and the resulting social media tactics.

This class is therefore based on my research on social media in the public sector. It observes in real-time who the public, news organizations and the candidates are using social media until election day. It grounds the observations in theoretical sociological and information management concepts. The goal is to teach the underlying concepts and managerial skills future social media managers need – not only in government, but also in the nonprofit and corporate world. Guest speakers will complement lectures and class discussions.

Here is the syllabus. I would love to hear your comments and suggestions for improvements!

A tale of the social media one-way street: City of Manor reverts back to the dark ages…

Remember the unbelievable Government 2.0 and social media success story of the City of Manor – a small town in Texas? Dustin Haisler, former city manager & CIO of Manor, was the driving force behind the use of WordPress to redesign the city’s homepage. He developed beta cities – a promise to revert other cities’ websites to the brand-new world of social media within 24 hours using the same template. He also started to use QR codes around the city to keep citizens up-to-date on the progress of construction sites or inform them about the history of a building or park.

 

 

With his enthusiasm, he caught the attention of the White House and many innovators travelled to Manor to participate in a conference celebrating Manor’s research and open innovation initiative “Manor Labs”. It is one of the first open innovation platforms in the country, using a virtual currency called “InnoBucks” that incentivized citizens to participate in ideation exercises to improve the effectiveness of the city. Another part of the initiative took the city’s “alpha” version (launched in 1913) to what Dustin Haisler called “open beta“: A version of the city’s website on which all citizens can openly engage with the government and each other.

Dustin’s success story was prominently featured in an article on Fastcompany.com titled “How an army of techies is taking on city hall“. He is a frequent guest speaker in my social media classes here at the Maxwell School and I consider him a true innovator and leader in his field. I recently checked back to take a screenshot of the City of Manor’s website for a presentation and was surprised to see that they apparently went back in time and put up a horrific website in a design that reminds me of the early days of the Internet.

 

Here it goes: The “new” design of the once so innovative City of Manor, a leader in open government, open innovation, and social media goodness in the U.S. What happened, Manor?

White House releases new #DigitalGov Strategy

The White House released a new digital government strategy “Digital Government: Building a 21st century platform to serve the American people“, followed by a Presidential Directive. In the directive the President points out: “

The main points of the roadmap include:

- data.gov is a starting point for new forms of data innovation
- tool/device agnostic (Bring Your Own Device)
- digital government is not about mobile apps/smartphones, instead it’s about connecting people to the data and have them help each other -> important for mobile-not-haves
- private sector and citizen innovators as central parts of the strategy

Department of Veterans Affairs publishes updated social media policy

 

The Department of Veterans Administration has recently published an updated social media policy (VA Directive 6515 – Use of Web-Based Collaboration Technologies).

Prior to this release the Government Accountability Office has published a report for Congress titled “Federal Agencies Need Policies and Procedures for Managing and Protecting Information They Access and Disseminate” – highlighting that federal agencies need to think about ways to ensure secure use of social media tools by their employees.

The updated VA social media policy focuses not only on secure and safe use of social media tools, but also on the conduct of employees engaging in real-time exchanges with the public:

“This isn’t about using social media because it’s cool or because it’s a fad,” said VA Director of Online Communications Brandon Friedman.  “It’s about getting the right information to the right Veteran at the right time.  This policy sets us on a path toward changing how we talk—and listen—to Vets.” 

Media coverage:

1. Washington Post: VA revamps its social media policy (08/16/2011)
2. Washington Post Blog: VA establishes social media policy (08/16/2011)
3. Gov Info Security: VA aggressively adopts social media (08/17/2011)

4. Stars & Stripes blog: VA seeks new impact with latest social media push (08/17/2011)

Enhanced by Zemanta

Data.gov in the classroom: Government 2.0 syllabus

Data.gov in the classroom features resources for K-12, Universities, and Education in the World. Among them is Karim Lakhani’s Data.gov case study developed at Harvard Business School, Beth Noveck’s Democracy Design Workshop Do Tank, and now also my Government 2.0 syllabus.

I have been teaching this class for the last three years and the online syllabus shows a combination of resources I use for a semester-long course. As one of the motivations why my MPA students might find it valuable to participate, I use President Obama’s Open Government and Transparency memo, that asks the executive departments and agencies to be more participatory, transparent and collaborative. Especially in the class on Transparency, I refer to data.gov and the students have to think about ways to motive (local) government to provide datasets, make those datasets machine readable and how citizens can use the data provided.

OpenGovRD: Towards an “Open Public Administration Commons”

Day 2 of the OpenGovRD workshop started with a session to collect open-ended research questions that the academics in the room can tackle. We reviewed our initial wish-list of what research on Open Government should look like. Some of the keywords people in the room used to describe OG research included: interdisciplinary, rigorous, robust, actionable, fundable and most of all FUN (that was my favorite keyword). I believe fun will be a result of a research agenda that will include researchers from different disciplines, but also includes a constant feedback cycle between academics and practitioners.

I would like to push even further – not just showcasing research findings, but constantly including practitioners into the research process and not only as subjects (i.e., interview partners), but as equal partners who guide the research, evaluate its feasibility and to keep the research grounded and unbiased. The findings need to be actionable right away and not after a 2-3 year publishing cycle in academic journals that are “hiding” the results for exclusive access in University libraries.

Obviously, an OG research agenda needs to be fundable. The group highlighted that there is no digital government program at NSF anymore, so that new funding sources need to be discovered and we probably need to work closely with directorates or programs at NSF to identify the right venues for proposal submissions.

The practitioners and academics in the room mentioned one gap over and over again: We don’t know what we know! There are several platforms out there that are collecting, harvesting and displaying some of the research and reports that are available on specific subtopics, but there is not one place that helps to compile everything we already know. I suggested to create an “Open Public Administration Commons“. This place can serve as a networking platform that provides the opportunity to connect to ongoing research projects, give direct feedback not only to the final results, but on an ongoing basis while the discovery is happening, to test ideas in early stages, but foremost to provide a channel that helps to push findings directly to government so that public managers can act on the findings and find a guide on how to tackle current and urgent problems. Many agencies face similar or at least comparable problems and while it is helpful to understand that there are best practices cases out there, it is much more important to actually make the social connections between government officials to share insights on the day-to-day “How To” questions that are coming up while people are trying to solve problems. I would like to take the platform idea a step further and make this platform a place for those of us who are teaching OG-related topics to find up-to-date case studies that can be used in classrooms. We can educate cohorts of MPA or IS students that already know about the newest developments when they enter their first jobs.

Together with my co-authors, I have written about the idea of an “Open Public Administration Commons” in a recently published article in the Journal of Public Administration Research and Theory: “Towards Open Public Administration Scholarship” (email me if you want to read a copy of this article).

Related reading:

Schweik, C., Mergel, I., Sanford, J., Zhao, J. (2011): Toward Open Public Administration Scholarship, in: Journal of Public Administration Research and Theory (J-PART), Minnowbrook III: A Special Issue, Special Issue Editors: Beth Gazley and David M. Van Slyke, Vol. 21, Supplement 1, January 2011, pp.i175-198.

Open Government Research and Development Agenda Setting (#OpenGovRD) workshop

I am currently attending the Open Government Research and Development Agenda Setting workshop at the Center for Technology in Government at the University of Albany (you can follow our tweets with the hashtag #OpenGovRD). The workshop is organized by CTG, TWC, New York Law School – IILP, and Civic Commons.

We started out by collecting unanswered research questions, then clustered the questions into broader categories and went off to work in groups on defining the questions in more detail. The major themes that need to be tackled include:

1. What are citizen needs? How do they gain access? How do citizens know what they need to know? How can government provide enough digital literacy education to help citizens understand what they need to know and how to access what they need to know?
2. Incentivizing government (employees) to participate in Open Government; incentivizing the public to use the data and give feedback about the value and quality of the data they receive
3. Government (current and future) capabilities and capacity
4. Creating a business case for Open Government: Is #OpenGov worthwhile? What is the actual value? And what will we lose if Open Government dies?
5. Efficiency/effectiveness of tools, interoperability,
6. What does the ecosystem of Open Government looks like? How can government collaborate across all sectors?
7. How do we build the Open Government tool kit?
8. How can we make Open Government sustainable?
9. How can we create valuable and high quality data that citizens want to see and (re)use?

I brought up that we have very little understanding of how the ecosystem of Open Government functions, who the main players are and how government can interact with them in meaningful ways.

I also believe that Government is not the sole provider/user/standard of citizen-relevant information. I observe many more data providers and sources of data creation that need to be taken into account in an OG research agenda to understand how the overall ecosystem of Open Government works.

Take for example SeeClickFix: Citizens are creating data on a daily basis that is highly relevant for their local context, but has very low relevance for the federal Open Government Initiative. Ben Berkowitz – founder of SCF – has framed the term distributed democracy for this form of data creation. I believe it is a form of distributing indirect responsibility for local issues and problems to citizens. In this process government takes on a responder role instead of proactively sending a city manager out to observe the problems as a government task. Data is created in a decentralized manner and pushed toward government.

My group had the task to think about necessary, current and future government capabilities and in our brainstorming session I suggested to include the SCF example as a way to think about bidirectional and decentralized data/information/knowledge creation that is not necessarily always initiated by government. Our research statement was therefore:

How can the Open Government Initiative drive innovation and improvements in government capabilities to collect, manage, use, integrate (combine), and share information?

We suggest to study this question
- internal to government,
- across all levels of government,
- and in collaboration with the private sector and civil society

Many of these question don’t sound particularly new or innovative – or as Alex Howard from O’Reilly Radar pointed out are not even unique to Open Government, instead are a general theme in Government 2.0. I agree wholeheartedly, but just because the questions are not new, does not mean that they were answered. I believe that research – especially in public administration and public management, as well as at the intersection of public administration and the use of (new) technologies in the public sector has to catch up with the reality of government. We need more research on the social processes both inside government, but also in government’s interactions with all their stakeholders. We even need to go beyond the pure government focus and need to understand how citizens are creating data (see for example CrisisCommons or GovLoop as knowledge incubation location outside the governmental boundaries and context) and are “socializing” the data without government intervention.

Here is the workshop summary from the event listing on the CTG website:

With the one-year anniversary of the Open Government Directive behind us, the field of Open Government is at an important crossroads. While much work has been done by government agencies in trying to make their data, operations, and services more open to the public, the actual impact of these efforts and their value to the public and government alike is mostly unknown.

The Open Government Research and Development Workshop will focus on strategies for and the impact of opening up, federating, and creating value from government data. This workshop, organized by the Center for Technology in Government at the University at Albany, the Tetherless World Constellation at the Rensselaer Polytechnic Institute, the Democracy Design Workshop/Do Tank at New York Law School, and Civic Commons will define a research roadmap that looks at the legal, policy, and technical questions that must be addressed in using government data to improve the lives of everyday citizens.

The workshop will build on the work started at the Open Government R&D Summit convened by the Networking and Information Technology Research and Development Program (NITRD) and the National Archives and Records Administration (NARA) on March 21st- 22nd in Washington, DC (http://www.nitrd.gov/opengov/).

The workshop will take an interactive and interdisciplinary approach to creating an actionable and relevant multi-year open government research and development program focused on identifying critical needs, mapping needs to potential solutions, identifying legal and policy barriers, exploring critical evaluative approaches, and laying out strategies for attaining future research funding.