Digital Engagement

Government launches one-stop shop for data

Today we are making the beta of data.gov.uk available for general access. It contains more than 2500 sets of data from across government.  All of the data is non-personal and in a format that can be reused by any individual or business to create innovative new software tools, such as applications about house prices, local amenities and services, or access to local hospitals. This delivers on the commitment in “Putting the Frontline First: Smarter Government” to go live with data.gov.uk, as well as to integrate data from the Publications Hub for National Statistics and release more health data.

It's now just over 7 months since the Prime Minister appointed Sir Tim Berners-Lee and Professor Nigel Shadbolt to help drive the Government's programme to free up its public data for re-use. And indeed just over 7 months since I took up my role as Director for Digital Engagement. We have involved over 2,400 people in our developer community to learn what data they want and how they want to access it; and we have brought together into one place an initial collection of over 2500 datasets from across government which can be re-used freely and easily.

“Putting the Frontline First: Smarter Government”, published in December, set out the overall strategy on Public Data. It published key principles for the release and reuse of Government's non-personal data; a commitment to release certain key, and much requested, datasets on weather, transport and health over the next few months, and consultation on freeing up some of the products of the Ordnance Survey; providing data under a licence interoperable with the internationally-recognised Creative Commons model; and an initiative to work with local government to extend Public Data principles there, through a Local Public Data Panel which Professor Nigel Shadbolt chairs. We are also moving towards greater use of Linked Data standards.

Freeing up public data is a journey, and today marks another step along the path: making the "beta" of data.gov.uk available to everyone, in the wider community of interest as well as developers who have already worked with it.  Thanks to the advice and feedback that our previewers have given us, there are a number of new features on the site which we have been working on and improving over the last few months. These include:

Datasets – we have both increased the number of datasets available on the site and made the information about each dataset more extensive.

Browsing – you can now browse datasets by subject tags

Wiki - The site has now integrated a wiki which enables the sharing of community knowledge. Every dataset now links to a corresponding wiki page: this will encourage information about using the data with sample queries and example source code to be shared.

Forum – The site now has a forum which allows registered users to discuss aspects of the project in more depth.

Over the next few months we will build on this beta with more functionality. We are already working with departments, agencies and local authorities to release more data month by month and we will increase the use of Linked Data standards.

The benefits of blogging

It’s been about 15 months now since we got into blogging here at the Department for International Development (DFID). I say “we” - actually, it’s our staff around the world who write the blogs while we support them through style tips and technical help.

So what spurred us to join the blogosphere in the first place? The big idea for us was to give our web visitors another channel, beyond the main website, where they can get the real-life story of our work from those who are actually out there doing it and to allow visitors to ask questions directly and continue the debate. We wanted to move away from a purely one-way online presence, to a two-way interaction with our audience.

As with much of the social media we’ve ventured into (including YouTube, Twitter, Facebook and Flickr) it’s been a learning curve as we’ve gone along. That’s not to say we hadn’t scoped it out beforehand (far from it), but we have realised that social media is very much about the doing (rather than just planning).

Once we’d figured out the mechanics of the blog and had a platform we were happy with (we use WordPress, tailored to our needs thanks to the skills of Puffbox) we soon realised the biggest challenge was the ongoing management of bloggers and the actual content.

In terms of the latter, our first issue became apparent as soon as we mentioned the project to our colleagues in the Press Team – and gave a few of them a mild heart murmur (“…so let me get this straight, you want to let our staff write what they want about their work, then make it public?!”).

I’m hamming it up here a bit of course, but the point is you may face some valid internal concerns about giving staff free reign to publish. But it shouldn’t be an insurmountable obstacle. After consulting with our Press colleagues further and agreeing on some sensible precautions we’ve come up with a solution that everyone’s happy with.

The other big ‘lesson learnt’ was management of the bloggers and the supply of content. We help to support them to develop their own style and to get the best out of their blogs with practical tips such as linking, using photos, incorporating a story, etc.

With this support, several bloggers have taken to it easily and require little management. Some may also be bloggers in their spare time or have a keen personal interest so run away with it. However, we like to ensure we have a range of bloggers, and that also means a range of abilities and time to upload the blog themselves.

For us, we think getting the range of content is crucial (for example, from the heads of our offices down to the programme workers), so we’ve tried to adapt our approach to support bloggers on their terms. What this means is that if a blogger is happy to do it all themselves, they’re free to do so. If they’re tight for time and can only bash out the text of a blog in between other commitments, they can send it to us and we do the rest: format it, add photos and upload it to our system.

So basically, once the blog is up and running, we’ve found it’s still a hands-on process. And it will vary from blogger to blogger. I know it’s been a different experience for other departments and agencies, and it makes sense that organisations adapt their strategy according to their staff, purpose and audience.

And of course we are still refining the process, so it would be great to hear what approaches others have taken and the lessons they’ve learnt as we try to improve. Any thoughts anyone?

While we’re looking to do better, we’re happy to be reaping the benefits of our blog already. Since we began we’ve had more than 50,000 visits to the site and our number of daily visits has steadily climbed, roughly doubling since October 2008.

It’s not just numbers. There is a real engagement with our audience now, with several comments being posted, many beyond the UK (who’d have guessed Nepal would be such a hotbed of discussion?). For one reader, it even gave him “an emotional connection to a government department – something I never thought I would say!”

Our bloggers are also making new connections – for example, Martin Leach’s blog on the Rwandan genocide prompted a researcher from the London School of Economics (LSE) to get in touch, while Emily Poskett’s post on the Comic Relief celebrity climb of Mt Kilimanjaro broadened our exposure to thousands of visitors beyond our typical audience.

But see for yourself and let us know what you think we could do to improve. And, if you’re a fan, let us know which you think was the best DFID blog of last year in our round-up on Facebook.

 Julia Chandler and Simon Davis are members of the Department for International Development's Online Content team

Met Office releases first Weather Widget

The Met Office generates vast quantities of data on the way to delivering forecasts and climate predictions. Much of this data (more than 60Gb per day) is distributed already in some way — either to other meteorological and climate organisations, or to people for whom the weather can have an impact. The Met Office website is also very popular, especially at times of significant weather events, such as the snow of February 2009 when there were nearly 20 million visits to the site. This year the Met Office has significantly expanded capabilities to distribute data with advisories, warnings and alerts available as RSS feeds, email, Twitter (and, via Twitter, SMS), and a range of platform specific tools such as those for Google, Firefox and Windows Vista/7. Forecast information will also be using these channels early next year. See here for more information.

The interpretation of weather data can be a highly skilled task, particularly if that data is computer generated model outputs or observation data such as satellite or radar information. Research suggests many requests for data are not to re-use or add value to the data in any way, but simply to visualise it. Many websites would like to offer their users local or contextual weather and, until now, the only choice available to them has been to develop their own web-based weather visualisation applications based on a subscribed weather data feed. The Met Office will continue to supply weather data and indeed plans to widen the catalogue of data available to the general public over the coming months, but this is often not what website owners want.

To meet this need, the Met Office has released the first of a new series of “Weather Widgets”. These tools are primarily aimed at website owners and operators who may only have a limited web development capability. A simple to use web wizard here, walks users through the configuration of a weather visualisation tool. It is possible to select the location of interest, number of forecast days and more besides. It is even possible to include simple radar imagery, animated precipitation forecasts, isobaric charts and temperature maps into the widget. When the configuration is completed, a small piece of script is generated that can be embedded into the host website, allowing the user to present a professional and polished weather tool at no cost to them. As the Met Office serves the content on behalf of the host site, there are also no ongoing running costs to the host website.

Much of the technology to support the gadgets was developed in the Met Office public innovation environment, called “Invent”, where new ways to visualise and distribute weather content are always being developed and tested — so visit often to catch the latest in weather technology.

Charlie Ewen - Head of Web Business and Service at the Met Office.

Local Government Data

The Prime Minister said that "there are many hundreds more datasets that can be opened up - not only from central government but also from local councils, the NHS, police and education authorities."; and the Secretary of State for Communities said "we plan to give local people far better access to information held by local public organisations so they can challenge, compare or scrutinise their local services in order to drive up standards in their area."

The Government will encourage local government to release local public data and make it free for reuse, and establish an open-platform local data exchange. Professor Nigel Shadbolt from the University of Southampton has been asked to head up a panel of experts to oversee the release of local public data and ensure that data are linked effectively across local authorities, the Local Government Association, government departments and agencies.

The Local Public Data Panel members will include:

• Tim Allen, Programme Director for Analysis and Research, Local Government Association
• Roger Hampson, Chief Executive of Redbridge
• Dave Smith, Chief Executive of Sunderland City Council
• Janet Hughes, Head of Scrutiny and Investigations at the Greater London Authority
• Jos Creese, Head of IT at Hampshire County Council
• Nick Aldridge, CEO of Mission Fish UK (eBay for Charity)
• William Perrin - Government web innovator and community activist
• Chris Taggart - web developer and founder of OpenlyLocal.com

The Panel will work closely with local authorities, strategic partners, government departments and agencies, developers and community organisations to help improve local public services and empower citizens. The Panel will operate for a two year period to the end of 2011. It is expected to hold its first meeting in January 2010. Key aims are to:

• Ensure understanding of the case for making local public data freely available for re-use
• Promote innovative uses of local public data
• Sponsor the further development of a single place on line (‘data.gov.uk’)for all public sector data, while meeting the specific needs of the local government sector
• Encourage agreed standards for greater data and information sharing by local strategic partnerships.

Putting the Frontline First: Smarter Government

This morning the Prime Minister launched Putting the Frontline First: Smarter Government. You can read his speech on the No. 10 website

The document sets out plans for strengthening the role of citizens and civic society; recasting the relationships between the centre and the frontline and between the citizen and the State; and streamlining government. It is the culmination of work carried out across the public sector over the past year, including the Power of Information Taskforce Report, and thanks are given for the vision and advice received from industry leaders and distinguished public sector thinkers - including Sir Tim Berners-Lee and Professor Nigel Shadbolt on radically opening up publicly held data to promote transparency.

A major theme of "Putting the Frontline First: Smarter Government" is radically opening up data and public information to promote transparent and effective government and social innovation. Significant announcements on the Public Data theme include:

Establishing Public Data Principles

• Public data will be published in reusable, machine-readable form
• Public data will be available and easy to find through a single easy to use online access point (http://www.data.gov.uk/)
• Public data will be published using open standards and following the recommendations of the World Wide Web Consortium
• Any 'raw' dataset will be re-presented in linked data form
• More public data will be released under an open licence which enables free reuse, including commercial reuse
• Data underlying the Government's own websites will be published in reusable form for others to use
• Personal, classified, commercially sensitive and third-party data will continue to be protected

Opening up specific datasets and promoting transparency

The Government will release valuable public datasets and making them free for re-use, including:

• Releasing health data such as the NHS Choices data
• Consulting on making Ordnance Survey mapping and postcode datasets available for free reuse from April 2010
• Increasing access to and reuse of public transport data including the National Public Transport Access Node database, with information available to the development community by April 2010
• Opening Met Office Public Weather Service data to include: releasing significant underlying data for weather forecasts for free download and reuse by April 2010, and working to further expand the release of weather data, while recognising all public safety considerations; and making available more information on Met Office scientists, their work and scientific papers, free of charge
• Publishing, by spring 2010, details of how the fiscal stimulus announced in the Pre-Budget Report 2008 has been spent, disaggregated to local level
• Launching a public consultation early in 2010 to seek views on how we could publish further financial data so that it is user-friendly and accessible, with a view to putting a live system in place by summer 2010
• Integrating ONS data with http://www.data.gov.uk/ from January 2010.

Encompassing local government and the wider public sector

• The Government will encourage local government to release local public data and make it free for reuse, and establish an open-platform local data exchange. Professor Nigel Shadbolt will lead a local public data panel to ensure that data are linked effectively across local authorities, the Local Government Association, government departments and agencies.
• The Prime Minister said that "there are many hundreds more datasets that can be opened up - not only from central government but also from local councils, the NHS, police and education authorities."
• More details about the Local Public Data Panel.

Actively publishing comparative data

The Government will make it easier to compare performance across frontline services, by publishing data on public service performance, citizen outcomes and value for money in achieving those outcomes. This data will be published in reusable form on www.data.gov.uk by 2011. Specifically this means that:

• In local government we will work with the sector to develop comparable measures of value for money across a range of local government services. We will consult on these from spring 2010 and publish them in 2011.
• In the health sector, we will publish hospital-trust-level reference costs for specific treatment categories (or healthcare resource groups) online in early 2010.
• In education, we will publish as much data as possible from the National Pupil Database and other sources that is relevant and compatible with preserving individual anonymity from April 2010 onwards.
• In the criminal justice system, we will benchmark offender management in prisons and probation during 2010, and benchmark the whole of the prison and probation system, by the end of 2011.
• For police force performance, we will publish quarterly 12-month moving average crime data at police authority level, by 2010, and develop value for money data which will enable comparison of forces’ costs and productivity.

"Raw data now"

Some of the data promised today will not be available for a few months. However to go alongside the launch we've been working with the data teams around Whitehall to put another 146 raw data sets live on our developer preview data site right now.

Finally as the Prime Minister said this morning:

“Releasing data can and must unleash the innovation and entrepreneurship at which Britain excels - one of the most powerful forces of change we can harness.”

Thinking Big

The Bebo social networking website recently hosted a major competition, called the Big Think, to find new, practical ideas from young people on how to tackle the specific issues of climate change, crime and careers. The best three ideas were presented by their authors to the Prime Minister and Cabinet last week.

Big Think authors visit Number 10.

With approximately 8 million unique users, most of whom are in their teens or early twenties, Bebo provided a direct link to the target audience – and was the perfect medium to listen to the aspirations and concerns of young Britain in a credible, youth-led, non PR-driven context. 

This search for practical solutions to some of the major challenges facing Britain followed on from the Big Think’s two week online debate on young people’s priorities held earlier in the summer. Both phases are part of the wider consultation held around the Building Britain’s Future programme. This first phase culminated with 10 young ambassadors and 5 Bebo competition winners having the chance to meet the Prime Minister to discuss their ideas.

For the second phase of the Big Think the Cabinet Office, in partnership with No10, Bebo and Livity (a non-profit social enterprise agency) sought to stimulate thought, interest and discussions by posting various polls to the Big Think main profile page on Bebo. In total we received over 12,000 votes across all polls in reply to questions on how to improve schools, concepts of identity and whether Britain ought to have a death penalty.

The polls, in turn, drove young people to want to contribute more on a serious debatable topic. Rather than giving a simple yes or no answer they commented to explain their opinions and provide new ideas. In total the Bebo profile page had 6,798 comments while the Big Think teaser video that was added to the Bebo homepage was viewed 1,592,643 times.

We received many ideas of which the best six were chosen to go through to the next stage. Their authors were then invited to travel to Westminster to meet with expert mentors Henry Bonsu, co-founder of Colourful Radio and Andy Hamflett, Chief Executive of the UK Youth Parliament, who coached them on the best way to present their ideas to a panel.

After their coaching session it was time to pitch their ideas to a high profile panel led by the Minister for the Cabinet Office, Tessa Jowell, and including drum ‘n’ bass star Goldie and record-breaking young mountaineer Jake Meyer. The panel then decided on the top three ideas with their authors then given the task of presenting them to the PM and Cabinet!

The Big Think represents an excellent example of the real capability of social media for Government not only to gather feedback and participate in discussions with previously difficult to reach stakeholder groups but also to engage with the replies it receives. The Big Think demonstrates the Government’s commitment to begin the journey to move its public interaction from broadcast to conversation and collaboration.

Stephen Timms reports progress on Making Public Data Public

Yesterday the Minister for Digital Britain, Stephen Timms, gave the keynote speech for the RSA/Intellect “Technology in a Cold Climate” symposium in which he outlined progress to date on the HMG data site.

“Information is the essential “raw material” of a new digital society, opening up solutions to these kind of challenges.  And Government must play its part by setting a framework for new approaches to using data – and, as they say, “mashing” data from different sources to provide new services which enhance our lives.  In particular, we want Government information to be accessible and useful for the widest possible spectrum of people. 

That is why the Prime Minister asked Sir Tim Berners-Lee to advise on how Government can best use the internet to make non-personal public data as widely available as possible.  We are supporting Sir Tim in a major new project, aiming for a single online point of contact for government data, and to extend access to data from the wider public sector.  We want this project for “Making Public Data Public” to put UK businesses and other organisations at the forefront of the new semantic web, and to be a platform for developing new technologies and new services.”

So far our request for developers to “get excited and make things” has exceeded our initial expectations. Not only is the number of people signing up to the developer forum higher (currently more than 1,300), but also the discussion board is very active with a healthy list of ideas for the site and, perhaps most excitingly, a few applications are beginning to see the light of day.

Working in partnership with Guardian Professional, we held 3 developer days hosted at The Guardian's Kings Place offices in central London on the 14th-16th September. As an organisation they were best placed to help us undertake this task, having built a community of talented developers and opened up their API. You can have a look here at the excellent postcode paper concept and the rather wonderful traffic data visualisations here, which were just two of the many ideas for applications that emerged over the course of the camp. Ideas about their priorities for further data releases (to add to the 1,100 datasets currently on the site) were shared and important foundations for further iterations of the HMG Data site were laid.

International links

The Digital Engagement team is looking to improve our links with international colleagues to gather ideas and share experiences. We are hoping you might be able to point us towards interesting open Public Sector Information data initiatives worldwide. Or, perhaps you know examples of the use of social media for improved cross-Government communications? Maybe you have a story to tell about other governments' uses of social media to interact with citizens? Please let us know in the comments section below.

Recently, the team met (virtually) with colleagues in the Government of Ontario to see their very impressive Opspedia social media suite for government. Opspedia is an enterprise pilot providing wikis, blogs and a professional network to enable staff to create professional profiles, connect with current and past colleagues, collaborate on subjects of professional interest and share recent work. We look forward to staying in touch.

On 19 October, the Digital Engagement team, with COI's Interactive Services, had a video conference with the US General Services Administration, the US counterpart of our Cabinet Office. This was the first held directly with the GSA and we have now agreed a monthly video conference to share best practice. They kindly provided us with details of their Search strategy with us and were keen to hear more about our experiences of using Linked Data and our approach to a cross-Government Digital Engagement strategy. Finally Andrew Stott contributed an article discussing the UK approach to Citizen Engagement to the General Services Administration’s Intergovernmental Solutions Newsletter.

Help shape URI standards for the UK Public Sector

As part of the government’s work on open standards and especially Linked Data for publishing, we have been working on  Uniform Resource Identifiers (URIs)  and how we should use them.

URIs will be a crucial part of the UK public sector’s technical information architecture, helping information sharing across government, providing better public services, and where possible, making re-use of published data much easier. URIs allow different organisations’ systems to share common meaning and common identifiers. Used across the public sector this will help join-up otherwise disparate data sets into a  web of data  that can be linked, queried and aggregated.

The Chief Technology Officer Council’s Information Domain have written some guidance on design considerations for how UK public sector URIs should be developed and maintained. It has been published as an interim standard , and we would greatly appreciate your feedback, so the people at Write to Reply have kindly hosted a version open for your input on which you can comment, commend or criticise.

Your comments are important as they will help us make the best choices so that Government data is as open and easy to re-use as possible.

Calling Open Data Developers: We need your help

From today we are inviting developers to show government how to get the future public data site right - how to find and use public sector information.

The developer community through initiatives such as Show Us a Better Way, the Power of Information Taskforce, MySociety and Rewired State have consistently demonstrated their eagerness and abilities to "Code a Better Country".  You have given us evidence and examples to help drive this forward within government.

We have an early preview of what the site could look like; we are now inviting interaction and comment from the developer community. With over 1000 existing data sets, from 7 departments (brought together in re-useable form for the first time) and community resources, we want developers to work with us to use the data to create great applications; give us feedback on the early operational community; and tell us how to develop what we have into a single point of access for government-held public data.

We know it is still work in progress, and there’s still a lot to do. That’s why we need you to help us get this right. Let us know what features or changes would make the site better for your and what other data sources you would like to see here.

To join in please sign up to the Google Group and we will get in touch.

Let your developer colleagues know and use #opendata on Twitter.

Reproduced with kind permission from Matt Jones