1.1 O4C hackathon background

With more than a half of global population living in cities (a proportion that is expected to increase to two thirds by 2050) the quantity and quality of urban services made available to citizens by the local government – directly or through specific agencies and/or alliances with civil society – becomes of prominent importance. Indeed, urban services deeply affect our lives by shaping and constraining the way we experience, and interact with, the city organization in its manifold aspects. Traditionally, this was related to the fulfilment of the basic needs of a decent life - from street cleaning to domestic water supply, from waste collection to sanitary sewer systems - issues that have been in focus of city managers for thousands of years now. With the emergence of the Welfare State concept about a century ago, and even more recently with the diffusion of an environmental consciousness among the general public, urban planning and administrative processes have been growingly held responsible for a variety of additional functions, such as the protection of citizens’ health and safety and the ecological preservation of urban landscape. This has gradually led to a holistic notion of urban services, touching upon all aspects of individual and collective life, including education, culture and leisure time; not to mention social integration and civil rights.

However, experience shows that usually there is a considerable gap between the way urban services are designed and what the citizens expect from them. Such gaps may emerge because of the service design logic – being driven by too technical goals or adopting too bureaucratic approaches – or as a consequence of substantial societal changes, which the existing services are unable to address. This gap generates constraints on the everyday life of citizens, that makes such services socially unsustainable. As a result of this situation, civil society or the business community can come up with alternative ways of fulfilling the same specific needs or service expectations, but the impact of these on city organization can be unpredictable, if not controversial – as the example of “Uber” is there to show. Also, the local government itself, supported by disruptive technologies (such as those typically gathered under the Smart City label) and adopting innovative delivery models (such as the public private people partnership, engaging citizens in service co-creation), can decide to undertake an urban service transformation pathway of its own.

This handbook addresses the information needs of anyone involved in city management and administration – whether as a “regular citizen” or an elected official, civil servant, or member of a nonprofit or nongovernmental organization – willing to seize the opportunities offered by the availability of open data to address service transformation goals or to increase the social sustainability of urban services. We offer a deeper insight into the global practice of hackathons, which has been popularized by many successful instantiations (particularly from the US) and that we are trying to make even more inclusive than in the concrete examples we know about: not only a technical marathon for coders and service hackers, but also an opportunity for the general public to co-design the urban services they need, together with the public agencies (and equivalent stakeholders) in charge of delivering them.

1.2 Stakeholder engagement

Fundamentally a successful O4C hackathon is an event that 1) enables participants to make real and meaningful use of open data, and 2) invites the participating teams to address problems of relevance to specific topics and actual “problem owners”. This requires a continual involvement of a wide range of stakeholders spanning both pre-hack planning and preparation, the hackathon itself – but also facilitation in the post-hack follow-up.

As you begin reaching out, promoting and recruiting, it might be helpful to think of potential stakeholders on a spectrum between Collaborators that help “build the ground” as you arrange the whole hackathon process – and the Participants you expect to engage at the actual hack-events (of course this may overlap; collaborators might also participate and vice versa).

Table 2 Stakeholders of the different hackathon events

	Pre-Hack	Hackathon	Post-Hack
Problem owners, issue/subject matter experts and/or passionate “grassroots” / curious citizens	Need to be involved to ensure definition of relevant topics and themes for the hackathon. Only in this way actual problems of real value can be addressed.	“Problem holders” or “issue experts” are crucial for inspiration and continual relevance and context of the problem/solutions being worked on. They are “experts” as they are directly touched by the main theme. Some may even represent the problem in all its aspects; as problem holders.	To allow for a “continued life” for the solutions and ideas proposed during the hackathon, problem holders and issue experts are key. (e.g. figuring as relevant partners proposing data or enabling testing).
Public authorities/services, Representatives from Municipal, Regional or national level and/or Interest groups and NGO’s	Can provide important insights about existing organizational, business related and institutional settings. Also they can help define a relevant problem and/or provide access to relevant data.	Able to provide leverage and context. Can provide informed understanding of existing (social) infrastructure as well as relevant political contexts.	Can play a role in ensuring the continued work, development and incubation.
Students and researchers (e.g. Programmers, Gamers, Coders, topic related study fields, designers, digital methods, data journalists or other)	Can engage projects that inform the planning of the hackathon events.	Participation, Ideation, Networking and Learning.	Can pursue projects that test and develop concepts and prototypes further after the event.
Open data enthusiasts, scientists, journalists or other mindful techies and data explorers	Can provide expertise that allow for thinking creatively about data acquisition and data possibilities. Can assess and search for relevant open data, examine it and rate it in terms of openness and relevance. Some data might also need to be converted into the relevant format.	Ensure provision of documentation of the data made available for the “hack”. Assisting the participants to access the data - or develop ideas with the data in mind; incorporating it into their concept/product (without having to actually tinker with it themselves)1.	Can advice in the continued post-hack process of updating and providing relevant data sets.
Start-ups and creative industry	Networking in terms of promotion inspiration and ensuring the recruitment of a critically thinking and creative crowd.	Incorporating attention towards sustainability and continued economical life of projects.	Can play a role in ensuring the continued work, development and incubation.
Sponsors of various kind	Think already early about making agreements with relevant collaborations.	Making it possible to host an event with food, attractive venue, swag and maybe even a prize to win.	Can play a role in ensuring the continued work, development and incubation. Supervision and mentoring.

We recommend keeping an eye on involving representatives from the clusters suggested in Table 2 – however who’s crucial or not may depend on the specific topic and challenge to be worked on. Also, the proposed inputs and activities in the matrix might very well overlap, we just aim to sharpen your attention towards these dimensions.

1.3 Evaluation

The O4C Evaluation Framework takes a pragmatic and flexible approach to assessing the impacts and benefits of our processes and instruments. The evaluation framework is intended to guide the O4C consortium partners in the five pilot locations to consistently and coherently measure the hackathon process through which the project hopes to achieve its overarching aims. The outcome towards which the project is acting is a positive (social) impact of the OpenDataLabs on the social sustainability of urban services, to be achieved by narrowing the gap between: 1) the way services are designed and citizens' everyday lives; 2) the technological possibilities offered by open data availability and stakeholders' ability to understand and use the possibility. The full evaluation framework includes tools and guidelines for data collection and analysis, as well as the timings of these activities. Reflections on the data gathered will take place at regular intervals in order to share learning, mitigate problems and negative outcomes, as well as maximize positive outcomes locally/nationally and across the project partners.

This evaluation contains both formative and summative elements. The focus is on process evaluation, to facilitate learning and sustainability, taking inspiration in particular from the performance audit approach and tools laid out by Ling and Villalba van Dijk (eds.) (2009)². The detailed evaluation framework will be found in O4C Deliverable 4.1 Evaluation Framework, together with supporting tools and guidelines and including theoretical reflections on choices made in its development. Points regarding evaluation in this deliverable will therefore serve as reminders of activities, resources needed and timings. For full detail of templates to be used and guidelines, please see Deliverable 4.1 as well as information related to task and Deliverable 4.2 Data Collection and Evaluation.

2.1 Pre-hack: Interconnected exploration of Challenge topic, Data and People

By outlining our recommendations in this handbook, we won’t attempt at supporting any predetermined recipe. Through an illustration of what we’re up to, we hope to inspire and/or provoke innovative responses and similar initiatives.

It turns out there is a lot more at play in the early planning and preparation phases than what might originally be envisaged. It is therefore helpful to map out and label the myriad of activities that lead to the accomplishment of successful hackathons, aligned across locations within the O4C Consortium (as well as to be replicated beyond them). In the following we detail the scoping phase³ - or what we refer to as Pre-hack activities.

At its core, “pre-hacking” is open, dynamic, agile and evolves over several months. As all pilot domains differ, pre-hacks are all about exploring options, collaborations, datasets and topics - and being able to respond to what is being learned and discovered in the process.

Figure 3 Illustrating the 3 dimensions of co-creation during hackathons

To prepare a relevant context and line-up for the co-creation during the hackathons many elements and resources have to be brought productively together. In particular three overall dimensions deserve attention: People, Data and Challenge Topic (Figure 3).

The pre-hack phase is about defining, framing and reframing the challenge topic to be worked on⁴ (Challenges). Ideally this process is closely weaved together with the exploration and examination of the data possibilities for a given topic; what is available and how open is it? (Data). As well as continual engagement and involvement of relevant citizens, stakeholders and interest groups (People).

2.2 Challenge topic: What makes for a good challenge topic?

The O4C hackathon aims to engage citizens and communities around open data under an overarching challenge topic. This corresponds what might be labelled a “themed hackathon”. The advantage of this compared to “technology hackathons” is that it is relatively easier to attract “domain experts” with knowledge and passion about the selected subject matter. As an organizer, you will need to be mindful about how you plan the facilitation of projects and work that revolves around the problems brought to the table by these topic “experts” or enthusiasts.

As you narrow down the ideal problem and challenge topic to focus on, you might start with the availability of interesting open data - or you might be triggered by the urgency of a particular problem or urgent crisis to tackle. Either way there are several aspects to keep in mind and fold together:

• Strive for a topic that is clearly articulated either as a coherent question or formulation of a clear social need(s). It should provide clarity and focus; inviting someone to solve the problem with a relatively specific solution. At the same time room must be maintained for innovative, diverse and creative responses.

• Continuous involvement of issue experts/enthusiasts is crucial as they enable real-world relevance. Without relevant people continually and actively involved you risk ending up having hackathon participants “solve problems that don’t really exist”

• Engage in dialogue with both domain experts/enthusiasts with knowledge about the topic and people that are knowledgeable about open data. Both parties can give valuable inputs.

• Ongoing search for and analysis of the data sources available. Make sure the necessary resources (e.g. datasets) exist; that they can be made available for participants to design effective propositions.

2.3 Data: Searching for data, selecting relevant tools and allowing for inclusion

Data exploration informs the overall formulation of what topic to be worked on at an O4C hackathon. You should make sure to begin data investigation as early as you can, in order to make sure that relevant datasets can be collected to form the ground for the development of adequate solutions in the hackathons.

Ideally, bundles of potentially relevant datasets are to be collected and organized on the online platform for the project (OpenDataLab.eu) together with a set of tools and guidelines that will support the visualization and manipulation of data. For a quick overview over the functionality of the OpenDataLab Platform see section 2.9 and for more a more detailed description, see O4C D4.5 Data Management plan section 3.

A big challenge however, is the relative immaturity of open data as a broader phenomenon and the varying local dedication to the publication of such data. Some datasets might not be of actual value due to their formats, the lack of continual update, missing information, incomplete metadata or other – if the datasets are even made publicly available at all.

Therefore, you had better prepare yourself to such a time-consuming task as to find, collect and prepare relevant datasets. Also you might want to think alternatively about what data to make ready (for instance, data on e.g., empty apartments in an urban area, may not be immediately relevant for a hackathon that is working on migrant communities; nevertheless, the presence of such data could prove crucial to develop some of the solutions emerged during the hackathon). You are best-off approaching data collection with an open mind and a critical angle. For more details about the technical requirements of data for the platform, see D4.5 section 3.

When you strive to include technically skilled people or students – as well as ordinary citizens who are passionate about a particular topic, you need to make sure to design and facilitate the inclusiveness of the process; also when it comes to working with and making use of data.

If you value and support only certain forms of engagement (e.g., traditional hack skills such as programming and coding) you will limit the size and diversity of the pool of potential participants. You need to be mindful about how openly/broadly or narrowly you structure the opportunities for engagement and possible solution proposals.

If you aim at engaging a broad variety of people and citizens to interact and play with data – you will need to allow them to do so in the ways they feel inclined to. It is absolutely key to maintain an understanding of data that is more inclusive than exclusive!⁵

Pre-hack activities involving data that is recommended to enact include:

• Search for, analysis of and preparation of data sets. This should ideally involve assessment of relevant open data published by public sector organizations along with the examination and rating of the data in terms of its openness and quality.

• You might consider a demand-driven perspective on data during the pre-hack phase. This is done by approaching your stakeholders such as collaborators and/or your participants and try to bring the possible challenge topic into play as early as possible. This will help you create a prioritized list of open data to collect.

• Think quality not quantity! Rather than importing whatever ready-made found in open data repositories, be critical and go for datasets that are really ready to use (e.g. can be converted into the right format, organized in an accessible library with searchable, headings and meta-data that describe the contents of the dataset).

• Prepare yourself that working on the data might be time-consuming. Expect an iterative and ongoing process. Some of the datasets will not be available at the time you need them, some will be difficult or even impossible to get, or be out-of-date, and others will need to be changed into another format or aggregated.

• Start considering how the uncovered data will be made available for the participants and take the characteristics of the data into consideration - e.g. if a dataset is frequently updated a simple dump to an online library will expire over time and potentially lose its value. Beware that gathering data takes time and making it available is a grand task in itself.

• Clear documentation and overview need to be prepared for the participants. You will need detailed consideration of how to assist participants in terms of accessing the available data. It is necessary to provide overviews and clear guidance about what data is available and an assessment of their quality. You need to make a strategy for how you will support the understanding of participants and citizens of the potential of both the available datasets but also open data in general. They should be capable of and allowed to developing ideas and incorporating open data into their proposed products – even though they might be unable to find and assess the data themselves; tinkering and programming.

• Another key activity is selecting visualization tools and methods for engaging the hackathon participants through the OpenDataLab platform suite. Such instruments might be made available to the potential participants before the actual pre-hack; to invite participants to experiment, build understanding and expectations - or even proposals for the hackathons.

• Prepare for involving people with a wide range of knowledge in terms of IT and skill levels: from the ignorance of IT application and open data, to the expert knowledge into technological aspects that would support possible solutions and explore the existing datasets on the basis of their relevance for the theme of the hackathon.

2.4 People: Stakeholder engagement

There are numerous good reasons for working closely with collaborators and stakeholders in various ways - in terms of defining a relevant topic/problem to be worked on, accessing and fetching appropriate data sets, preparing the logistics of the event (food, venue, tech and sponsorship) as well as reaching out towards a diverse pool of participants in the pre-hack. In the following we will expand a bit on what we’ve already outlined in Table 2 in Section 1.2.

We suggest you strive to cover representatives that span the Quadruple Helix model⁶ for innovation processes. This means aiming to facilitate that civil participants work together with representatives from government, industry and academia. The ideal here is the enabling of a collaborative future of open data that aim to equally engage institutional bodies, public authorities, research arenas, business sector, creative industry and curious citizens.

Zooming a bit more in, it works well to have an eye for involving the proposed clusters below (Table 3). It can inspire new ideas for contacts to make if you critically care for mapping out the status of engagement as you plan an O4C hackathon yourself. Depending on what topic and data you work on – as well as the local collaborative composition available to you; some may be of particular relevance while others may be more peripheral.

Table 3 Stakeholders’ engagement at the pre-hack events

	Pre-hack
Problem owners, issue/subject matter experts and/or passionate “grassroots” / curious citizens	Need to be involved to ensure definition of relevant topics and themes for the hackathon. Only in this way actual problems of real value can be addressed.
Public authorities/services, Representatives from Municipal, Regional or national level and/or Interest groups and NGO’s	Can provide important insights about existing organizational, business related and institutional settings. Also they can help define a relevant problem and/or provide access to relevant data
Students and researchers (e.g. Programmers, Gamers, Coders, topic related study fields, designers, digital methods, data journalists or other)	Can engage projects that inform the planning of the hackathon events
Open data enthusiasts, scientists, journalists or other mindful techies and data explorers	Can provide expertise that allow for thinking creatively about data acquisition and data possibilities. Can assess and search for relevant open data, examine it and rate it in terms of openness and relevance. Some data might also need to be converted into the relevant format.
Start-ups and creative industry	Networking in terms of promotion inspiration and ensuring the recruitment of a critically thinking and creative crowd
Sponsors of various kind	Think already early about making agreements with relevant collaborations

Below we enlist a selection of relevant dimensions to think about when involving collaborators in various preparation stages of your process - as well as when you promote and recruit participants for the hack events⁷:

• Engage with the real people who express their interest in your project and aim to prioritize everyone you involve - think about what it means to build “with” and not “for” your collaborators and participants.

• Have in mind what motivations might make various stakeholders, collaborators and participants curious and interested enough to engage in your project efforts. Communicate clearly and directly to each of them - one size might not fit all!

• Take point of departure in existing initiatives and (social) infrastructures - this is crucial to gain leverage and navigate relevant sociopolitical contexts. There can be huge synergy-advantages in teaming up with existing organizations and institutions already doing work on the field you’re stepping into! Explore the best ways to partner up!

• Be kind, flexible and bold in a “participatory snowball method” of involvement: use the contacts you make to “co-strategize” about who is important or not; who would they recommend from their network of contacts? (As collaborators, sponsors and participants?)

• Caring for the relations you build is important because only through the active engagement and involvement of relevant people, communities, authorities and organizations; will you allow for addressing needs driven by communities.

• Aim to allow for a process that invites the expression of community ideas, needs, wants, opportunities and aim to let this input drive problem-identification and problem-solving. – And make sure to capture and document the needs, ideas, skills and contributions that are expressed in this process.

• Make sure that the people invited to the hackathon represent a reasonable cross-section of the social ecosystem related to the main theme the hackathons will cover. Keep, update and dare to share your list of relevant social groups to approach, to have them represented in the hackathons.

• In terms of participation: you need to strive for a match between the skills needed for the project/challenge topic to be worked on – and the skills expected to be brought by participants (and in sufficient quantity to form interesting groups).

2.5 A palette of suggested pre-hack activities

Keynote lectures. Work well as sharing of common ground at a meetup or as an inspirational lunch talk about open data or some case study to create awareness. Might tease in relevant students or other around data or topic-related issues.

Data exploration sessions or “data sprinting”. A format or a digital method to be scaled at different levels ranging from a 2 hours intensive exploration around either a particular sample of datasets or a given topic. Some variants of it are more well-defined and run for a whole week to analyses data, produce visualizations and network graphs as well as give shape to further analysis.

Gatherings and meetups. A meetup is a format for self-organized local groups often aimed towards improving the communities they live in or discuss a topic they are passionate about. As such it might be a relevant activity for you to prepare the hack; remember to allow for keynote-talks and mingling as these are the first steps towards the creation of a community.

Design challenge. Define a design brief related to the topic, that is attractive to students. Students do these things to challenge themselves and get a nice portfolio item out of it (the least), but if there are prizes it’s even better. Another motivational factor for students is to get to network with design agencies (or companies with designers), so jurying by some experts is also an interesting proposition.

Roundtables and workshops. These events work well as an occasion to bring together relevant subject/topic experts and open data experts. The purpose is to engage them in the design of challenges and problems to be worked on e.g. the specification of problems and opportunities you will be facilitating the hacking activities to address.

A narrative and scenario-based approach. It might be included in the range of pre-hack activities you choose to engage. As one of the concrete outputs from roundtables, workshops, interviews, meetings and other activities you might strive to produce a series of sample descriptions that use narrative styles to help the team approach potential hackathon participants to figure out the potential and opportunity of the co-creation activity in the hackathon. Concrete and detailed scenario-sketching with people moved by the problem in focus, also work well to suggest the inclusion of further datasets. Be aware though that such sample description do not represent the whole range of possibilities to explore in the hackathon - the hackathon must be open to further creative opportunities.

Interviews and meetings. Such events enable that you elaborate specific challenges through the description of problematic areas and opportunities, possibly using scenarios or other narrative techniques, that can be used in the following steps approaching the hackathons, to engage other relevant stakeholders.

2.6 Promotion planning and recruitment

As we have already proposed in terms of engaging stakeholders of various kind, you will need to make sure to become visible on the local arena of the topic or place to be explored, for example, through explorative dialogue, taking part in events and meetings. Seeking out particular people/organizations to contact either to directly involve them in workshops or try to recruit them to play a role in the hackathon events.

Promotion is a core dimension in most of the preparational activities that will fill your calendar (be that interviews, meetings, workshops, participation in other events and activities, roundtable panel discussions or other). At every step and with every contact pay attention to how best to enroll the relevant actors and care for what motivates them to become involved.

Make sure to define a strategy for promotion and recruitment that is tailored to the local specificities, interests, availabilities and preferences. Issues to take into account:

• Settle the dates for the hackathon you will host and choose a venue

• Make sure to have a place where all public information about the event can be accessed (e.g., a website or another publicly accessible place)

• Outline an ideal timeline of the promotion plan and recruitment:

  • When do you want people to sign up/register their interest?

  • What materials, tools and steps are needed - and when?
    (Adds, notices “save-the-dates-teasers, posters, Eventbrite accounts, animated videos, brochures, other?)

  • What social media platforms are of relevance reaching out to the group of people touched by the topic you’ll work on? Tailor assets/communication strategy towards how, when and what social media presence might make best sense (Facebook, Twitter, Instagram, Google groups, mailing lists or other…). However: Never spam anyone!

  • Choose a hashtag - and consider to set up a Tumblr⁸ or Hackpad⁹ page for teams to record progress and post links.

• If you run a themed hackathon on open data, you need to plan for how to reach different audiences: How will you reach the communities that will care about your event? This includes both people with interest in open data (the local open data / open knowledge community) as well as other curious citizens, students and many more. Some may be interested in open data, others may not bother about open data at the outset - but might be curious to engage intensive problem solving on a particular topic.

• Aim for clear communication in terms of what might motivate and pique people’s interest. Do they need input to organizational innovation development? Are they attracted by network and learning opportunities? Are they curious to receive support and guidance on particular tools or data-sets?

• Don’t forget about reaching out towards decision-makers; think about what story might be of interest to them.

• Be mindful about how you want to tell your story - how would you like people to remember it?

• Take photos - crucial part of telling and richly documenting the story of your hackathon!

2.7 Preparing logistics: Venue, Food, Swag and Sponsorship

2.8 Sign-up and managing applications

2.9 OpenDataLab Platform

2.10 Evaluation-related data gathering

Data gathering during the pre-hack phase involves establishing a baseline for future evaluation data and analysis related to 1) the O4C project in general and more specifically at each pilot, and 2) hackathon participants and other stakeholders.

The theory of change of the Open4Citizens project will be further clarified with the consortium partners, i.e. being explicit about our hypotheses regarding the O4C project, identifying assumptions we have made and exploring any implicit assumptions, as well as whether these hypotheses have changed since the project’s initial description of activities. Supporting this, we will build the contribution story at the pilot level, i.e. understanding why each pilot team believes that their use of O4C project resources in their particular context will create social value and which unintended consequences or side-effects there may be. We will return to these questions throughout the 30 months of the project in order to capture the causal effects and mechanisms with the aim of gathering evidence about what we think we are doing, why, and what is actually having an impact.

Example questions for O4C organizers to respond to:

• Do you have any concerns before the hackathon about how successful it will be?

• What do you think will be the most challenging aspect of the hackathon in terms of organization?

• What do you expect to be the most innovative and generative element of the hackathon?

• What one thing, if achieved, will have made the hackathon a success for you?

For hackathon participants, this involves determining their attitudes and knowledge, as well as collecting standard demographic and statistical data about participants and other stakeholders to contextualize other evaluation material. See Section 2.8 in this document on the registration process, where this baseline data gathering will be incorporated.

Baseline data about participants can be gathered during the registration process, as a short questionnaire. This could profitably be linked to on the O4C website or OpenDataLab platform as a Google Form or using SurveyMonkey, so that data gathering is standardized for all pilots. Some baseline questionnaires may have to be carried out manually at the beginning of the hackathon, if participants have not completed them before the hackathon.

Example questions to ask hackathon participants:

• What do you hope to get out of the hackathon (up to 3 things)?

• How confident are you that you will get these things out of the hackathon? (Likert scale)

• After this hackathon, how interested are you in continuing to engage with the Open4Citizens project? (this does not commit you to anything) (Likert scale)

Table 4 Pre-hack evaluation data gathering

Time	Stakeholder group	Evaluation activity	O4C pilot resources needed
M8 (August 2016) - Hackathon cycle 1 M20 (August 2017) - Hackathon cycle 2	O4C evaluation representative group (at least 1 person from each pilot and from each consortium member)	Baseline O4C theory of change (focus group interview)	Time for meeting, led by Antropologerne, or written response to guiding questions Possible individual follow-up meetings for clarification
M9 (September 2016) - Hackathon cycle 1 M21 (September 2017) - Hackathon cycle 2	O4C pilots, separately	Baseline pilot contribution story (focus group interview)	Time for meeting, led by Antropologerne, or written response to guiding questions Possible individual follow-up meetings for clarification
Ready for online hackathon registration to open (at least 1 month before the hackathon)	Hackathon participants (“hackers”)	Baseline determination of motivation and interest.	Template questionnaire translated into pilot language Uploaded Available for completion at the hackathon for late registration (online & printed)
Ready for online hackathon registration to open (at least 1 month before the hackathon)	Other stakeholders related to hackathon activities (e.g. inspiring speakers, data experts)	Baseline determination of motivation and interest.	Template questionnaire translated into pilot language Uploaded Available for completion at the hackathon for late registration (online & printed)

2.11 Overall pre-hack timeline and checklist

Figure 4 Illustrative generic mapping/visualization of key Pre-Hack Activities in a proposed timeline counting down to event.

A more detailed list of activities and the timeline is shown in Table 5.

Table 5 Timeline checklist towards the pre-hack

What	When	Type of activity / Color label
Challenge topic definition begins	6 months before	Challenge topic
Data search intensifies	4 months before	Data
Final topic is announced	3 months before
Save the date notifications are promoted	3 months before	Promotion and recruitment
Registration opens	2 months before
Coordination with collaborators	1 months before
Find helpers to assist you in the execution of the event (registration table, tech support or other tasks…)	3 weeks before
Make sure to reach out to the channels where you want the story of your hackathon to be told (media and other)	2 weeks before
Make sure you have available relevant amount of post-it notes, whiteboards and flipcharts, pens etc.	2 weeks before
Email with information is send to participants	1 week before
Double check food and other logistics to be in place: make food catering orders (if needed)	3 days before
Check locations: are you all set?	1 day before	Logistics / Practicalities

4.1 Sustaining the momentum from hackathons

There is an obvious limit to how detailed and ready an idea can become during a 3-day hack-session. This means that often ideas are not taken any further – as multidisciplinary teams dissolve and pack their bags to return to their various “homes” (e.g. often teams are formed the hackathon as an occasion). This dissolution is a recurring challenge in relation to the pros and cons of using the hackathon as a format for innovation. The question of how to support and elaborate the “innovative baby-steps’ taken during a hackathon is therefore an important consideration worthy of exploration.

As part of our O4C endeavor we therefore advise, to carefully think through how support will be given to nurture the ideas ignited during a hackathon. Ideally strive for making arrangements for a tempting and tailored “post-hack” package and communicate it clearly already early on; it might work as a motivator for many of the potential participants to know what support is offered elaborating their ideas further.

There is a wide palette of options to consider - and how the post-hack is best configured will depend heavily on the partnerships and collaborators around the core organizer team for a hackathon - as well as the topic to be addressed.

Suggested overall dimensions to guide you to set-up arrangements that allow for hackathon outputs to be further worked upon, tested, and deployed in real-life contexts so as to have an impact:

• Assistance and mentoring in terms of how to engage in app development;

• Assistance and mentoring about taking user-perspectives into account;

• Loops and iterations with relevant user-groups;

• Assistance and mentoring in terms of business models.

While planning the hack-events, every local partner in the O4C consortium has also been working with the progressive establishment of the OpenDataLab platform advocated and updated by and through the local gatekeeper to the resources available.

We strive to ignite a network promoting meaningful use of open data - and to nurture ideas growing out of the hackathons. We are the “middlemen” that involves relevant partners and gathers appropriate stakeholders. Ideally this takes the shape of an “incubator structure” that includes both partnership with companies and start-ups, (existing local) makerspaces or other existing structures of overlapping interest. A key concern in this process is the attention towards how relationships are framed between stakeholders of various kind (citizens, startups, companies, universities, public authorities, etc.).

Depending on the level of readiness for a proposal – and the composition of a specific team behind the idea; a key aspect might be the support the software development of an application. Another aspect is the support that enables groups to ensure business plans and inspiring economic sustainability for the service/application proposals that stem from the hackathon. Ideally, real start-ups and companies are involved as supporters/partners/ mentors here: helping them with facilities and with legal, programming/development and communication matters. It is crucial to strive to nourish economical sustainability and help implement viable business models for the ideas that are developed.

Post-hack activities might include:

• Events, Open Data Day gatherings, keynotes, sprints or service-jams;

• Mentoring and or a package of “consultation hours” for the teams from the hackathon (App development, business models, entrepreneurship, legal advice or other);

• Student project work;

• Planned follow-up activities/events that allow for further learning as well as invites further work and testing to be demonstrated, explored and applauded.

A nice example of making sure to actively integrate the post-hack preparations into the planning of the hack-event, would be the Swedish O4C take on how to design and interesting post-hack phase: they ensure continued focus and learning to continue as the proposals and prototypes from the hack-event will be demonstrated and presented at a convention/conference a month after the actual hackathon.

If you are not able to invite everyone to take part in a follow-up post-hack development phase, you better plan for how to select an appropriate number of teams / ideas to be further worked upon. If you set new teams cross-cutting geographies or educations or other affiliations, how do you plan to nurture and host a continued collaboration?

4.2 Implementation follow-up

In the post-hack phase, it is crucial that the solutions that were developed during the hackathons can be tested and re-iterated. This surely puts a demand on the continued availability of tools, data and other technical resources that participants might have had access to during the hackathon.

4.3 Evaluation

Data gathering during the post-hack phase involves following up on the evaluation data and analysis collected in the pre-hack phase and during the hackathon related to 1) the O4C project in general and more specifically at each pilot, and 2) hackathon participants and other stakeholders.

The theory of change of the Open4Citizens project will be further developed and reflected upon with the consortium partners. Supporting this, we will reflect on the contribution story at the pilot level. We will return to these questions in both hackathon cycles and throughout the 30 months of the project in order to capture the causal effects and mechanism with the aim of gathering evidence about what we think we’re doing, why, and what is actually having an impact.

Example questions for O4C organizers to respond to:

• How successful was the hackathon?

• What was the most challenging aspect of the hackathon in terms of organization?

• What was the most innovative and generative element of the hackathon?

• What one thing that you achieved made the hackathon a success for you?

For hackathon participants, this involves determining their attitudes and knowledge, as well as collecting standard demographic and statistical data about participants and other stakeholders to contextualize other evaluation material. See the section of the handbook on section 4.4 on Community follow-up for where best to incorporate post-hack follow-up with various participants and stakeholders. Follow-up should take place as soon as possible so that the memories of the hackathon are still fresh in participants’ minds.

Follow-up data about participants’ experience, knowledge and interest can be gathered as a short questionnaire with a similar structure to that for baseline data gathering. This could profitably be linked to on the O4C website or OpenData platform as a Google Form or using SurveyMonkey so that data gathering is standardized for all pilots. Some follow-up questionnaires, e.g. with particularly valuable stakeholders, may have to be carried out in a phone call or face-to-face meeting or at least by sending them in an e-mail, if some participants are hesitant to complete them. However, this is a time-consuming approach, as all data will need to be consolidated online.

Example questions to ask hackathon participants:

• What did you get out of the hackathon (up to 3 things)?

• Did you get as much of these things as you had hoped these things out of the hackathon? (Likert scale)

• How interested are you in continuing to engage with the Open4Citizens project? (this does not commit you to anything) (Likert scale)

Table 9 Post-hack evaluation data gathering

Time	Stakeholder group	Evaluation activity	O4C pilot resources needed
2 weeks after the hackathon, enough time for O4C pilots to have gathered and initially reflected on evaluation material from their hackathon	O4C evaluation representative group (at least 1 person from each pilot and from each consortium member)	Developments in O4C theory of change (focus group interview)	Time for meeting, led by Antropologerne, or written response to guiding questions Possible individual follow-up meetings for clarification
1 week after the hackathon, allowing enough time for O4C pilots to have gathered and initially reflected on evaluation material from the hackathon	O4C pilots, separately	Developments in pilot contribution story (focus group interview)	Time for meeting, led by Antropologerne, or written response to guiding questions Possible individual follow-up meetings for clarification
Ready to send the day after the hackathon	Hackathon participants (“hackers”)	Follow-up determination of motivation and interest.	Template questionnaire translated into pilot language Uploaded Available to post or send in e-mail
Ready to send the day after the hackathon	Other stakeholders related to hackathon activities (e.g. inspiring speakers, data experts)	Follow-up determination of motivation and interest.	Template questionnaire translated into pilot language Uploaded Available to post or send in e-mail

4.4 Community follow-up

In the post-hack phase, community involvement is as crucial an element of the Open4Citizens hackathon process as in the previous phases. The continued effort and interest of the community in supporting the implementation of pilot projects is the best guarantee that the ODL will achieve a real impact at the local level. The shape and procedure of community involvement will vary depending on the type of stakeholders that takes the lead in the post-hack development. We can already foresee that a range of stakeholders may want to follow up on hackathon outputs, to further develop those scenarios of usage of open data that hold the promise of bringing value to their everyday activities. These stakeholders may be civil society organizations, public administrations, SMEs, or tech-savvy volunteers.

In Figure 7 below, the different pathways and rationales for community involvement in the development of pilot projects are explained in more detail.

Figure 7 Pathways for community involvement in the post-hack phase

In a fully-functioning, post-Open4Citizens OpenDataLab, the hackathon would be the centerpiece of the Open4Citizens methodology, serving the role of a regularly held event that brings in new co-created proposals for scenarios of usage of open data into the ODL’s everyday activities.

Regarding the terms of participation of the community in the post-hack phase, one of the most crucial points pertains to the intellectual property rights (IPR) of the developed solution. In this sense, it is essential that the rights to own and the rights to use of the developed solution are fully clarified before work commences, by the signing of a document stating how the IPR of the solution are to be allocated. The role of OpenDataLab role in the whole process is that of the facilitator and counselor, providing information to all parties on legal possibilities (i.e. Creative Commons licenses available) so that they can come to an optimal understanding, and will not claim total or partial ownership in any community-developed project.