Hackathons are great, unless you want solutions

This recent paper about the value of hackathons made a few waves, with the surprising-but-not-at-all-surprising claim that most hackathon projects don’t continue after the event in any meaningful way.

Paper examining hackathons suggests that only about 5% of all projects are continued for more than 5 months [PDF] https://t.co/M3Z7N4A1Ve (maybe that's not a good metric, but was always curious about it)

— Tiago C. Peixoto (@participatory) August 27, 2020

I must admit — my first reaction to these responses was one of relief — knowing others out there aren’t afraid to take on the mantle of wet-blanket when it comes to dousing the hype fires of technological innovation when it sweeps through.

FINALLY have data to back up my hatred. Please no more hackathons. Even if 5% go forward, those concepts are highly unlikely to be driven by demand (and therefore impactful and sustainable) or to have been designed with the user #digitalprinciples #humtech #ict4d https://t.co/qvd0WA8bCm

— Laura WalkerMcDonald (@techladylaura) August 29, 2020

At the same time though, I don’t think that hackathons are useless (well, 95% useless) — it’s just that often there is a misunderstanding about the value of hackathons in the first place, who they benefit (and when), and what we should think of around our ideas of success.

As someone who has been on both sides of hackathon events (participant and planner) — I find this paper absolutely confirms my experiences of the hackathon deliverables themselves, but it misses a slightly longer-tail value of hackathons events, to organizations and people involved, and should also serve as a reminder to organizations who want to run hackathons as a way to build actual, viable solutions. Hackathons might not be the best way to go.

What we think hackathons do

Hackathons are often meant to bring people together to solve a problem through technological tools and development. They are often seen (particularly by leadership, but also from techies who let’s just say aren’t always in abundant supplies of humility when it comes to what they can achieve in a weekend) as ways of:

• getting quick wins in the form of rapid-cycle prototypes,
• incentivizing concerted efforts on a particular problem,
• breaking logjams on particular issues of data or platforms that might be useful to in-house teams,
• simply drawing attention to a particular problem, and coordinating teams to address it, and
• provide pizza and snacks to coders on a weekend where they have nothing better to do.

What hackathons don’t do

if you're a company and need your api testing, order $500 worth of pizza and call it a hackathon. job done.

— I Am Devloper (@iamdevloper) September 3, 2020

The above-paper-notwithstanding when it comes to the sustainability of hackathon projects, there are some other challenges when it comes to hackathon events that are challenging to address effectively, such as:

• Getting good products — ones that aren’t just demo-ready, but the basis for continued work. This is maybe one of my biggest challenges (both as a participant or from the perspective of a judge — it’s just really hard to evaluate code/product over polish/presentation. Additionally, hackathon projects aren’t saddled with the unsexy aspects of software development such as research, buy-in, technical debt, contractual constraints, software privacy and reliability constraints, and many other constraints…
• Cost-effective — Even “small” hackathons of 100 people can easily cost over $10K, and when factoring in event space (well, pre-COVID), food, prizes, and however you want to slice and dice costs of labor.
• Complexity— Hackathon topics are outside of internal-technical organizations are often wicked problems — complex, nuanced, requiring an understanding of dynamic environments that can’t be conveyed through a problem statement and 10K CSV file, nor solved by a native smartphone app.
• Diversity — Yeah, this one is big. It often gets truncated to “we need more than developers, we also need UX designers, content specialists, graphic designers, security experts, etc.” BUT it’s more than just the roles. Turns out that it’s really hard to do hackathons well when it comes to diversity of demographics — so things like gender, background, opportunity cost (when was the last time you had an entirely free weekend, without errands to run, kids to manage?)

What hackathons should do

Despite the challenges, there are real advantages to holding hackathon events.

• Attention — they can be very good at focusing a team or organization’s efforts on a particular problem area, project, or set of problem statements whose process of creation may be valuable in its own right. They can also be good for sending a signal to partners and clients that this is a priority area of work.
• Team-building — I’ve met some really interesting folks at these events, and even when they are completely in-house, they can be a great way to socialize team-members who don’t otherwise get a chance to get to know one another.
• Skills development — Similar to team-building, it’s helpful just to see how others work. Seeing an expert fly through wireframes in PowerPoint and share them with anyone/everyone helped me realize I could up my own game, for example, and it’s helped immensely.
• Proof-of-concept — Yes, I saved this one for last because I do think that it’s in many ways the most challenging to get right. But every so often there is a project or two that really can have this effect, and start you off down the right path. I learned about the Meteor framework through a Berkely hackathon, and that got me started down the path of JavaScript frameworks (I like Vue.js these days).

There are many good summaries of how to manage successful hackathons that are out there — and many of them touch upon these points.

I also think it’s worth stepping back and asking if “hackathons” in the classical sense are the most effective ways of getting there. I’ve seen a lot of success from events focused on the data itself (a data-cleaning hackathon!) that has useful deliverables to stakeholders.

I hate to say that the value of hackathons lies in everything but the finished product, but that has been my experience. If you’re looking to build software solutions, there are plenty of other apps for that.