Publications & References

VOLCROWE Publications

Authors: Masters, K., Oh, E., Cox, J., Simmons, B., Lintott, C., Graham, G., Greenhill, A. and Holmes, K.
Title: Science learning via participation in online citizen science
Download Link
Year of Publication: 2016
Publication Details: Journal of Science Communication (Special Issue on Citizen Science)
Abstract: We investigate the development of scientific content knowledge of volunteers participating in online citizen science projects in the Zooniverse ( We use econometric methods to test how measures of project participation relate to success in a science quiz, controlling for factors known to correlate with scientific knowledge. Citizen scientists believe they are learning about both the content and processes of science through their participation. We don’t directly test the latter, but we find evidence to support the former – that more actively engaged participants perform better in a project-specific science knowledge quiz, even after controlling for their general science knowledge. We interpret this as evidence of learning of science content inspired by participation in online citizen science.

Authors: Greenhill, A., Holmes, K., Woodcock, J., Graham, G., Lintott, C., Simmons, B., Cox, J., Oh, E.Y and Masters, K.
Title: Playing with science: Exploring how game activity motivates users participation on an online citizen science platform
Download Link
Year of Publication: 2016
Publication Details: Aslib Journal of Information Management
Abstract: Purpose – This paper examines intrinsic forms of motivation and particular incidents of play, socialisation, fun and amusement on an online crowdsourced citizen science platform. The paper also investigates gamised activity (Greenhill et al., 2014) as a form of intrinsic motivation adding a sense of play to work and tasks (Xu et al., 2012). These concepts are explored through close scrutiny of the online citizen science platform
Design/methodology/approach – Qualitative techniques with an interpretivist approach are used to analyse online content found within citizen science platforms, related forums and social media by examining incidents of play, socialisation, fun and amusement to investigate how these aspects are applied as a form of user motivation.
Findings – We find that when users classify crowdsourced tasks voluntarily it does not matter how users are classifying as long as it is accurately. However, what does matter is why they are doing it particularly because of the complex processes that build relationships between users and the platform. We present a conceptual model to enable deeper understandings of how forms of social interaction and play are motivating users contributing to citizen science projects to participate in the online processes.
Practical implications – The findings of this paper provide practical implications for how citizen science, and also other crowdsourcing platforms, can engage with notions of play and gamification to motivate participation.
Originality/value – Using detailed examples of online content, we reveal how participants of the demonstrate aspects of ‘gamised’ behaviour. We argue that the exploration of gaming as well as play provides evidence that contributing to citizen science projects can be both utilitarian and hedonic.

Authors: Cox, J., Oh, E., Simmons, B., Lintott, C., Masters, K., Graham, G., Greenhill, A. and Holmes, K.
Title: How is success defined and measured in online citizen science?  A case study of Zooniverse projects
Download Link
Year of Publication: 2015
Publication Details: Computing in Science and Engineering (Special Issue on Citizen Science)
Abstract: While the literature highlights a wide variety of potential citizen science project outcomes, no prior studies have systematically assessed performance against a comprehensive set of criteria. Our study is the first to propose a novel framework for assessing citizen science projects against multiple dimensions of success. We apply this framework to a sample of projects forming part of the online Zooniverse platform and position these projects against a ‘success matrix’ measuring both contribution to science and public engagement levels relative to others in the sample. Our results indicate that better performing projects tend to be those which are more established, as well as those in the area of astronomy. Implications for citizen science practitioners include the need to consider the impact of core competencies on project performance, as well as the importance of relationships between the central organisation and science teams.

Authors: Greenhill, A., Holmes, K., Lintott, C., Simmons, B., Masters, K., Cox, J. and Graham, G.
Title: Playing with science: Gamised aspects of gamification found on the online citizen science project – Zooniverse
Download Link
Year of Publication: 2014
Publication Details: GAME-ON 2014: EUROSIS, pp. 15-24. ISBN 9789077381854
Abstract: This paper examines incidents of play, socialisation, fun and amusement to consider how these forms of social interaction relate to the serious gaming elements of the citizen science platform. Through an ethnographic study we reveal how participants of citizen science projects demonstrate aspects of ‘Gamised’ behaviour. ‘Gamised’ behaviour is defined as user generated play in a digital platform and contrasts to incidents of ‘gamification’ where a platform designer purposely embeds games into a computer platform. The paper therefore examines incidents of play, socialisation, fun and amusement and compares them with the serious gaming elements of the citizen science platform.


Authors: Cox, J., Oh, E., Simmons, B., Graham, G., Greenhill, A., Lintott, C., Masters, K. and Woodcock, J.
Title: Doing good online: An investigation into the characteristics and motivations of digital volunteers.
Download Link
Publication Details: SSRN Working Paper
Abstract: The rapid growth in Internet technology is making it possible to volunteer in online settings, with participants able to contribute directly to research-based activities supporting non-profit groups and charitable organisations. This study undertakes an investigation into the profile and motivations of contributors to these online volunteering projects. We specifically investigate volunteer activity and retention for the online crowdsourcing platform known as the Zooniverse, which is home to around thirty online volunteering projects. Through a survey undertaken with a representative sample of contributors and reconciling against records of actual voluntary activity, we are able to measure motivations against the Volunteer Functions Inventory (VFI) and explore relationships with observed levels of activity and retention. Our results show that a unique combination of ‘other’ and ‘self’ oriented motivation, specifically Protective & Enhancement, Values and particularly Understanding, associate significantly and positively with observed variations in volunteering activity and retention in an online setting.


Other References

Club Goods

Al-Nowaihi, A. Fraser, C.D. (2012). Does the public sector over-provide club goods? A general result. Economic Letters, 117 pp.397-400.

Buchanan, J.M. (1965). An Economic Theory of Clubs. Economica, 32(125) pp.1-14.

Conley, J. and Dix, M. (1999). Optimal and Equilibrium Membership in Clubs in the Presence of Spillovers. Journal of Urban Economics, 46, pp.215-229.

Economics of Volunteering

Andreoni, J. (1989). Giving with Impure Altruism: Applications to Charity and Ricardian Equivalence. Journal of Political Economy. 97(6), pp.1447-1458.

Andreoni, J. (1990). Impure Altruism and Donations to Public Goods: A Theory of Warm-Glow Giving. The Economic Journal, 100(401), pp.464-477.

Freeman, R.B. (1997). Working for nothing: The supply of volunteer labor. Journal of Labor Economics, 15(1), Part 2: S140-S166.

Meier, S. and Stutzer, A. (2008). Is Volunteering Rewarding in Itself? Economica, 75, pp. 39-59.

Mueller, M.W. (1975). Economic Determinants of Volunteer Work by Women. Signs, 1(2) pp.325-338.

Myatt, D. P. (2009). Evolution, Teamwork and Collective Action: Production Targets in the Private Provision of Public Goods. The Economic Journal, 119, pp.61-90.

Rege, M. (2004). Social Norms and Private Provision of Public Goods. Journal of Public Economic Theory, 6(1), pp.65-77.

Roy, K. and Ziemek, S. (2000). On the Economics of Volunteering. ZEF Discussion Papers on Development Policy, 26.

Yeung, A.B., (2004). An Intricate Triangle – Religiosity, Volunteering, and Social Capital: The European Perspective, the Case of Finland. Nonprofit and Voluntary Sector Quarterly, 33, 401-422.

Social Media

Kaplan, A, M. and Haenlein, M. (2010). Users of the world, unite! The challenges and opportunities of Social Media. Business Horizons, 53, pp.59-68.

Weinberg, B.D. and Pehlivan, E. (2011). Social spending: Managing the social media mix. Business Horizons, 54, pp.275-282.

(2012) A customer engagement cycle for Web 2.0: practical guide to social media and the consumer. Strategic Direction, 28(10), pp.6-8.


Franke, N., von Hippel, E. and Schreier, M. (2006). Finding Commercially Attractive User Innovations: A Test of Lead-User Theory. Journal of Product Innovation Management, 23, pp.301-315.

Franzoni, C. and Sauermann, H, (2014). Crowd science: The organization of scientific research in open collaborative projects. Research Policy, 43(1), pp. 1-20.

Raddick, J.M., Bracey, G., et al. (2013). Galaxy Zoo: Motivations of Citizen Scientists. Astronomy Education Review, 12(1).

Kaganer, E., Carmel, E., Hirscheim, R. and Olsen, T. (2013). Managing the Human Cloud. MIT Sloan Management Review, 54(2) pp.23-32.

von Hippel, E. and von Krogh, G. (2003). Open Source Software and the “Private-Collective” Innovation Model: Issues for Organization Science. Organization Science, 14(2) pp.209-223.

Whitla, P. (2009). Crowdsourcing and Its Application in Marketing Activities. Contemporary Management Research, 5(1), pp.15-28.

Wirtz, B. W., Schilke, O and Ullrich, S. (2010). Strategic Development of Business Models: Implications of the Web 2.0 for Creating Value on the Internet. Long Range Planning, 43, pp.272-290.

Books and Conference Papers

Cottman-Fields, M., Brereton, M. and Roe P. (2013). Virtual Birding: Extending an Environmental Pastime into the Virtual World for Citizen Science. CHI ’13: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems.

Eveleigh, A., Jennett, C. Lynn, S. and Cox, A.L. I want to be a Captain! I want to be a Captain!: Gamification in the Old Weather Citizen Science Project. Gamification 2013 Proceedings of the First International Conference on Gameful Design, Research, and Applications October 2-4, 2013 Stratford, Ontario, Canada. pp.79-82.

Iacovides, I., Jennet, C., Cornish-Trestrail, T. and Cox, A.L. (2013). Do Games Attract or Sustain Engagement in Citizen Science? A Study of Volunteer Motivations. CHI ’13: Extended Abstracts on Human Factors in Computing Systems. pp.1101-1106.

Salamon, L.M., Sokolowski, S.W. and Haddock, M.A. (2011). Measuring the Economic Value of Volunteerwork Globally: Concepts, Estimates, and a Roadmap to the Future. Annals of Public and Cooperative Economics, 82(3), pp.217-252.

Stanoevska-Slabeva, K. (2011). Enabled Innovation: Instruments and Methods of Internet-based Collaborative Innovation. Draft paper prepared for the 1st Berlin Symposium in Internet and Society, October 25 – 27, 2011.

Waterhouse, T.P. (2013). Pay by the Bit: An Information-Theoretic Metric for Collective Human Judgment.

Zittrain, J. (2008). The future of the Internet and How to Stop it. Yale University Press