Julie Mugford (pictured), a PhD student at the University of Canterbury and Chair of Te Pūnaha Matatini Whānau, is researching and developing statistical tools to improve the accuracy of classification-based crowdsourcing, aka citizen science.
Citizen science is the involvement of volunteers in helping scientists collect and analyse information, and Julie’s research aims to measure the accuracy of users and to develop efficient ways to improve the overall accuracy of such data.
Typically, classification-based citizen science projects ask multiple participants to identify each object and consensus methods are used to decide the classification of the object. Commonly, simple consensus methods – for example, majority vote – are used. However, majority vote weights the contributions from each participant equally but the participants may vary in accuracy with which they can label objects.
“Our approach is to use Bayesian statistics to estimate users’ accuracies at identifying objects and include these accuracies in the classification process,” explained Julie.
“Although this approach complicates the classification process compared to a simple majority vote rule, it improves the accuracy of the classification decisions and provides more robust measures of classification certainty.”
How Kiwis are helping to answer important scientific questions
Citizen scientists can encompass a wide range of members within our society – from school children to trained scientists – who participate in a variety of research projects. These projects are often set up and managed by professional scientists, and specifically designed to give volunteers a role. For example, sharing and classifying bird and other observations of nature, classifying land types in satellite images of Earth, or classifying galaxies.
The popularity of citizen science projects has risen enormously in the last two decades, providing researchers with access to data from a large range of locations at unprecedented frequencies with minimal costs. This has become increasingly important as costly expert resources struggle to match the effort required to answer scientific questions. However, there is ongoing debate on the usefulness and accuracy of citizen science data as it may be prone to greater variability due to differences in volunteer’s skills.
“Motivated by the vision of Biosecurity New Zealand to have a biosecurity team of 4.7 million [New Zealand’s resident population], we have initially focused on improving the accuracy of classification-based citizen science projects that could be used as a tool to monitor invasive pests in New Zealand,” said Julie.
Biosecurity New Zealand, a part of the Ministry for Primary Industries, has set out a vision for 2025, and one of its five strategic directions aims is to make all New Zealanders aware of the importance of biosecurity and to get them involved in pest and disease management. It hopes to encourage a collective effort across the country – in which ‘every New Zealander becomes a biosecurity risk manager and every business manages their own biosecurity risk’.