Evaluating a tandem human-machine approach to labelling of wildlife in remote camera monitoring
Dates
Publication Date
2023-08-30
Start Date
2022-01-01
End Date
2022-09-30
Citation
Clarfeld, L., Donovan, T., Siren, A., Mulhall, B., Bernier, E., Farrell, J., Lunde, G., Hardy, N., Abrams, R., Staats, S., and McLellan, S., 2023, Evaluating a tandem human-machine approach to labelling of wildlife in remote camera monitoring: U.S. Geological Survey data release, https://doi.org/10.5066/P9FGUQEZ.
Summary
Remote cameras (“trail cameras”) are a popular tool for non-invasive, continuous wildlife monitoring, and as they become more prevalent in wildlife research, machine learning (ML) is increasingly used to automate or accelerate the labor-intensive process of labelling (i.e., tagging) photos. Human-machine hybrid tagging approaches have been shown to greatly increase tagging efficiency (i.e., time to tag a single image). However, those potential increases hinge on the extent to which an ML model makes correct vs. incorrect predictions. We performed an experiment using a ML model that produces bounding boxes around animals, people, and vehicles in remote camera imagery (MegaDetector), to consider the impact of a ML model’s performance [...]
Summary
Remote cameras (“trail cameras”) are a popular tool for non-invasive, continuous wildlife monitoring, and as they become more prevalent in wildlife research, machine learning (ML) is increasingly used to automate or accelerate the labor-intensive process of labelling (i.e., tagging) photos. Human-machine hybrid tagging approaches have been shown to greatly increase tagging efficiency (i.e., time to tag a single image). However, those potential increases hinge on the extent to which an ML model makes correct vs. incorrect predictions. We performed an experiment using a ML model that produces bounding boxes around animals, people, and vehicles in remote camera imagery (MegaDetector), to consider the impact of a ML model’s performance on its ability to accelerate human labeling. Six participants tagged trail camera images collected from 12 sites in Vermont and Maine, USA (January-September 2022) using three tagging methods (one with ML bounding box assistance and two without assistance).
Click on title to download individual files attached to this item.
Tandem_ML_evaluation.xml Original FGDC Metadata
View
20.08 KB
application/fgdc+xml
Tandem_ML_evaluation_paper_data.csv
293.85 KB
text/csv
Purpose
Remote cameras (“trail cameras”) are a popular tool for non-invasive, continuous wildlife monitoring, and as they become more prevalent in wildlife research, machine learning (ML) is increasingly used to automate or accelerate the labor-intensive process of labelling (i.e., tagging) photos. Human-machine hybrid tagging approaches have been shown to greatly increase tagging efficiency (i.e., time to tag a single image). However, those potential increases hinge on the extent to which an ML model makes correct vs. incorrect predictions. We performed an experiment using a ML model that produces bounding boxes around animals, people, and vehicles in remote camera imagery (MegaDetector), to consider the impact of a ML model’s performance on its ability to accelerate human labeling. Six participants tagged trail camera images collected from 12 sites in Vermont and Maine, USA (January-September 2022) using three tagging methods (one with ML bounding box assistance and two without assistance).
