Using machine vision to estimate fish length from images using regional convolutional neural networks

Graham G. Monkman, Kieran Hyder, Michel J. Kaiser, Franck P. Vidal

Research output: Contribution to journalArticle

Abstract

An image can encode date, time, location and camera information as metadata and implicitly encodes species information and data on human activity, for example the size distribution of fish removals. Accurate length estimates can be made from images using a fiducial marker; however, their manual extraction is time-consuming and estimates are inaccurate without control over the imaging system. This article presents a methodology which uses machine vision to estimate the total length (TL) of a fusiform fish (European sea bass). Three regional convolutional neural networks (R-CNN) were trained from public images. Images of European sea bass were captured with a fiducial marker with three non-specialist cameras. Images were undistorted using the intrinsic lens properties calculated for the camera in OpenCV; then TL was estimated using machine vision (MV) to detect both marker and subject. MV performance was evaluated for the three R-CNNs under downsampling and rotation of the captured images. Each R-CNN accurately predicted the location of fish in test images (mean intersection over union, 93%) and estimates of TL were accurate, with percent mean bias error (%MBE [95% CIs]) = 2.2% [2.0, 2.4]). Detections were robust to horizontal flipping and downsampling. TL estimates at absolute image rotations >20° became increasingly inaccurate but %MBE [95% CIs] was reduced to −0.1% [−0.2, 0.1] using machine learning to remove outliers and model bias. Machine vision can classify and derive measurements of species from images without specialist equipment. It is anticipated that ecological researchers and managers will make increasing use of MV where image data are collected (e.g. in remote electronic monitoring, virtual observations, wildlife surveys and morphometrics) and MV will be of particular utility where large volumes of image data are gathered.

Original languageEnglish
Pages (from-to)2045-2056
Number of pages12
JournalMethods in Ecology and Evolution
Volume10
Issue number12
Early online date10 Aug 2019
DOIs
Publication statusPublished - Dec 2019

Fingerprint

computer vision
neural networks
fish
cameras
Dicentrarchus labrax
artificial intelligence
Lens
electronics
wildlife
managers
date (time)
researchers
image analysis
metadata
monitoring
outlier
human activity
testing

Keywords

  • European sea bass
  • convolutional neural networks
  • fiducial marker
  • fish length
  • machine vision
  • photogrammetry
  • regional convolutional neural network
  • videogrammetry

ASJC Scopus subject areas

  • Ecology, Evolution, Behavior and Systematics
  • Ecological Modelling

Cite this

Monkman, Graham G. ; Hyder, Kieran ; Kaiser, Michel J. ; Vidal, Franck P. / Using machine vision to estimate fish length from images using regional convolutional neural networks. In: Methods in Ecology and Evolution. 2019 ; Vol. 10, No. 12. pp. 2045-2056.
@article{2ec5dfb960cf4ac880d7c94deb0e5664,
title = "Using machine vision to estimate fish length from images using regional convolutional neural networks",
abstract = "An image can encode date, time, location and camera information as metadata and implicitly encodes species information and data on human activity, for example the size distribution of fish removals. Accurate length estimates can be made from images using a fiducial marker; however, their manual extraction is time-consuming and estimates are inaccurate without control over the imaging system. This article presents a methodology which uses machine vision to estimate the total length (TL) of a fusiform fish (European sea bass). Three regional convolutional neural networks (R-CNN) were trained from public images. Images of European sea bass were captured with a fiducial marker with three non-specialist cameras. Images were undistorted using the intrinsic lens properties calculated for the camera in OpenCV; then TL was estimated using machine vision (MV) to detect both marker and subject. MV performance was evaluated for the three R-CNNs under downsampling and rotation of the captured images. Each R-CNN accurately predicted the location of fish in test images (mean intersection over union, 93{\%}) and estimates of TL were accurate, with percent mean bias error ({\%}MBE [95{\%} CIs]) = 2.2{\%} [2.0, 2.4]). Detections were robust to horizontal flipping and downsampling. TL estimates at absolute image rotations >20° became increasingly inaccurate but {\%}MBE [95{\%} CIs] was reduced to −0.1{\%} [−0.2, 0.1] using machine learning to remove outliers and model bias. Machine vision can classify and derive measurements of species from images without specialist equipment. It is anticipated that ecological researchers and managers will make increasing use of MV where image data are collected (e.g. in remote electronic monitoring, virtual observations, wildlife surveys and morphometrics) and MV will be of particular utility where large volumes of image data are gathered.",
keywords = "European sea bass, convolutional neural networks, fiducial marker, fish length, machine vision, photogrammetry, regional convolutional neural network, videogrammetry",
author = "Monkman, {Graham G.} and Kieran Hyder and Kaiser, {Michel J.} and Vidal, {Franck P.}",
year = "2019",
month = "12",
doi = "10.1111/2041-210X.13282",
language = "English",
volume = "10",
pages = "2045--2056",
journal = "Methods in Ecology and Evolution",
issn = "2041-210X",
publisher = "John Wiley and Sons Inc.",
number = "12",

}

Using machine vision to estimate fish length from images using regional convolutional neural networks. / Monkman, Graham G.; Hyder, Kieran; Kaiser, Michel J.; Vidal, Franck P.

In: Methods in Ecology and Evolution, Vol. 10, No. 12, 12.2019, p. 2045-2056.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Using machine vision to estimate fish length from images using regional convolutional neural networks

AU - Monkman, Graham G.

AU - Hyder, Kieran

AU - Kaiser, Michel J.

AU - Vidal, Franck P.

PY - 2019/12

Y1 - 2019/12

N2 - An image can encode date, time, location and camera information as metadata and implicitly encodes species information and data on human activity, for example the size distribution of fish removals. Accurate length estimates can be made from images using a fiducial marker; however, their manual extraction is time-consuming and estimates are inaccurate without control over the imaging system. This article presents a methodology which uses machine vision to estimate the total length (TL) of a fusiform fish (European sea bass). Three regional convolutional neural networks (R-CNN) were trained from public images. Images of European sea bass were captured with a fiducial marker with three non-specialist cameras. Images were undistorted using the intrinsic lens properties calculated for the camera in OpenCV; then TL was estimated using machine vision (MV) to detect both marker and subject. MV performance was evaluated for the three R-CNNs under downsampling and rotation of the captured images. Each R-CNN accurately predicted the location of fish in test images (mean intersection over union, 93%) and estimates of TL were accurate, with percent mean bias error (%MBE [95% CIs]) = 2.2% [2.0, 2.4]). Detections were robust to horizontal flipping and downsampling. TL estimates at absolute image rotations >20° became increasingly inaccurate but %MBE [95% CIs] was reduced to −0.1% [−0.2, 0.1] using machine learning to remove outliers and model bias. Machine vision can classify and derive measurements of species from images without specialist equipment. It is anticipated that ecological researchers and managers will make increasing use of MV where image data are collected (e.g. in remote electronic monitoring, virtual observations, wildlife surveys and morphometrics) and MV will be of particular utility where large volumes of image data are gathered.

AB - An image can encode date, time, location and camera information as metadata and implicitly encodes species information and data on human activity, for example the size distribution of fish removals. Accurate length estimates can be made from images using a fiducial marker; however, their manual extraction is time-consuming and estimates are inaccurate without control over the imaging system. This article presents a methodology which uses machine vision to estimate the total length (TL) of a fusiform fish (European sea bass). Three regional convolutional neural networks (R-CNN) were trained from public images. Images of European sea bass were captured with a fiducial marker with three non-specialist cameras. Images were undistorted using the intrinsic lens properties calculated for the camera in OpenCV; then TL was estimated using machine vision (MV) to detect both marker and subject. MV performance was evaluated for the three R-CNNs under downsampling and rotation of the captured images. Each R-CNN accurately predicted the location of fish in test images (mean intersection over union, 93%) and estimates of TL were accurate, with percent mean bias error (%MBE [95% CIs]) = 2.2% [2.0, 2.4]). Detections were robust to horizontal flipping and downsampling. TL estimates at absolute image rotations >20° became increasingly inaccurate but %MBE [95% CIs] was reduced to −0.1% [−0.2, 0.1] using machine learning to remove outliers and model bias. Machine vision can classify and derive measurements of species from images without specialist equipment. It is anticipated that ecological researchers and managers will make increasing use of MV where image data are collected (e.g. in remote electronic monitoring, virtual observations, wildlife surveys and morphometrics) and MV will be of particular utility where large volumes of image data are gathered.

KW - European sea bass

KW - convolutional neural networks

KW - fiducial marker

KW - fish length

KW - machine vision

KW - photogrammetry

KW - regional convolutional neural network

KW - videogrammetry

UR - http://www.scopus.com/inward/record.url?scp=85074749294&partnerID=8YFLogxK

U2 - 10.1111/2041-210X.13282

DO - 10.1111/2041-210X.13282

M3 - Article

VL - 10

SP - 2045

EP - 2056

JO - Methods in Ecology and Evolution

JF - Methods in Ecology and Evolution

SN - 2041-210X

IS - 12

ER -