This paper presents an experimental protocol developed for the design, performance estimation and comparison of underwater video classifer systems. Such systems have to be designed using application data that is small, sparse and extremely variable. The proposed protocol uses outlier rejection, data pairing, Bootstrap performance estimation and hypothesis testing to achieve a robust performance estimate and comparison between classifer designs. The protocol is demonstrated and assessed on an application experiment. The application involves the design of a classification system for the automated detection of trawling marks from mission video. Two systems are proposed using selective and geometric feature types and an ensemble classifier. The protocol robustly identifies differences between the two proposed system designs using error and discrimination rates. Overall the geometric feature system is chosen as the final system. The protocol was also compared with other performance estimates and found to have the closest match to actual test data performance. © 2005 IEEE.
|Title of host publication||Oceans 2005 - Europe|
|Number of pages||6|
|Publication status||Published - 2005|
|Event||Oceans 2005 - Europe - Brest, France|
Duration: 20 Jun 2005 → 23 Jun 2005
|Conference||Oceans 2005 - Europe|
|Period||20/06/05 → 23/06/05|