Object Detection in Paddy Field for Robotic Combine Harvester Based on Semantic Segmentation

概要

For decades, agricultural vehicles such as robotic tractor and robotic combine harvester have
been utilizing commercially available auto-steering and auto-driving systems for automatic and
efficient operations. However, human supervision is still required for safe operations. To
achieve fully autonomous farming, sensor technologies need to match or surpass human
performance to detect all objects. Additionally, detection algorithms must operate in real-time
to ensure the safety and efficient operations of the robotic vehicles. In this thesis, a semantic
segmentation (SS) method was applied to detect all significant objects in paddy fields for
robotic combine harvester.
In Chapter 1, the background of the research was introduced at first, then the research related
to field objects detection was summarized. And then, the objective and overview of this
research were described.
Chapter 2 introduced the experiment apparatus used in this thesis, including the Kubota
WRH1200A, a commercialized robotic combine harvester equipped with an RTK-GNSS for
navigation. Additionally, two RGB cameras and one depth camera installed on the harvester's
cabin roof were introduced for obtaining paddy field images.
Chapter 3 described the utilization of a deep learning-based SS method, image cascade
network (ICNet), for detecting objects in paddy fields. Six ICNet models and one fully
convolution networks (FCN) model were developed for training and testing. The results
showed that ICNet-VGG11 achieved the best performance in segmenting paddy field images,
with high accuracy in pixel, class mean accuracy, and mean intersection over union. However,
ridge detection was only successful when the harvester was close to the ridge, and the
segmentation of unharvested and lodging areas was unstable. Nevertheless, the best model
successfully detected lodging existence with high accuracy, which is crucial for the harvester's
operation. Overall, the study concluded that the SS method effectively detected harvested rice,
unharvested rice, lodging rice, humans, and paddy field ridges for the robotic combine
harvester, despite the slow prediction speed and low segmentation accuracy in the lodging area. ...