IoT-aware Lightweight Deep Learning for Farm Insect Counting

Keywords

TinyML, Neural Architecture Search, Embedded systems, Image processing, Sound processing, Digital agriculture

Suitable Departments

Computer Science & Engineering, Electronics Engineering

Summary

Herbivorous insects on a farm can cause significant economic losses through several pathways. Certain symbiotic carnivorous insects prey on the harmful ones and thus, mitigate the impact of presence of harmful insects. As a matter of standard practise today, a greenhouse farmer follows a pre-set schedule of insecticide application. This is non-responsive to actual insect populations in the field. It leads to insecticide overuse which raises operating cost, lowers soil fertility, leaching and poisoning of waterbodies and, high residual insecticide in the food chain. A device for automatically classifying and counting insects helps towards targeted selection and metering of insecticides or simply, airdropping of beneficial insects.

The literature has highlighted the need for multiple types of sensors to distinguish between farm insects. These include visible band camera, microphone array, near-IR wingbeat count sensor and thermal IR camera. For economic feasibility and avoiding modulating insect behavior by its presence, the IoT device must be small and dissipate very low power. Such devices are unfortunately also severely constrained in computational resources in their ability to clean noise, perform sensor fusion, pre-process, run digital signal processing (DSP) primitives, tensor calculations or run deep learning (DL)-based classifiers. A cloud accelerator could be used for one-off tasks such as training or recalibration. But all other tasks, including classification, must be executed on the resource constrained IoT else to conserve energy, server costs and data transfer charges. This calls for developing efficient hierarchical, parallelized algorithms for DL-based classification.

A DL-based classifier is likely to have of the order of hundred million parameters. A feasible IoT device is too computational resource constrained to train or implement it. Therefore, in addition to designing an appropriate hierarchical decision methodology and developing hardware/firmware architecture aware efficient implementations, one must obtain a ‘tiny DLN’ from the ‘base’ DLN which, while is somewhat less accurate but significantly smaller.

Programming Skills Needed

Python/Matlab C++

Hardware Skills Needed

Atleast a semester with Arduino/Raspberry Pi or a good understanding of MCU/DSP/GPU/TPU and IoT architectures

Other Skills

Parallel Computing, Algorithm Design, Machine Learning, Image Processing, Acoustic Signal Processing

Partners

India :Dr. Anantha Narayanan V. & Dr. N. Anandaraja
Outside : Dr. Julien Malard-Adam

Funding

Amrita Vishwa Vidyapeetham provides stipends and teaching assistantships to selected candidates. Once you join, you could leverage existing proposals within the team to apply for additional corporate or government funding.

Time Period

3 ½ to 4 ½ years based on full-time committed

Sample References

• Abhishek Ramdas Nair, Pallab Kumar Nath, Shantanu Chakrabartty et. al., “Multiplierless in-filter computing for tinyML platforms”, Apr 2023 pre-print.
• Andrew Sabot, Vikas Nates, H. T. Kung et. al., “mema runtime framework: minimizing external memory”, Apr 2023 pre-print.
• Clark F. Olson, “Parallel algorithms for hierarchical clustering”, Parallel Computing, vol. 21, no. 8, pp. 1313-1325, Aug 1995.
• Colby Banbury, Chuteng Zhou, Igor Fedorov et. al., “Micronets: neural network architectures for deploying TinyML applications on commodity microcontrollers”, Conference on Machine Learning and Systems, Apr 2021.
• Davis Blalock, Jose Javier Gonzalez Ortiz, Jonathan Frankle et. al., “What is the state of neural network pruning?”, Conference on Machine Learning and Systems, Mar 2020.
• Jun Fang, Ali Shafiee, Hamzah Abdel-Aziz et. al, “Post-training piecewise linear quantization for deep neural networks”, European Conference on Computer Vision, Oct 2020.
• Jutta Eymann, Jérôme Degreef, Christopher Hauser, et. al., “Manual on field recording techniques and protocols for all Taxa Biodiversity Inventories and Monitoring”, ABC Taxa, Royal Belgian Institute of Natural Sciences, vol. 8, part 1, 2010.
• Yu-Qi Liu, Xin Du, Hui-Liang Shen et. al., “estimating generalized gaussian blur kernels for out-of-focus image deblurring”, IEEE Transactions on Circuits and Systems for Video Technology, vol. 31, no. 3, Mar 2021.
• Qianyun Lu and Boris Murmann, “enhancing the energy efficiency and robustness of tinyML computer vision using coarsely-quantized log-gradient input images”, ACM Transactions on Embedded Computing Systems, Apr 2023.
• Maxime Martineau, Donatello Conte, Romain Raveaux et. al., “A survey on image-based insect classification”, J. of Pattern Recognition, vol. 65, pp. 273-284, May 2017.
• Yanping Chen, Adena Why and Gustavo Batista et. al. “Flying insect classification with inexpensive sensors”, J. of Insect Behavior, vol. 27, pp. 657-677, 2014.
• Md Mohaimenuzzaman, Christoph Bergmeir, Ian West et. al., “Environmental sound classification on the edge: A pipeline for deep acoustic networks on extremely resource-constrained devices”, J. of Pattern Recognition, vol. 133, Jan 2023.
• Michele Preti, François Verheggen and Sergio Angeli, “Insect pest monitoring with camera‑equipped traps: strengths and limitations”, J. of Pest Science, vol. 94, Dec 2020.