ReefNet: Expert-informed Benchmarks and Machine Learning Systems for Coral Reef 2D /3D Visual Understanding

Project overview

Over the last two decades, ecological monitoring of coral reefs has undergone rapid digitalization. Traditional in-situ surveys were initially substituted with photographic transects and more recently with full 3-dimensional digital twins created through Structure-from-Motion.

Underwater imagery provides a permanent record of the benthic communities. However, their content needs to be classified to access the wealth of ecological information contained within them. This is an extremely resource-intensive procedure, which thus requires automation.

AI has recently revolutionized image analysis, but its application in the marine environment is lagging behind due to a lack of 2D/3D training data. The proposed project aims to address this shortfall in the context of coral reef benthos by creating a benchmark of densely annotated 2D/3D imagery as well as machine learning systems that facilitate dense visual understanding of coral reefs.

Special emphasis will be placed on the creation of a taxonomically informed labeling scheme, which we believe is essential to accurately measure AI progress towards accelerating marine ecology research. Based on this dataset, we will create AI models for 3D coral reconstruction and segmentation to help build digital twins of the underwater environment.

This project will develop AI-based methods for coral identification, coral reef 2D/3D segmentation, and coral reef 3D reconstruction. AI methods will play an important role in three aspects: 1) this project aims to build a semantic-aware hierarchy classification method for taxonomic coral identification based on large vision-language models; 2) build a Segment Anything Model (SAM)--based model for coral segmentation; 3) build an accurate and robust coral reef 3D reconstruction methods by adapting the recent advances in Neural Radiance Fields (NeRF) and 3D Gaussian Splatting.

This project addresses biodiversity by creating an expert-informed dataset and AI models to classify coral species at fine taxonomic levels, enabling precise assessments of coral biodiversity and dynamics. Automated identification and segmentation of coral genera, along with functional trait mapping, will yield critical insights into community composition, species interactions, and ecosystem health.

Additionally, 3D coral reef reconstructions will allow researchers to evaluate structural complexity, advancing our understanding of how biodiversity supports habitat provision and resilience to environmental stressors in vulnerable reef ecosystems.

Training Opportunities

A comprehensive training programme will be provided, comprising training both in applied AI and biodiversity, and transferable professional and research skills. The project includes a placement with an AI-INTERVENE project partner of between 3-18 months in duration. The student will present at national and international conferences, placing the student at the forefront of the discipline, leading to excellent future employment opportunities.

Student profile

This project would be suitable for students with a degree in computer science, remote sensing, marine biology, or a closely related field. Ideally candidates should have experience in machine learning, computer vision, or ecological data analysis, with strong programming skills (Python preferred).

Familiarity with deep learning frameworks (e.g., PyTorch or TensorFlow) and a background in marine ecology, biodiversity assessment, or 3D modelling would be advantageous. This interdisciplinary project is well-suited for students interested in applying advanced AI.