Publication: GANDALF: A LLM-based approach to map bark beetle outbreaks in semantic stories of Sentinel-2 images

14 May 2025

Huge spruce forest areas have been damaged by massive bark beetle outbreaks across Europe during the past few years. Forest health management requires large-scale inventory of bark beetle outbreaks to plan actions for promptly mitigating forest tree dieback. Deep learning techniques have recently achieved amazing results in imagery semantic segmentation tasks by dominating the recent research for mapping bark beetle outbreaks in Sentinel-2 images of forest areas. In addition, due to the impressive performance of Large Language Models (LLMs) in natural language understanding and generation tasks, LLMs have started attracting attention in multiple fields.

In this work, SWIFTT partners from the University of Bari Aldo Moro describe GANDALF, an approach that leverages the potential of LLMs for mapping bark beetle outbreaks in Sentinel-2 images of forest areas. The authors take advantage of the rich context of textual data to transform Sentinel-2 images in smart data ready for boosting accurate semantic segmentation modeling. They use a foundation LLM model to account for the text encoding of the spectral-spatial imagery context information and fine-tune the LLM model to perform the semantic segmentation of forest images and use the Integrated Gradients (IG) algorithm to explain how each spectral-spatial information influences the bark beetle outbreak detection.

Read the paper in the link below.