Beyond Single-Scale A Multi-Scale Approach to Semantic-Enhanced Crop Disease Recognition Through Image–Text Fusion

Accurate crop disease recognition is essential for precision agriculture, yet most existing systems rely solely on visual features, ignoring the rich semantic cues found in textual disease descriptions. This paper presents DiENT, a novel multimodal framework that fuses image and text representations for plant leaf disease classification across nine disease categories. Using Phi-3.5-Vision, we generate structured scientific descriptions (lesion morphology, color transitions, texture) for 4,499 training images. Two model variants are explored: Model 1 combines EfficientNet‑B3 with a general‑purpose SentenceTransformer (MiniLM), while Model 2 upgrades to EfficientNet‑B4 paired with BioBERT, a domain‑specific encoder pre‑trained on biomedical literature. Both variants share a cross‑attention module to align text semantics with spatial image regions, a multi‑scale activation module to highlight discriminative disease patterns, and a reasoning module for cross‑modal integration. For local front‑end deployment, BLIP is used for on‑device text generation. Grad‑CAM visualizations further enhance model interpretability. The proposed system offers a robust, explainable solution for agricultural monitoring applications.

Keywords: Crop disease recognition, multimodal fusion, image–text alignment, BioBERT, EfficientNet, cross‑attention, BLIP, precision agriculture