Table of Contents
Highlights
- ML and AI improve harvest forecasting, yield prediction, and pest management using satellite, soil, and weather data.
- Challenges include data quality, transferability, uncertainty, and limited access for small farmers.
- Future directions emphasize explainable AI, climate resilience, and smallholder-friendly forecasting tools.
Introduction
Agriculture has long relied on weather, soil, and the unpredictable interplay of pests and pathogens. For centuries, farmers have relied on their experience, seasonal indicators, and trial-and-error to choose when to sow, how much fertiliser to apply, and when to harvest. Now in the 21st century, machine learning (ML) is revolutionising agriculture by increasing the capability of forecasting based on data.
These forecasts are not just about forecasting yields at harvest, but becoming powerful tools for forecasting disease and pest outbreaks, optimal harvest windows, and even helping smallholders to make tactical decisions days and weeks ahead of actual decision times. This feature will look at how ML is used for harvest forecasting and related agricultural forecasting and phenomena, what techniques are leading, what problems remain, and what is potentially to come.
How Forecasting Works: Data, Models, and Use Cases
Data sources
To forecast crop yields, or yield-related phenomena, an ML system draws from several kinds of data:
Remote sensing/satellite imagery: vegetative indices such as NDVI (Normalized Difference Vegetation Index), EVI, and spectral bands that show plant health. These could be considered spatially broad coverage, with temporal revisit times.
Soil data: soil moisture, nutrient levels, texture, and historical yields.
Weather/climate data (rain, temperature, humidity, solar radiation), both historical and forecasts.
Management practices: Data on when to plant, apply fertilizer, irrigate, and conduct pest management.
Ground truth data: Yield estimations, pest trap counts, phenology (when things flower, maturity) to calibrate the models.
ML and DL models
Different ML and DL methods are used, often in an ensemble:
Traditional ML: Random forests, support vector machines (SVM), and gradient boosting (e.g., XGBoost, LightGBM) for tabular datasets.
Deep Learning: CNNs (for image datasets, or time series environmental data), RNNs/LSTMs (for time series weather, phenology, etc.), and hybrid models (e.g., CNN + LSTM).
Ensemble methods: Combine multiple models to reduce variance and increase robustness.
Feature engineering & explainable ML: Interpretability tools (e.g., SHAP values) to understand what features have the most importance. This relates very closely to building trust and understanding insight.
Real-time/online learning: Some systems will modify their forecast during the season as observations/events/input data become available (weather, satellite observations, in-field sensors).
Use cases
Yield prediction: Yield prediction is predicting how much crop will actually be in season. Governments are important for local governments, supply chains, cooperatives, insurance companies, and agribusinesses. For example, one study of 115 articles showed many used ML (40-70% of the studies) for yield prediction across a number of data types across crops and geographies.
Prediction of pest or disease outbreaks: Utilizing weather, plant development, pest traps, and satellite data to predict when and where pest or disease outbreaks may occur, enabling the most timely interventions.
Optimization of harvest timing: Determining the optimal time to harvest for maximum yield, best quality, or to avert loss due to weather or over-ripening. Input or resource allocations: Advising on how much fertilizer, pesticide, water to use, or investing in labour or machinery, based on forecasts.
Recent Developments and Empirical Studies
Here are some recent empirical or view studies (2023 – 2025) that provide evidence of what works and what limitations still exist: A review of AI-based yield prediction across multiple crops and countries shows that deep learning models (CNNs, LSTMs, DNNs) are often performing better than simpler ML tools, particularly in situations where large and heterogeneous data (soil, environment, remote sensing data) have been used.
A Canadian study compared ensemble ML methods (AdaBoost, Gradient Boosting, etc.) across crop types and found that ensemble methods perform better than single model types, especially when the climate is variable. A study predicting yields globally (across 37 developing countries and spanning 27 years of data) found that structure-based forecasting of yield, incorporating factors such as insecticide use, rainfall, and temperature, was highly accurate (R² ~ 0.94) using Random Forest methods across regions.
Another study: Utilizing a CNN regressor to predict winter wheat yield in Germany using weather, soil, and phenological data, CNNs resulted in a 7-14% lower root mean squared error (RMSE) and mean absolute error (MAE) compared to baseline models, and improved correlation with actual yield.
Additionally, using SHAP and force plots allowed the researchers to interpret which features (i.e., soil moisture, radiation, wind speed) mattered most during which weeks.
Challenges
Despite the advances made in this area, and some additional ones, it will remain a challenge to demonstrate these forecasts reliably in practice.
1. Data quality and spatial resolution
Satellite data often has cloud cover and is limited in spatial resolution (may not capture small, fragmented farms). Soil/management data is commonly missing or inaccurate in small fields or small spatial scales. Ground truth (actual yield) is expensive to collect, and any inaccuracies persist in the model error.
Models trained in one region or climate regime often have poor performance in another region or climate regime, without recalibrating. This will likely require local calibration (i.e., soil, cultivar, weather patterns).
3. Uncertainty and interpretability
Farmers (and policy makers) are not just interested in forecasts, but also in how much confidence is associated with each prediction, as well as which factors are contributing to the predicted yield. Without explainability, black box models have a lower degree of trust.
4. Temporal dynamics & early warning
Throughout the season, forecasts must be updated as new meteorological or satellite data is available. An early-season forecast can look very different if there are late rains or pest outbreaks.
5. Infrastructure and access
Smallholder farmers may not have access to weather stations, high-quality internet, and sensors. The translation of forecasts into actionable information is another area of relative neglect.
6. Economic and risk factor
Even with a good forecast, if farmers do not have access to the labor, cash, or inputs required to respond, the forecast will not help. There are also risks associated with over-reliance on forecasts that turn out to be wrong due to events that cannot be predicted (storms, extreme weather).
What Works: Design Principles & Best Practices
From both the literature and trials in the field, there are some types of practices that are likely to positively impact the utility and adoption of ML forecasting systems:
Hybrid models: Using process-based crop models (that capture biological/phenological understanding) linking with ML models that can capture environmental variabilities tends to be better than using a pure ML or a pure crop model.Frequent updating and real-time data integration: Routine, regular ingestion of updated satellite/remote sensing data and weather data generates an opportunity to increase the refining of the forecast over time.
Explainability tools: Use SHAP, feature importance, time-based sensitivity analysis, etc., to demonstrate which inputs are driving the forecast.
User-focused interfaces: Present predictions in ways farmers can comprehend (SMS messages, mobile dashboards, visuals), providing recommendations on actions to take with different forecast situations.
Model validation in the field: Beyond just the statistical validation of a model, it will include field trials in real production systems reflecting economic impacts, whether it be yield or lower labor costs.
Future Directions & Emerging Ideas
Some of the highlighting, or promising, emerging directions include: Explainable AI (XAI) in agriculture: ensuring a farmer understands why a forecast was produced will ultimately have higher trust and therefore adoption.
Multimodal data fusion: Multiple sensors (drones, IoT soil sensors), more satellite bands, and combining biological measurements (e.g., chlorophyll fluorescence) to monitor stress long before you see damage. A group led by Ying Sun (Cornell) has been looking at solar-induced chlorophyll fluorescence (SIF) remote sensing as a cost-effective indirect measurement of plant health, indicative of estimated yield. Improving forecasting of insect and disease outbreaks more precisely, perhaps using ML with pest trap data, weather, phenology, and satellite maps as indicators of pest and disease levels, is possible in a cropping system.
Smallholder-friendly tools: Lightweight models, reduction of large and live data collection requirements, offline, smartphone models, participatory design with farmers so that some aspect of data collection and or use is sustainable.
Climate resilience: As climate change leads to increased unpredictable weather patterns, forecasts must account for extremes on both sides of the average condition, rather than just the average. If ML/AI-focused modeling can adapt similarly from year to year using changing baseline climatic conditions, it could, with the likelihood, increase resilience to climate change during commodity losses.
Ramifications and ethical / policy considerations
Data access: Governments and NGOs fund and open> weather, soil, and satellite data to the public and private sectors.
Equity: Ensuring small holders, marginal farms, and underserved places benefit not just large commercial farms.
Risk sharing and insurance: Forecasting can better inform crop insurance and risk mitigation strategies; however, it must be fair how losses or predictions are applied.
Transparency and false confidence: Models ‘ overconfidence can lead to misplacement of resources, and forecasts should share uncertainty.
Environmental impact: Better predictions will lead to lower usage of pesticides and fertilizers, resulting in better sustainability. However, if predictions are wrong, or the prediction error leads to inappropriate or misapplication, the net result can be highly negative for the environment.
Conclusion
Machine learning is becoming an increasingly powerful tool for agriculture – helping farmers and others better anticipate yields, pest management, timing of harvest, and decision making. Science has matured, with better models and better data sources, and more robust methods. But there is still a long way to go between academic or pilot testing results and usable systems that are trusted, usable, and robust to the variability of the environment. The next frontier is in exploring new sources of data, developing adaptive models (continual learning), ensuring that the models are interpretable, and embedding forecasts into farmer-centric workflows.
