Background
Forecasting renewable-linked electricity demand is challenging because energy systems combine long-term trends and seasonality with short-term variation driven by weather, demand, and changing generation patterns. These forecasts also matter beyond predictive accuracy: they inform planning and operational decisions in systems where uncertainty cannot be eliminated.
Most studies analyse electricity consumption or renewable generation separately. This project instead examines Green Energy Demand (GED), a measure derived from the share of renewable generation in total electricity production and overall electricity consumption.
I constructed a daily-resolution UK GED dataset spanning 2000–2024, integrating:
- Electricity consumption and renewable generation
- Weather features including temperature, wind, solar radiation, and daylight
- Electricity price and derived temporal variables
Because observed solar-generation data were unavailable for the earlier part of the study period, pre-2010 solar generation was reconstructed using a ridge-regression proxy.
Research Goals
- Construct a daily GED dataset from heterogeneous energy, weather, and market data.
- Compare statistical and neural forecasting models under a consistent chronological evaluation framework.
- Examine how different model classes respond to seasonality, volatility, and changing data conditions.
- Identify model-specific strengths and failure modes relevant to downstream planning and decision support.
Methodology
ARIMA: Univariate statistical baseline for modelling temporal dependence in GED.
SARIMAX: Extended the statistical model with exogenous weather, demand, and market variables.
SARIMAX + GARCH: Modelled conditional heteroskedasticity in SARIMAX residuals to examine whether explicit volatility modelling improved forecasts.
Baseline LSTM: Recurrent neural-network baseline using recent observations to learn nonlinear temporal dependencies.
Fourier-Based Dual-Branch LSTM with Attention:
- Trend branch: Processes longer-term structure represented through Fourier-based trend features.
- Residual branch: Processes shorter-term weather, demand, and market-related variation.
- Attention fusion: Combines information from the two branches before producing the final forecast.
The experimental pipeline was evaluated chronologically to preserve the temporal structure of the forecasting problem. I subsequently revisited the preprocessing and evaluation procedure, identified sources of temporal data leakage, and re-ran the experiments under leakage-free conditions.
Results
The corrected experiments changed an important conclusion of the original analysis.
- ARIMA: Provided a useful statistical baseline but struggled with the more complex seasonal and exogenous structure of GED.
- SARIMAX and SARIMAX + GARCH: Remained highly competitive after leakage correction, demonstrating the value of explicitly modelling temporal structure, external predictors, and volatility.
- Baseline LSTM: Captured some nonlinear temporal behaviour but did not consistently outperform the statistical approaches over the long forecasting horizon.
- Fourier-Based Dual-Branch LSTM with Attention: Captured complementary trend and residual information, but its performance advantage was substantially smaller under the corrected evaluation than suggested by the initial experiments.
Rather than identifying a universally superior model, the analysis revealed that different forecasting approaches fail in different ways as system and data conditions change.
This shifted the central question from which model has the highest average predictive score? to when a model can be trusted and how its reliability changes across conditions.
Contributions
- Constructed a daily UK Green Energy Demand dataset spanning 2000–2024 from heterogeneous energy, weather, and market sources.
- Reconstructed unavailable historical solar-generation values using a ridge-regression proxy, achieving R² = 0.70, MAE = 3.40 GWh, and RMSE = 4.33 GWh.
- Benchmarked ARIMA, SARIMAX, SARIMAX–GARCH, and LSTM-based models within a chronological forecasting framework.
- Developed a Dual-Branch LSTM combining Fourier-based trend decomposition with residual modelling and attention-based fusion.
- Used leakage correction and comparative error analysis to examine forecast reliability and model-specific failure modes, rather than relying only on aggregate predictive performance.
Impact
Forecasts in energy systems ultimately support decisions about planning, renewable integration, scheduling, and resource allocation. In these settings, a model with strong average predictive accuracy may still be unsuitable if its errors become unreliable under particular weather patterns, system states, or periods of distributional change.
This project therefore became an early foundation for my broader research interest in AI-enabled decision-making under uncertainty: understanding not only how to generate accurate predictions, but also how model reliability, changing conditions, and uncertainty should influence the decisions built on top of them.