LLM advice shapes rewards during training, improving sample efficiency for autonomous cyber defense without LLM calls at deployment.
Overview
Ask the Expert summarizes challenging cyber-defense states, requests constrained recommendations from an LLM, and converts that advice into reward bonuses for PPO. Experiments on CAGE Challenge 1 and 2 compare learning efficiency against PPO and potential-based reward-shaping baselines.
Accepted at IEEE GLOBECOM 2026; proceedings forthcoming. The linked version is the public preprint.
* Equal contribution by Fernando Martinez and Abhishek Satyam.