RESEARCH · RESEARCH · #1257
Apply CVaR to immediate cost for risk-averse online POMDP planning with performance guarantees
The paper proposes applying Conditional Value at Risk (CVaR) to the immediate cost computed over the belief at each step in online POMDP planning, directly addressing within-belief uncertainty while keeping the standard expected cumulative return objective. This preserves an MDP structure so existing expectation-based POMDP planners can be reused by changing only the cost computation, and the authors provide finite-time guarantees for policy evaluation and sparse sampling plus a finite-time bound on the gap between the particle belief MDP surrogate and the original POMDP; the formulation reduces to standard expectation-based planning in the risk-neutral limit.
KEY POINTS
- The paper proposes applying Conditional Value at Risk (CVaR) to the immediate cost computed over the belief at each step in online POMDP planning, directly addressing within-belief uncertainty while keeping the standard expected cumulative return objective.
- This preserves an MDP structure so existing expectation-based POMDP planners can be reused by changing only the cost computation, and the authors provide finite-time guarantees for policy evaluation and sparse sampling plus a finite-time bound on the gap between the particle belief MDP surrogate and the original POMDP; the formulation reduces to standard expectation-based planning in the risk-neutral limit.
- Addresses within-belief risk directly while allowing reuse of standard planners and supplying end-to-end finite-time guarantees, which matters for safer online decision-making under state uncertainty.
WHY IT MATTERS
Addresses within-belief risk directly while allowing reuse of standard planners and supplying end-to-end finite-time guarantees, which matters for safer online decision-making under state uncertainty.