R package pomdp: Partially Observable Markov Decision Processes

CRAN version R build status CRAN RStudio mirror downloads

Provides the infrastructure to define and analyze the solutions of Partially Observable Markov Decision Processes (POMDP) models. The package includes pomdp-solve (Cassandra, 2015) to solve POMDPs using a variety of algorithms.

The package provides the following algorithms:


Stable CRAN version: install from within R with


Current development version: install from GitHub (needs devtools).



Solving the simple infinite-horizon Tiger problem.

R> library("pomdp")
R> data("Tiger")
R> Tiger
Unsolved POMDP model: Tiger Problem 
    horizon: Inf 
> sol <- solve_POMDP(model = Tiger)
> sol
Solved POMDP model: Tiger Problem 
    solution method: grid 
    horizon: Inf 
    converged: TRUE 
    total expected reward (for start probabilities): 1.933439 
> policy(sol)
  tiger-left tiger-right     action tiger-left tiger-right
1 -98.549921   11.450079  open-left          3           3
2 -10.854299    6.516937     listen          3           1
3   1.933439    1.933439     listen          4           2
4   6.516937  -10.854299     listen          5           3
5  11.450079  -98.549921 open-right          3           3



Development of this package was supported in part by National Institute of Standards and Technology (NIST) under grant number 60NANB17D180.