R package pomdp: Partially Observable Markov Decision Processes

CRAN version R build status CRAN RStudio mirror downloads

Provides the infrastructure to define and analyze the solutions of Partially Observable Markov Decision Processes (POMDP) models. The package includes pomdp-solve (Cassandra, 2015) to solve POMDPs using a variety of algorithms.

The package provides the following algorithms:

Installation

Stable CRAN version: install from within R with

install.packages("pomdp")

Current development version: install from GitHub (needs devtools).

library("devtools")
install_github("mhahsler/pomdp")

Usage

Solving the simple infinite-horizon Tiger problem.

R> library("pomdp")
R> data("Tiger")
R> Tiger
Unsolved POMDP model: Tiger Problem 
    horizon: Inf 
> sol <- solve_POMDP(model = Tiger)
> sol
Solved POMDP model: Tiger Problem 
    solution method: grid 
    horizon: Inf 
    converged: TRUE 
    total expected reward (for start probabilities): 1.933439 
> policy(sol)
[[1]]
  tiger-left tiger-right     action tiger-left tiger-right
1 -98.549921   11.450079  open-left          3           3
2 -10.854299    6.516937     listen          3           1
3   1.933439    1.933439     listen          4           2
4   6.516937  -10.854299     listen          5           3
5  11.450079  -98.549921 open-right          3           3

References

Acknowledgments

Development of this package was supported in part by National Institute of Standards and Technology (NIST) under grant number 60NANB17D180.