Improving value function approximation in factored POMDPs by exploiting model structure

Tiago S. Veiga; Matthijs T.J. Spaan; Pedro U. Lima

Improving value function approximation in factored POMDPs by exploiting model structure

Tiago S. Veiga, Matthijs T.J. Spaan, Pedro U. Lima

Algorithmics

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

Abstract

Linear value function approximation in Markov decision processes (MDPs) has been studied extensively, but there are several challenges when applying such techniques to partially observable MDPs (POMDPs). Furthermore, the system designer often has to choose a set of basis functions. We propose an automatic method to derive a suitable set of basis functions by exploiting the structure of factored models. We experimentally show that our approximation can reduce the solution size by several orders of magnitude in large problems.

Original language	English
Title of host publication	AAMAS 2015 - Proceedings of the 2015 International Conference on Autonomous Agents and Multiagent Systems
Publisher	International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS)
Pages	1827-1828
Number of pages	2
Volume	3
ISBN (Electronic)	9781450337717
Publication status	Published - 1 Jan 2015
Event	AAMAS 2015: 14th International Conference on Autonomous Agents and Multiagent Systems - Istanbul, Turkey Duration: 4 May 2015 → 8 May 2015 Conference number: 14

Conference

Conference	AAMAS 2015: 14th International Conference on Autonomous Agents and Multiagent Systems
Abbreviated title	AAMAS 2015
Country/Territory	Turkey
City	Istanbul
Period	4/05/15 → 8/05/15

Keywords

POMDP
Value function approximation

Cite this

@inproceedings{7225d49128514c429efca1f84f9d3748,

title = "Improving value function approximation in factored POMDPs by exploiting model structure",

abstract = "Linear value function approximation in Markov decision processes (MDPs) has been studied extensively, but there are several challenges when applying such techniques to partially observable MDPs (POMDPs). Furthermore, the system designer often has to choose a set of basis functions. We propose an automatic method to derive a suitable set of basis functions by exploiting the structure of factored models. We experimentally show that our approximation can reduce the solution size by several orders of magnitude in large problems.",

keywords = "POMDP, Value function approximation",

author = "Veiga, {Tiago S.} and Spaan, {Matthijs T.J.} and Lima, {Pedro U.}",

year = "2015",

month = jan,

day = "1",

language = "English",

volume = "3",

pages = "1827--1828",

booktitle = "AAMAS 2015 - Proceedings of the 2015 International Conference on Autonomous Agents and Multiagent Systems",

publisher = "International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS)",

note = "AAMAS 2015: 14th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2015 ; Conference date: 04-05-2015 Through 08-05-2015",

}

Veiga, TS, Spaan, MTJ & Lima, PU 2015, Improving value function approximation in factored POMDPs by exploiting model structure. in AAMAS 2015 - Proceedings of the 2015 International Conference on Autonomous Agents and Multiagent Systems. vol. 3, International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS), pp. 1827-1828, AAMAS 2015: 14th International Conference on Autonomous Agents and Multiagent Systems, Istanbul, Turkey, 4/05/15.

Improving value function approximation in factored POMDPs by exploiting model structure. / Veiga, Tiago S.; Spaan, Matthijs T.J.; Lima, Pedro U.
AAMAS 2015 - Proceedings of the 2015 International Conference on Autonomous Agents and Multiagent Systems. Vol. 3 International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS), 2015. p. 1827-1828.

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

TY - GEN

T1 - Improving value function approximation in factored POMDPs by exploiting model structure

AU - Veiga, Tiago S.

AU - Spaan, Matthijs T.J.

AU - Lima, Pedro U.

N1 - Conference code: 14

PY - 2015/1/1

Y1 - 2015/1/1

N2 - Linear value function approximation in Markov decision processes (MDPs) has been studied extensively, but there are several challenges when applying such techniques to partially observable MDPs (POMDPs). Furthermore, the system designer often has to choose a set of basis functions. We propose an automatic method to derive a suitable set of basis functions by exploiting the structure of factored models. We experimentally show that our approximation can reduce the solution size by several orders of magnitude in large problems.

AB - Linear value function approximation in Markov decision processes (MDPs) has been studied extensively, but there are several challenges when applying such techniques to partially observable MDPs (POMDPs). Furthermore, the system designer often has to choose a set of basis functions. We propose an automatic method to derive a suitable set of basis functions by exploiting the structure of factored models. We experimentally show that our approximation can reduce the solution size by several orders of magnitude in large problems.

KW - POMDP

KW - Value function approximation

UR - http://www.scopus.com/inward/record.url?scp=84944678155&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:84944678155

VL - 3

SP - 1827

EP - 1828

BT - AAMAS 2015 - Proceedings of the 2015 International Conference on Autonomous Agents and Multiagent Systems

PB - International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS)

T2 - AAMAS 2015: 14th International Conference on Autonomous Agents and Multiagent Systems

Y2 - 4 May 2015 through 8 May 2015

ER -

Improving value function approximation in factored POMDPs by exploiting model structure

Abstract

Conference

Keywords

Other files and links

Fingerprint

Cite this