Fetching the paper…
Reading the bibliography…
This article provides an introductory tutorial on structural results in partially observed Markov decision processes (POMDPs).
Matrix Theory
F. R. Gantmacher · 1960
Earlier work this paper cites.
Conditional Markov processes
R. L. Stratonovich · 1960
Earlier work this paper cites.
On optimum methods in quickest detection problems
A..N. Shiryaev · 1963
Earlier work this paper cites.
Optimal control of Markov processes with incomplete state information
K. J. Åström · 1965
Earlier work this paper cites.
Controlled random sequences
E. Dynkin · 1965
Earlier work this paper cites.
Total Positivity
S. Karlin · 1968
Earlier work this paper cites.
A simple model of search for a moving target
S.M. Pollock · 1970
Earlier work this paper cites.
The optimal control of partially observed Markov processes
E. J. Sondik · 1971
Earlier work this paper cites.
Optimal control of partially observable Markov processes over a finite horizon
R. D. Smallwood and E. J. Sondik · 1973
Earlier work this paper cites.
Optimal system allocations with penalty cost
C. Derman, G. J. Lieberman, and S. M. Ross · 1976
Earlier work this paper cites.
Principles of Mathematical Analysis
W. Rudin · 1976
Earlier work this paper cites.
Monotone matrices and monotone Markov processes
J. Keilson and A. Kester · 1977
Earlier work this paper cites.
The optimal control of partially observable Markov processes over the infinite horizon: discounted costs
E. J. Sondik · 1978
Earlier work this paper cites.
Minimizing a submodular function on a lattice
D. M. Topkis · 1978
Earlier work this paper cites.
Structural results for partially observed Markov decision processes
S.C. Albright · 1979
Earlier work this paper cites.
Optimal stopping, exponential utility, and linear programming
E. Denardo and U. Rothblum · 1979
Earlier work this paper cites.
Classes of orderings of measures and related correlation inequalities. I. Multivariate totally positive distributions
S. Karlin and Y. Rinott · 1980
Earlier work this paper cites.
Application of Jensen’s inequality to adaptive suboptimal design
C. C. White and D. P. Harrington · 1980
Earlier work this paper cites.
Multi-armed bandits and the Gittins index
P. Whittle · 1980
Earlier work this paper cites.
A Second Course in Stochastic Processes
S. Karlin and H. M. Taylor · 1981
Earlier work this paper cites.
Good news and bad news: Representation theorems and applications
P. Milgrom · 1981
Earlier work this paper cites.
A survey of partially observable Markov decision processes: theory, models and algorithms
G. E. Monahan · 1982
Earlier work this paper cites.
Multivariate monotone likelihood ratio and uniform conditional stochastic order
W. Whitt · 1982
Earlier work this paper cites.
Introduction to Stochastic Dynamic Programming
S. Ross · 1983
Earlier work this paper cites.
The optimal search for a moving target when the search path is constrained
J. N. Eagle · 1984
Earlier work this paper cites.
Stochastic Models in Operations Research
D. P. Heyman and M. J. Sobel · 1984
Earlier work this paper cites.
Optimal stopping times for detecting changes in distributions
G.B. Moustakides · 1986
Earlier work this paper cites.
On the convexity of policy regions in partially observed systems
W. S. Lovejoy · 1987
Earlier work this paper cites.
Ordered solutions for dynamic programs
W. S. Lovejoy · 1987
Earlier work this paper cites.
Some monotonicity results for partially observed Markov decision processes
W. S. Lovejoy · 1987
Earlier work this paper cites.
The compexity of Markov decision processes
C. H. Papadimitriou and J.N. Tsitsiklis · 1987
Earlier work this paper cites.
Optimal sensor scheduling in nonlinear filtering of diffusion processes
J.S. Baras and A. Bensoussan · 1989
Earlier work this paper cites.
Multi–armed Bandit Allocation Indices
J. C. Gittins · 1989
Earlier work this paper cites.
Structured stochastic matrices of M/G/1 type and their applications
M. F. Neuts · 1989
Earlier work this paper cites.
A tutorial on hidden Markov models and selected applications in speech recognition
L.R. Rabiner · 1989
Earlier work this paper cites.
What’s happened in search theory since the 1975 Lanchester prize
L. Stone · 1989
Earlier work this paper cites.
Studies on the Optimal Search Plan
K. Iida · 1990
Earlier work this paper cites.
Computationally feasible bounds for partially observed Markov decision processes
W. S. Lovejoy · 1991
Earlier work this paper cites.
A survey of algorithmic methods for partially observed Markov decision processes
W. S. Lovejoy · 1991
Earlier work this paper cites.
Structural results for partially observed control models
U. Rieder · 1991
Earlier work this paper cites.
Stochastic Control of Partially Observable Systems
A. Bensoussan · 1992
Earlier work this paper cites.
Detection of Abrupt Changes — Theory and Applications
M. Basseville and I.V. Nikiforov · 1993
Earlier work this paper cites.
An Introduction to Signal Detection and Estimation
H.V. Poor · 1993
Earlier work this paper cites.
Acting optimally in partially observable stochastic domains
A. R. Cassandra, L. Kaelbling, and M. L. Littman · 1994
Earlier work this paper cites.
Risk-sensitive control and dynamic games for partially observed discrete-time nonlinear systems
M.R. James, J.S. Baras, and R.J. Elliott · 1994
Earlier work this paper cites.
Monotone comparative statics
P. Milgrom and C. Shannon · 1994
Cited alongside, same era.
Monotonicity and bounds for convex stochastic control models
U. Rieder and R. Zagst · 1994
Cited alongside, same era.
Monotone Structure in Discete-Event Systems
D. Yao and P. Glasserman · 1994
Cited alongside, same era.
Hidden Markov Models – Estimation and Control
R. J. Elliott, L. Aggoun, and J. B. Moore · 1995
Cited alongside, same era.
Optimal search for a moving target
I. MacPhee and B. Jordan · 1995
Cited alongside, same era.
Time discretization of continuous-time filters and smoothers for HMM parameter estimation
M. R. James, V. Krishnamurthy, and F. LeGland · 1996
Cited alongside, same era.
Q-learning algorithms for constrained Markov decision processes with randomized monotone policies: Applications in transmission control
D. Djonin and V. Krishnamurthy · 2007
Later among the works it cites.
Control limits for two-state partially observable Markov decision processes
A. Grosfeld-Nir · 2007
Later among the works it cites.
Structured threshold policies for dynamic sensor scheduling–a partially observed Markov decision process approach
V. Krishnamurthy and D. Djonin · 2007
Later among the works it cites.
Stochastic orders
M. Shaked and J. G. Shanthikumar · 2007
Later among the works it cites.
Forward search value iteration for POMDPs
G. Shani, R. Brafman, and S. Shimony · 2007
Later among the works it cites.
Approximate dynamic programming for communication-constrained sensor network management
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
M. L. Littman · 1996
Cited alongside, same era.
Regression shrinkage and selection via the lasso
R. Tibshirani · 1996
Cited alongside, same era.
Incremental pruning: A simple fast exact method for partially observed Markov decision processes
A. R. Cassandra, M. L. Littman, and N. L. Zhang · 1997
Cited alongside, same era.
Markov Processes for Stochastic Modelling
M. Kijima · 1997
Cited alongside, same era.
How does the value function of a Markov decision process depend on the transition probabilities?
A. Muller · 1997
Cited alongside, same era.
Exact and Approximate Algorithms for Partially Observed Markov Decision Process
A. R. Cassandra · 1998
Cited alongside, same era.
J. Williams, J. Fisher, , and A. Willsky · 2007
Later among the works it cites.
Decentralized cognitive MAC for opportunistic spectrum access in ad hoc networks: A POMDP framework
Q. Zhao, L. Tong, A. Swami, and Y. Chen · 2007
Later among the works it cites.
Algorithmic copositivity detection by simplicial partition
S. Bundfuss and M. Dür · 2008
Later among the works it cites.
Decentralalized activation in dense sensor networks via global games
V. Krishnamurthy · 2008
Later among the works it cites.
Sarsop: Efficient point-based POMDP planning by approximating optimally reachable belief spaces
H. Kurniawati, D. Hsu, and W. S. Lee · 2008
Later among the works it cites.
Quickest Detection
H. V. Poor and O. Hadjiliadis · 2008
Later among the works it cites.
An adaptive linear approximation algorithm for copositive programs
S. Bundfuss and M. Dür · 2009
Later among the works it cites.
The power of convex relaxation: Near-optimal matrix completion
E. J. Candès and T. Tao · 2009
Later among the works it cites.
Detection and identification of an unobservable change in the distribution of a Markov-modulated random sequence
S. Dayanik and C. Goulding · 2009
Later among the works it cites.
The elements of statistical learning
T. Hastie, R. Tibshirani, and J. Friedman · 2009
Later among the works it cites.
Decentralized spectrum access amongst cognitive radios-an interacting multivariate global game-theoretic approach
V. Krishnamurthy · 2009
Later among the works it cites.
Optimal threshold policies for multivariate POMDPs in radar resource management
V. Krishnamurthy and D.V. Djonin · 2009
Later among the works it cites.
POMDP multiarmed bandits – structural results
V. Krishnamurthy and B. Wahlberg · 2009
Later among the works it cites.
A tutorial on partially observable Markov decision processes
M. L. Littman · 2009
Later among the works it cites.
Interior-point method for nuclear norm approximation with application to system identification
Z. Liu and L. Vandenberghe · 2009
Later among the works it cites.
Optimality of threshold policies for transmission scheduling in correlated fading channels
M. H. Ngo and V. Krishnamurthy · 2009
Later among the works it cites.
Bayesian quickest change process detection
V. Raghavan and V. Veeravalli · 2009
Later among the works it cites.
How does a stochastic optimization/approximation algorithm adapt to a randomly evolving optimum/root with jump Markov sample paths
G. Yin, C. Ion, and V. Krishnamurthy · 2009
Later among the works it cites.
Transmission control in cognitive radio as a markovian dynamic game: Structural result on randomized threshold policies
J. Huang and V. Krishnamurthy · 2010
Later among the works it cites.
Indexability of restless bandit problems and optimality of Whittle index for dynamic multichannel access
K. Liu and Q. Zhao · 2010
Later among the works it cites.
Monotonicity of constrained optimal transmission policies in correlated fading channels with ARQ
M. H. Ngo and V. Krishnamurthy · 2010
Later among the works it cites.
On pth roots of stochastic matrices
N. Higham and L. Lin · 2011
Later among the works it cites.
Data-efficient quickest change detection with on-off observation control
T. Banerjee and V. Veeravalli · 2012
Later among the works it cites.
Quickest detection of market shocks in agent based models of the order book
V. Krishnamurthy and A. Aryan · 2012
Later among the works it cites.
Sequential detection with mutual information stopping cost: Application in GMTI radar
V. Krishnamurthy, R. Bitmead, M. Gevers, and E. Miehling · 2012
Later among the works it cites.
Aggregating the single crossing property
J. Quah and B. Strulovici · 2012
Later among the works it cites.
More risk-sensitive Markov decision processes
N. Bäuerle and U. Rieder · 2013
Later among the works it cites.
Partially observed Markov decision processes with binomial observations
T. Ben-Zvi and A. Grosfeld-Nir · 2013
Later among the works it cites.
How to schedule measurements of a noisy Markov chain in decision making?
V. Krishnamurthy · 2013
Later among the works it cites.
Detecting asset value dislocations in multi-agent models for market microstructure
V. Krishnamurthy and A. Aryan · 2013
Later among the works it cites.
Computing monotone policies for Markov decision processes by exploiting sparsity
V. Krishnamurthy, C. Rojas, and B. Wahlberg · 2013
Later among the works it cites.
A survey of point-based POMDP solvers
G. Shani, J. Pineau, and R. Kaplow · 2013
Later among the works it cites.
POMDP-based statistical spoken dialog systems: A review
S. Young, M. Gasic, B. Thomson, and J. Williams · 2013
Later among the works it cites.
Interactive sensing and decision making in social networks
V. Krishnamurthy, O. Namvar Gharehshiran, and M. Hamdi · 2014
Later among the works it cites.
A tutorial on interactive sensing in social networks
V. Krishnamurthy and H. V. Poor · 2014
Later among the works it cites.
Reduced complexity HMM filtering with stochastic dominance bounds: A convex optimization approach
V. Krishnamurthy and C. Rojas · 2014
Later among the works it cites.
Online reputation and polling systems: Data incest, social learning and revealed preferences
V. Krishnamurthy and W. Hoiles · 2015
Closest in time.
Myopic bounds for optimal policy of POMDPs: An extension of Lovejoy’s structural results
V. Krishnamurthy and U. Pareek · 2015
Closest in time.
Partially Observed Markov Decision Processes. From Filtering to Controlled Sensing
V. Krishnamurthy · 2016
Closest in time.