A Data-Driven Framework for Decision Making Under Uncertainty: Integrating Markov Decision Processes, Hidden Markov Models and Predictive Modeling
The problem of decision making under uncertainty can be broken down into two parts. First, how do we learn about the world? This involves the problem of modeling the system and its uncertainty. Secondly, given what we currently know about the world, how should we decide what to do, taking into account uncertainty of future events and observations that may change our conclusions. Many systems evolve over time and often the next state of the system is not known with certainty, often modeled as a probability distribution over system states. Dealing with such systems especially when we can make a decision at diﬀerent points in time is diﬃcult due to uncertainty. Making optimal decisions requires understanding the system including its characteristics, how it evolves and changes over time, and how taken actions aﬀect the system. There are multiple dimensions to this problem, and each dimension might require its own speciﬁc method. We need a descriptive method that can summarize the system and its evolution, a predictive model that is used to extract information from the complicated systems and also a prescriptive model that works as the main decision model and incorporates the eﬀects of actions. In this thesis I consider Partially Observable Markov Decision Process (POMDP) as the main decision-making/prescriptive model, Hidden Markov Models (HMM) as the descriptive model of system evolution, and a predictive model to create observations for the POMDP. In this research, I develop a framework by combining these methods and demonstrate its use with two applications. I apply the proposed framework to the problem of diabetes screening and also resource allocation under uncertainty for emergency management. I demonstrate using simulation that implementing the proposed policy will bring about signiﬁcant improvements in both systems compared to the existing policies.
Engineering, Management, Information, and Systems
Creative Commons License
This work is licensed under a Creative Commons Attribution-Noncommercial 4.0 License
Kamalzadeh, Hossein, "A Data-Driven Framework for Decision Making Under Uncertainty: Integrating Markov Decision Processes, Hidden Markov Models and Predictive Modeling" (2020). Operations Research and Engineering Management Theses and Dissertations. 11.
Operational Research Commons, Other Operations Research, Systems Engineering and Industrial Engineering Commons