Abstract
The problem of decision making under uncertainty can be broken down into two parts. First, how do we learn about the world? This involves the problem of modeling the system and its uncertainty. Secondly, given what we currently know about the world, how should we decide what to do, taking into account uncertainty of future events and observations that may change our conclusions. Many systems evolve over time and often the next state of the system is not known with certainty, often modeled as a probability distribution over system states. Dealing with such systems especially when we can make a decision at different points in time is difficult due to uncertainty. Making optimal decisions requires understanding the system including its characteristics, how it evolves and changes over time, and how taken actions affect the system. There are multiple dimensions to this problem, and each dimension might require its own specific method. We need a descriptive method that can summarize the system and its evolution, a predictive model that is used to extract information from the complicated systems and also a prescriptive model that works as the main decision model and incorporates the effects of actions. In this thesis I consider Partially Observable Markov Decision Process (POMDP) as the main decision-making/prescriptive model, Hidden Markov Models (HMM) as the descriptive model of system evolution, and a predictive model to create observations for the POMDP. In this research, I develop a framework by combining these methods and demonstrate its use with two applications. I apply the proposed framework to the problem of diabetes screening and also resource allocation under uncertainty for emergency management. I demonstrate using simulation that implementing the proposed policy will bring about significant improvements in both systems compared to the existing policies.
Degree Date
Spring 5-16-2020
Document Type
Dissertation
Degree Name
Ph.D.
Department
Engineering, Management, Information, and Systems
Advisor
Michael Hahsler
Format
Creative Commons License
This work is licensed under a Creative Commons Attribution-Noncommercial 4.0 License
Recommended Citation
Kamalzadeh, Hossein, "A Data-Driven Framework for Decision Making Under Uncertainty: Integrating Markov Decision Processes, Hidden Markov Models and Predictive Modeling" (2020). Operations Research and Engineering Management Theses and Dissertations. 11.
https://scholar.smu.edu/engineering_managment_etds/11
Included in
Operational Research Commons, Other Operations Research, Systems Engineering and Industrial Engineering Commons
