ctutest-Intro.tex

\chapter{Introduction}

Markov Decision Problems are one of the most well-known methods used for solving stochastic planning problems. In the last couple of years, a new programming language, Julia, has emerged into the scientific community's consciousness. While offering a high-level approach comparable to the one of Python, it also offers a similar speed to languages like C or C++ at the same time. As new users come to this language, the demand for new tools rises. The subject of this work answers these calls and contributes to the open-source framework \textit{POMDPs.jl} 

The POMDPs.jl framework is an open-source framework for defining, solving, and simulating both MDP and POMDP problems. It is built on several interoperating interfaces whose goal is to clearly define the structure of all code associated with the framework. Thanks to this, it is easy to change the script's functionality by only a few changes, implement new methods, or improve existing ones.


This work aims to survey and extend the methods implemented in the POMDPs.jl framework with an emphasis on the finite horizon ones. The task includes the addition of any other methods or interfaces missing to implement such methods and maintain interoperability with the rest of the framework. Finally, this work evaluates the performance of the implemented methods on benchmarks and new test instances designed for their purpose.

As a part of our contribution, we introduce the interface for defining Finite Horizon (PO)MDPs \textit{FiniteHorizonPOMDPs.jl}. With this interface, the user can either define his Finite Horizon problem or use the \textit{fixhorizon} utility to transform an existing Infinite Horizon (PO)MDP into a Finite Horizon one.

To present the idea of finite horizon solver, we implement the \textit{FiniteHorizonValueIteration.jl} algorithm, which solves problems defined as Finite Horizon MDPs. In addition, we implement FiVI TODO:DAT TOMU NAZEV A ZVEREJNIT TO, which serves as the ideal starting point for future framework solver contributions. To benchmark the FIVI, we also implement the \textit{PointbasedValueIteration.jl} algorithm as a bonus.



% \section{Julia}
% Julia is a young language that starts to receive traction for its features. According to \cite{JuliaStars} the number of stars on its repository \cite{JuliaLang} tripled over the last 20 months and is rising steadily.
% The development started in 2009 at MIT, and the project went public in 2012 \cite{JuliaHistory}. 
% It combines the performance of low-level languages like Fortran or C with the high-level approach of Matlab, R and Python. In addition to that, Julia is dynamically typed, designed for parallelism and distributed computation, it offers multiple dispatch paradigms and is also reproducible while relying on community packages \cite{JuliaLangorg}.