bibtype	K - Conference Paper (Czech conference)
ARLID	0517875
utime	20240103223157.9
mtime	20191213235959.9
title (primary) (eng)	Second Order Optimality in Markov and Semi-Markov Decision Processes
specification	page_count 6 s. media_type E
serial	ARLID cav_un_epca*0509647 ISBN 978-80-7394-760-6 title Conference Proceedings. 37th International Conference on Mathematical Methods in Economics 2019 page_num 338-343 publisher place České Budějovice name University of South Bohemia in České Budějovice, Faculty of Economics year 2019 editor name1 Houda name2 M. editor name1 Remeš name2 R.
keyword	semi-Markov processes with rewards
keyword	discrete and continuous-time Markov reward chains
keyword	risk-sensitive optimality
keyword	average reward and variance over time
author (primary)	ARLID cav_un_auth*0101196 name1 Sladký name2 Karel institution UTIA-B full_dept (cz) Ekonometrie full_dept (eng) Department of Econometrics department (cz) E department (eng) E full_dept Department of Econometrics fullinstit Ústav teorie informace a automatizace AV ČR, v. v. i.
source	url http://library.utia.cas.cz/separaty/2019/E/sladky-0517875.pdf
cas_special	project ARLID cav_un_auth*0363963 project_id GA18-02739S agency GA ČR abstract (eng) Semi-Markov decision processes can be considered as an extension of discrete- and continuous-time Markov reward models. Unfortunately, traditional optimality criteria as long-run average reward per time may be quite insufficient to characterize the problem from the point of a decision maker. To this end it may be preferable if not necessary to select more sophisticated criteria that also reflect variability-risk features of the problem. Perhaps the best known approaches stem from the classical work of Markowitz on mean-variance selection rules, i.e. we optimize the weighted sum of average or total reward and its variance. Such approach has been already studied for very special classes of semi-Markov decision processes, in particular, for Markov decision processes in discrete - and continuous-time setting. In this note these approaches are summarized and possible extensions to the wider class of semi-Markov decision processes is discussed. Attention is mostly restricted to uncontrolled models in which the chain is aperiodic and contains a single class of recurrent states. Considering finite time horizons, explicit formulas for the first and second moments of total reward as well as for the corresponding variance are produced. action ARLID cav_un_auth*0379399 name MME 2019: International Conference on Mathematical Methods in Economics /37./ dates 20190911 mrcbC20-s 20190913 place České Budějovice country CZ RIV BB FORD0 10000 FORD1 10100 FORD2 10103 reportyear 2020 num_of_auth 1 presentation_type PR inst_support RVO:67985556 permalink http://hdl.handle.net/11104/0303159 confidential S arlyear 2019 mrcbU14 SCOPUS mrcbU24 PUBMED mrcbU34 WOS mrcbU63 cav_un_epca*0509647 Conference Proceedings. 37th International Conference on Mathematical Methods in Economics 2019 978-80-7394-760-6 338 343 České Budějovice University of South Bohemia in České Budějovice, Faculty of Economics 2019 mrcbU67 Houda M. 340 mrcbU67 340 Remeš R.