0485146 20240903170638.920180119235959.9 85040739483 000424732300008 10.14736/kyb-2017-6-1086 14 s. P cav_un_epca*02971630023-5954536 (2017)1086-1099Ústav teorie informace a automatizace AV ČR, v. v. i. Markov decision chains second order optimality optimalilty conditions for transient, discounted and average models policy and value iterations cav_un_auth*0101196 Ekonometrie Department of Econometrics E E Department of Econometrics 100% Sladký Karel UTIA-B K Ústav teorie informace a automatizace AV ČR, v. v. i. http://library.utia.cas.cz/separaty/2017/E/sladky-0485146.pdf cav_un_auth*0321097 GA15-10331S GA ČR The article is devoted to Markov reward chains in discrete-time setting with finite state spaces. Unfortunately, the usual optimization criteria examined in the literature on Markov decision chains, such as a total discounted, total reward up to reaching some specific state (called the first passage models) or mean (average) reward optimality, may be quite insufficient to characterize the problem from the point of a decision maker. To this end it seems that it may be preferable if not necessary to select more sophisticated criteria that also reflect variability -risk features of the problem. Perhaps the best known approaches stem from the classical work of Markowitz on mean variance selection rules, i.e. we optimize the weighted sum of average or total reward and its variance. The article presents explicit formulae for calculating the variances for transient and discounted models (where the value of the discount factor depends on the current state and action taken) for finite and infinite time horizon. The same result is presented for the long run average nondiscounted models where finding stationary policies minimizing the average variance in the class of policies with a given long run average reward is discussed. BB 10000 10100 10103 2018 1 RVO:67985556 http://hdl.handle.net/11104/0280354 S 3+4 Article|Proceedings Paper Computer Science Cybernetics 3+4 Article|Proceedings Paper Computer Science Cybernetics 3+4 Article|Proceedings Paper Computer Science Cybernetics COMPUTERSCIENCE.CYBERNETICS 0.596 0.048 12.4 0.00096 0.224 808 0.321 0.513 63 Q4 18.907 11.4 Q4 Q3 0.2 Q4 11.364 2017 85040739483 SCOPUS PUBMED 000424732300008 WOS cav_un_epca*0297163 Kybernetika 0023-5954 Roč. 53 č. 6 2017 1086 1099 Ústav teorie informace a automatizace AV ČR, v. v. i.