報(bào) 告 人：劉衛(wèi)東 教授

報(bào)告題目：Online Estimation and Inference for Robust Policy Evaluation in Reinforcement Learning

報(bào)告時(shí)間：2023年10月14日（周六上午10：10 )

報(bào)告地點(diǎn)：江蘇師范大學(xué)數(shù)學(xué)與統(tǒng)計(jì)學(xué)院學(xué)術(shù)報(bào)告廳（靜遠(yuǎn)樓1506室）

主辦單位：數(shù)學(xué)研究院、數(shù)學(xué)與統(tǒng)計(jì)學(xué)院、科學(xué)技術(shù)研究院

報(bào)告人簡(jiǎn)介：

       劉衛(wèi)東，上海交通大學(xué)特聘教授，國(guó)家杰出青年科學(xué)基金獲得者，中國(guó)工業(yè)與應(yīng)用數(shù)學(xué)學(xué)會(huì)理事。主要研究方向?yàn)榻y(tǒng)計(jì)學(xué)和機(jī)器學(xué)習(xí)等，目前已在AOS、 JASA、JRSSB、Biometrika、JMLR、ICML、IJCAI、IEEE TSP等專(zhuān)業(yè)頂尖期刊/會(huì)議上發(fā)表論文六十余篇。主持國(guó)家重點(diǎn)研發(fā)計(jì)劃課題1項(xiàng)，國(guó)家杰出青年科學(xué)基金1項(xiàng)，國(guó)家優(yōu)秀青年科學(xué)基金1項(xiàng)。

報(bào)告摘要： 

       Recently, reinforcement learning has gained prominence in modern statistics, with policy evaluation being a key component. Unlike traditional machine learning literature on this topic, our work places emphasis on statistical inference for the parameter estimates computed using reinforcement learning algorithms. While most existing analyses assume random rewards to follow standard distributions, limiting their applicability, we embrace the concept of robust statistics in reinforcement learning by simultaneously addressing issues of outlier contamination and heavy-tailed rewards within a unified framework. In this paper, we develop an online robust policy evaluation procedure, and establish the limiting distribution of our estimator, based on its Bahadur representation. Furthermore, we develop a fully-online procedure to efficiently conduct statistical inference based on the asymptotic distribution. This paper bridges the gap between robust statistics and statistical inference in reinforcement learning, offering a more versatile and reliable approach to policy evaluation. Finally, we validate the efficacy of our algorithm through numerical experiments conducted in real-world reinforcement learning experiments.