Agent applications are ubiquitous in commerce and industry, and the sophistication, complexity, and importance of these applications is increasing rapidly; they include speech recognition systems, vision systems, search engines, planetary explorers, auto-pilots, spam filters, and robots [RN03]. Existing agent technology can be improved by developing systems that can automatically acquire during deployment much of the knowledge that would otherwise be required to be built in by agent designers. This greatly reduces the effort required for agent construction, and results in agents that are more adaptive and operate successfully in a wide variety of environments [LH07].
Technically, the project is about a recent general approach to learning that bridges the gap between theory and practice in reinforcement learning (RL). General-purpose, intelligent, learning agents cycle through sequences of observations, actions, and rewards that are complex, uncertain, unknown, and non-Markovian [RN03]. On the other hand, RL is well-developed for small finite state Markov decision processes (MDPs) [SB98]. Extracting the right state representations out of bare observations, that is, reducing the general agent setup to the MDP framework, is an art that involves significant effort by designers. The project is to investigate (by simulations or theoretical) recent models [Hut09] that automate the reduction process and thereby significantly expand the scope of many existing RL algorithms and the agents that employ them.
- background in Artificial Intelligence and Machine Learning
- good programming skills
- performing (computer) experiments and analyzing results
- good math skills; linear algebra at the very minimum
- mastering elementary probability calculus
- [RN10] S.J. Russell and P. Norvig. Artificial Intelligence: A Modern Approach Prentice-Hall, Englewood Cliffs, NJ, 3rd edition, 2010.
- [SB98] R.S. Sutton and A.G. Barto. Reinforcement Learning: An Introduction MIT Press, Cambridge, MA, 1998.
- [LH07] S. Legg and M. Hutter. Universal intelligence: A definition of machine intelligence. Minds & Machines, 17(4):391-444, 2007.
- [Hut09] M. Hutter. Feature reinforcement learning: Part I: Unstructured MDPs. Journal of Artificial General Intelligence, 1:3-24, 2009.
- [Hut14] M. Hutter. Extreme state aggregation beyond MDPs. In Proc. 25th Intl. Conf. on Algorithmic Learning Theory (ALT'14), LNAI 8776, pages 185--199.
- getting acquainted with state-of-the art RL algorithms
- improving your math skills: linear algebra, statistics, probability, and information theory