10h00   (Auditorium CW)

Kurt Driessens, K.U.Leuven
Title: All work and no play make Jack a dull boy: on Poker research at deptcw (abstract)

11h00   (Auditorium CW)

Kristian Kersting , Fraunhofer IAIS, Germany
Title: Lifting and Wrapping AI (abstract)

12h00   (De Moete)

Sandwich lunch

13h30   (Aula van de Tweede Hoofdwet, 01.02 (Thermotechnisch instituut))

Tom Croonenborghs
PhD defense

You are too late, you can no longer register.

• Auditorium CW: This is the auditorium (Room 00.225) of the Computer Science building, Celestijnanlaan 200A, B-3001 Heverlee
• De Moete: located right across the computer science building
• Aula van de Tweede Hoofdwet, 01.02, Kasteelpark Arenberg 41 (Heverlee).
  There will be signs. The ''machinezaal'' is located next to the auditorium.

All work and no play make Jack a dull boy: on Poker research at deptcw

Kurt Driessens

Now that Go is solved [1] many AI researchers consider Poker to be the next challenging testbed for AI game research. In terms of game theory, Poker is a zero sum, hidden information game, with a varying number of competing agents and a game tree size estimated at 10^71 nodes. In terms of game practice, it is a battle field where many top players believe that cunning and cleverness is much more important than the actual cards dealt. In this seminar, I will discuss the research directions that we did, are currently investigating and plan to investigate in the field of multiplayer, no-limit, Texas Hold'em Poker. These directions range from the hard, number crunching side of Poker that is game tree sampling, over the middle ground of opponent modeling, to the soft, human side that is body and composure reading.

[1] Robby Goetschalckx, liberal interpretation of a piece of personal communication pulled out of context

Lifting and Wrapping AI

Kristian Kersting

AI and ML have made tremendous progress so far. Reasonably large problems can be solved using current techniques. But what if we want to scale up to the problems that you face every day? They are often orders of magnitude larger than the biggest problem we can solve currently. In this talk, I will describe two pieces of work that try to begin to address working in truly huge environments.

The first method exploits symmetries in probabilistic models to speed up inference. Specifically, it automatically groups together nodes and factors that are indistinguishable in terms of messages received and sent given the evidence. This lifting can speed up belief propagation by orders of magnitude in a variety of important AI tasks such as inference for relational probabilistic models and boolean model counting.

AI and ML techniques, however, not only consist of the model but also of the data. So, how much can the data itself help us to solve problems? This direction is particularly appealing given that the Internet nowadays offers a plentiful supply of large-scale datasets for many challenging tasks. I will present a data-driven non-negative matrix factorization (NMF) approach that restricts the "clusters" to be combinations of vertices of the convex hull of the dataset; thus directly exploring the data itself to solve the convex NMF problem. I will show that this wrapping can indeed effectively extract meaningful "clusters" from datasets containing millions of images and rating.

This is joint work with Babak Ahmadi, Christian Bauckhage, Sriraam Natarajan, and Christian Thurau.

[PhD defense] Model-Assisted Approaches for Relational Reinforcement Learning

Tom Croonenborghs

Machine learning is concerned with developing software systems that learn from experience. An important subtopic of machine learning is reinforcement learning (RL) where the software systems need to learn through interaction with their environment based on the feedback they receive on the quality of their actions. The goal of reinforcement learning algorithms is to learn a policy, i.e., a function that indicates how such a software system ought to take actions in an environment, that maximizes some notion of long-term reward.

In order to apply these algorithms in more complex environments, there has been a lot of study on the integration of abstraction and generalization in reinforcement learning techniques. One method for which there is a growing interest is the use of relational representations. Relational reinforcement learning (RRL) combines the reinforcement learning setting with relational learning in order to represent the states, actions and policies using the structures and relations that identify them.

In this dissertation, we will investigate methods that improve the learning behavior of relational reinforcement learning techniques through assistance of various learned models. In a first part, three such methods are presented. First, a method is proposed that adds temporal abstraction to RRL. It is shown how these models can be learned online and can be used to equip an RL-agent with skills based on knowledge learned in previous tasks. Next, model-based RRL is presented where the learning agent learns a model of the environment that allows him to predict the effects of actions along with the expected feedback the environment will give him. Third, multi-agent RRL is introduced which investigates the setting where multiple learning agents are present in the environment. It is shown how RRL can aid in communication issues between different agents and how the agents can learn by observing each other.

In a second part, the focus shifts to methods that learn better models. A new incremental relational tree learning algorithm is presented that can deal with concept drift. Furthermore, a learning algorithm is presented that can learn the structure and parameters of directed probabilistic logical models which can for instance be used to learn a model of the environment.