Week 8: Simple bandit

Controls

0, 1, …, 9

Select (take) a specific action \(a_k = 0, \dots, 9\).

Space

Take a single action from the simple bandit algorithm

q

Show the true average rewards \(q^*_k\). The highest indicate the best arm.

p

Automatically take actions

r

Reset the game

Run locally

irlc/lectures/lec08/demo_bandit.py.

What you see

The example shows the simple bandit algorithm applies to to a 10-armed problem with binary rewards. In step \(k\), when you select an arm \(a_k\), the environment gives you a reward of \(r_k=1\) with probability \(q_k^*\) and otherwise a reward of 0.

The height of the bars show the current average reward for each arm, and the numbers how often they have been tried. You can show the actual average rewards \(q_k^*\) by pressing q (although this is cheating! if we knew these values there would be no point in running a bandit algorithm!).

How it works

todo.