N El Karoui, I Karatzas (1993) General Gittins index processes in discrete time. Proc Natl Acad Sci U S A 90: 4. 1232-1236 Feb Abstract: We combine the formulation of Mandelbaum [Mandelbaum, A. (1986) Probab. Theory Rel. Fields 71, 129-147] with ideas from Whittle [Whittle, P. (1980) J. R. Stat. Soc. B 42, 143-149] to obtain a simple and constructive proof for the optimality of Gittins index processes in the general, nonmarkovian dynamic allocation (or "multi-armed bandit") problem. Our approach also provides an explicit expression for the value of this problem.
Notes: