In this talk, I will address the problem of relating linguistic analysis and control --- specifically, mapping natural language instructions to executable actions. I will present a reinforcement learning algorithm for inducing these mappings by interacting with virtual computer environments and observing the outcome of the executed actions. This technique has enabled automation of tasks that until now have required human participation --- for example, automatically configuring software by consulting how-to guides. I will also briefly describe a recent extension for learning to interpret high-level instructions, ones that posit goals without explicitly describing the actions required to achieve them. Our results demonstrate that in both cases, the method can rival supervised learning techniques while requiring few or no annotated training examples.
Back to symposium main page