2nd European Workshop on Automatic Differentiation

Thursday November 17- Friday November 18, 2005

Whitworth Conference Centre

Cranfield University (Shrivenham Campus)

Defence Academy of the UK

Shrivenham, Swindon

Prof. Jeffrey Mark Siskind (Purdue University)

Nesting, Variable Capture, Programming Language Theory, and AD

Abstract:  A major barrier to the widespread use of AD is that current systems make AD difficult to use in a modular and convenient fashion. For instance, modularity and programmer convenience would seem to require that the author of a minimisation routine be able to use AD internally without exposing its use to the caller. Moreover, a routine that uses AD internally should be useable within a routine that will itself be processed via AD, and this should be allowed without any extra caveats. Supporting this sort of nested and modular use of AD requires viewing AD in terms of the mathematical and practical frameworks of programming language theory. Doing so reveals a difficult issue: variable capture. Solving this issue opens the door to the construction of AD systems that allow nested use of AD operators that interact orthogonally with other language constructs, and moreover expresses the AD operators in a fashion that makes them amenable to a variety of aggressive program optimisation techniques developed over the last two decades. After briefly introducing these issues, we will describe how AD operators can be incorporated as first-class citizens in an augmented lambda-calculus, which is the mathematical formalism in which programming language semantics are expressed. We will then exhibit a system that embodies this work and therefore allows free modular nested use of the AD operators, and show how it allows previously problematic constructs to be naturally expressed.

Slides: SiskindNov05.pdf