2nd European Workshops on AD

Abstract: We will discuss a number of constructs or situations which are problematic for current AD systems. These fall into two categories. First, those which we know how to do in principle but which current systems cannot do, or cannot do conveniently. These would include, for example: iterate-to-fixed point loops, just-in-time compilation and AD, nested application of AD operators, high-level transformation of high-level routines, and miscellaneous simple transforms that we can perform manually but which current automated systems do not support. The second class of challenges are more fundamental: situations in which it seems clear that some sort of automatic transformation should allow calculation of derivatives, but where current derivations are ideosyncratic and not easily amenable to mechanisation. These include: annealed gradient systems like the stochastic Boltzmann Machine; EM and Generalised EM models; and systems using the so-called kernel trick, (eg Support Vector Machines, kernel PCA) for which current AD systems give the incorrect gradient.