4.1 Methods for Estimation of Partial Derivatives

The following approaches are commonly used to estimate derivatives for a model [16,29]:

(a) Variational Equations: When the model equations are available, one can, in principle, compute the derivatives from these equations by directly differentiating the equations and performing the necessary algebraic manipulations. This approach results in an additional set of equations that can be solved either in a coupled manner [200], or in a decoupled manner [60]. For lower dimensional, linear models, these equations can be solved directly along with the model equations to estimate the derivatives. However, for nonlinear functions the derivatives are generally more complicated than the function itself. Further, if the original model has $n$ outputs and $m$model parameters, in order to estimate the partial derivatives of all the outputs with respect to all the parameters, the number of equations required would be $m\times n$. Hence, this approach requires a considerable amount of effort for generating all the necessary model equations. Further, this approach requires access to the original model formulation equations.

(b) ``Brute-force'' approach: The two commonly used techniques in this approach are:

• Central differences: For the model represented by Equation 2.5, the partial derivative of $u_i$ with respect to a parameter $k_j$ at a point $(k_{1,0},\ldots,k_{j,0},\ldots,k_{m,0})$ is estimated by solving the model at two points $(k_{1,0},\ldots,k_{j,0}-h,\ldots,k_{m,0})$ and $(k_{1,0},\ldots,k_{j,0}+h,\ldots,k_{m,0})$, and the respective outputs are obtained as $u_{i,l}$ and $u_{i,r}$. The partial derivative is then given by
  
  $$\frac{\partial u_i}{\partial k_j} \approx\ \frac{u_{i,r} - u_{i,l}}{2h} \ ;\ h \rightarrow 0$$
  
  

• One-sided differences: Here, the model is solved at a point $(k_{1,0},\ldots,k_{j,0},\ldots,k_{m,0})$, resulting in output $u_{i,0}$ and either at the point $(k_{1,0},\ldots,k_{j,0}-h,\ldots,k_{m,0})$ or at the point $(k_{1,0},\ldots,k_{j,0}+h,\ldots,k_{m,0})$. The left-side derivative approximation is given by
  
  $$\frac{\partial u_i}{\partial k_j} \approx\ \frac{u_{i,0} - u_{i,l}}{h} \ ;\ h \rightarrow 0$$
  
  
  and the right-side derivative approximation is given by
  
  $$\frac{\partial u_i}{\partial k_j} \approx\ \frac{u_{i,r} - u_{i,0}}{h} \ ;\ h \rightarrow 0$$
  
  

In order to estimate the partial derivatives of outputs with respect to $m$parameters, the central difference method requires solution of the model equations $2m$ times (i.e., $2m$ ``model runs''), that is two times for each parameter considered. On the other hand, the one-sided difference method requires $m+1$ model runs, one for each parameter considered, and one run at the point under consideration.

The primary disadvantage of these methods is the computational cost associated with a large number of model runs. Another disadvantage with these methods involves the selection of a small enough value of $h$; a small $h$ results in ``roundoff errors'', resulting from subtracting two almost equal numbers, and a large value results in truncation errors, as the omitted higher order terms become significant.

(c) Symbolic Differentiation: When the model equations are mathematically tractable, symbolic manipulators such as Maple [94] and MACSYMA [95] can be used for manipulating the model equations and obtaining expressions for the partial derivatives. The practical limitations are numerous: (a) many computational models do not have mathematically tractable equations, (b) the symbolic manipulators are in general, unable to deal with constructs such as branches, loops or subroutines that are inherent in computer codes, and (c) the derivative code expressions could be very complex, because, for every operation, the derivative expression in essence doubles, leading to a combinatorial explosion [16].

(d) Automated Differentiation: Automated differentiation, also known as automatic differentiation, relies on the fact that every function is executed on a computer as a sequence of elementary operations such as additions, multiplications, and elementary functions. By applying the chain rule

$$\left.\frac{\partial }{\partial t}f(g(t))\right\vert _{t=t_0}... ...ft.\frac{\partial }{\partial t}g(t)\right\vert _{t=t_0}\right)$$

over and over again to the composition of those elementary operations, one can compute, in a completely mechanical fashion, derivatives of $f$ that are correct up to the machine precision [16]. This approach essentially consists of using a computational tool that can ``understand'' the model code and subsequently produce the code for estimating partial derivatives. The main advantage of this method is that the partial derivatives can be estimated using a single model run; another advantage is that this method can be used in conjunction with complex numerical models, that can be considered as ``black-box'' structures.

The automatic differentiation method has been widely applied in sensitivity analysis of transport/transformation modeling, as mentioned in Section 2.5.4. ADIFOR (Automatic DIfferentiation of FORtran) is used in this work for coupling with the SRSM. The following sections describe the ADIFOR methodology and the coupling with the SRSM.

  

Figure 4.2: Schematic illustration of the Automatic Differentiation method (adapted from Bischof et al., 1994)