3.4 Step III: Estimation of Parameters in the Functional Approximation

The unknown coefficients in the polynomial chaos expansion ( $a_{ik\ldots}$'s) can be estimated by one of the following methods, depending on the complexity of the model:

• For cases in which the model is invertible, output metrics can be directly calculated as explicit functions of the srvs ($\xi_i$s); the expressions for input variables in terms of $\xi_i\,$s are substituted into the inverted model to obtain explicit functions for the output metrics in terms of $\xi_i\,$s. In such simple cases, there is no need to assume a functional form for the output metrics.
• For nonlinear operators with mathematically manipulable equations, the unknown coefficients in the polynomial chaos expansion can be determined by minimizing an appropriate norm of the residual, after substituting the transformed inputs into the model equations. Gelarkin's method [208] is commonly used with weight function corresponding to the expectation of the random variables [198].
• For cases in which the model equations are not easy to manipulate, or when the model is of ``black-box'' type, the unknown coefficients can be obtained by a collocation method [198,208]. This method imposes the requirement that the estimates of model outputs are exact at a set of selected collocation points, thus making the residual at those points equal to zero. The unknown coefficients are estimated by equating model outputs and the corresponding polynomial chaos expansion, at a set of collocation points in the parameter space; the number of collocation points should be equal to the number of unknown coefficients to be found. Thus, for each output metric, a set of linear equations results with the coefficients as the unknowns; these equations can be readily solved using standard linear solvers.

3.4.1 Deterministic Equivalent Modeling Method/Probabilistic Collocation Method (DEMM/PCM)

The Deterministic Equivalent Modeling Method (DEMM) [198] incorporates the polynomial chaos expansion technique, along with symbolic computation methods and compiler technology, to provide a prototype language for automated uncertainty analysis. This method uses the Galerkin's method to relate the input and output approximations for models with tractable equations. Depending on the complexity of the model, DEMM uses either the Galerkin's method or the Probabilistic Collocation Method (PCM), described here.

As a part of DEMM, the Probabilistic Collocation Method (PCM) [199,159], is used for applying DEMM in the case of black-box models. The coefficients of the polynomial chaos expansion are obtained using the model outputs at selected collocation points. Then, the next order polynomial chaos expansion is employed, and the solution process is repeated. If the estimates of the pdfs of the output metrics of concern, are approximately equal, the expansion is assumed to have converged; the higher order approximation is used to estimate the pdfs of output metrics.

In DEMM/PCM the collocation points are selected following the orthogonal collocation method suggested by Villadsen and Michelsen [208]. The collocation points correspond to the roots of the polynomial of one degree higher than the order of the polynomial chaos expansion. For one-dimensional problems, this method gives the same results as Galerkin's method [208], and hence is regarded as an ``optimal method''. This approach is adapted for multi-dimensional cases in DEMM [198]. For example, in order to solve for a two dimensional second order polynomial chaos expansion, the roots of the third order Hermite polynomial, $\sqrt{3}$,$-\sqrt{3}$ and 0 are used, hence the possible collocation points are (0,0), ($-\sqrt{3}$,0), (0,$\sqrt{3}$), (0,$-\sqrt{3}$), ($-\sqrt{3}$,$-\sqrt{3}$), ($\sqrt{3}$,$\sqrt{3}$), ($\sqrt{3}$,0), ($-\sqrt{3}$,$\sqrt{3}$) and ($\sqrt{3}$,$-\sqrt{3}$). There are nine possible collocation points, but from Equations 3.10 and 3.10 with $n$=2 and taking terms only up to $\Gamma_{2}$, there are only six unknowns. Similarly, for higher dimension systems and higher order approximations, the number of available collocation points is always greater than the number of collocation points needed, which introduces a problem of selecting the appropriate collocation points. In the absence of selection criteria at present, the collocation points are typically selected at random from the set of available points.

3.4.1.1 Limitations of DEMM/PCM

As noted above, the selection of the required collocation points in an efficient manner from the large number of possible points is not addressed in the present implementation of DEMM. In fact, as the number of degrees of freedom increases, the number of available collocation points increases exponentially. For example, for the case of $n=6$, using Equation 3.7, the number of collocation points required for second and third order expansions are 28 and 84 respectively. Since the collocation points are selected from the combinations of the roots of one order higher Hermite polynomial, the number of collocation points available for second and third order expansions are $3^6=729$ and $4^6=4096$, respectively - there are three roots for a third order Hermite polynomial (used for obtaining collocation points for a second order expansion), and there are six variables, hence the number of possible collocation points is $3^6$, and similarly $4^6$ for a third order expansion. As the number of inputs and the order of expansion increase, the number of available collocation points increases exponentially.

In principle, any choice of collocation points from the available ones, should give adequate estimates of the polynomial chaos expansion coefficients. However, different combinations of collocation points may result in substantially different estimates of the pdfs of output metrics; this poses the problem of the optimal selection of collocation points. Furthermore, in a computational setting, some of the collocation points could be outside the range of the algorithmic or numerical applicability of the model, and model results at these points cannot be obtained. These issues are addressed in the following section, where an algorithm for collocation point selection method is described.

Another shortcoming of the standard collocation technique suggested in DEMM is that sometimes the roots of the Hermite polynomials do not correspond to high probability regions and these regions are therefore not adequately represented in the polynomial chaos expansion. For example, for a third order approximation, the roots of the fourth order Hermite polynomial, namely $\sqrt{3+\sqrt{6}}, -\sqrt{3+\sqrt{6}}, \sqrt{3-\sqrt{6}}, \mbox{and} -\sqrt{3-\sqrt{6}}$ are used as the collocation points. However, the origin corresponds to the region of highest probability for a normal random variable. Hence, the exclusion of the origin as a collocation point could potentially lead to a poor approximation.

The limitations of DEMM/PCM are shown in the context of a case study involving a human Physiologically Based Pharmacokinetic (PBPK) Model, and the results are presented in Section 5.1.4.

  
3.4.2 Efficient Collocation Method (ECM)

This method addresses some limitations of DEMM/PCM. Here, the collocation points for the estimation of the coefficients of the polynomial chaos expansion are selected based on a modification of the standard orthogonal collocation method of [198,208]. The points are selected so that each standard normal random variable $\xi_i$ takes the values of either zero or of one of the roots of the higher order Hermite-polynomial.

A simple heuristic technique is used to select the required number of points from the large number of potential candidates: for each term of the series expansion, a ``corresponding'' collocation point is selected. For example, the collocation point corresponding to the constant is the origin; i.e., all the standard normal variables ($\xi$'s) are set to value zero. For terms involving only one variable, the collocation points are selected by setting all other $\xi$'s to zero value, and by letting the corresponding variable take values as the roots of the higher order Hermite polynomial. For terms involving two or more random variables, the values of the corresponding variables are set to the values of the roots of the higher order polynomial, and so on. If more points ``corresponding'' to a set of terms are available than needed, the points which are closer to the origin are preferred, as they fall in regions of higher probability. Further, when there is still an unresolved choice, the collocation points are selected such that the overall distribution of the collocation points is more symmetric with respect to the origin. If still, more points are available, the collocation point is selected randomly.

The advantage of this method is that the behavior of the model is captured reasonably well at points corresponding to regions of high probability. Furthermore, singularities can be avoided in the resultant linear equations for the unknown coefficients, as the collocation points are selected to correspond to the terms of the polynomial chaos expansion. Thus, the pdfs of the output metrics are likely to be approximated better than by a random choice of collocation points, while following a technique similar to the ``orthogonal collocation'' approach.

3.4.2.1 Applicability and Limitations of ECM

The ECM method was applied to a case study involving a human Physiologically Based Pharmacokinetic (PBPK) model, which is presented in Section 5.1.4. This method resulted in accurate estimates of the output pdfs while requiring much fewer model simulations as compared to standard Monte Carlo methods, as shown in Figures 5.4-5.7 in Section 5.1.4.

The ECM method was applied to a more complex model, the two-dimensional atmospheric photochemical plume model, the Reactive Plume Model, version 4, (RPM-IV), as described in Section 5.2.3. For that case study, the ECM method failed to converge for a third order polynomial chaos approximation. This is attributed to the inherent instability of the collocation based approaches, as described by Atkinson [6].

3.4.3 Regression Based SRSM

Collocation methods are inherently unstable, especially with polynomial approximations of high orders, since the requirement that the polynomial (curve or a surface) has to pass through all the collocation points. Thus any one collocation point in the model space could significantly alter the behavior of the polynomial [6]. In this context, regression-based methods provide slightly more computationally expensive, but robust means of estimation of coefficients of the functional approximation, because model results at more points are used in this approach, and the influence of each sample point is moderated by all other sample points. This extension to the collocation method uses regression in conjunction with an improved input sampling scheme to estimate the unknown coefficients in the polynomial expansion. In the regression based method, a set of sample points is selected in the same manner as the ECM method, as described earlier. The number of sample points selected must be higher than the number of unknown coefficients to be estimated; selecting a number of points equaling twice the number of coefficients is recommended for obtaining robust estimates, and is the approach used in the case studies presented in this work. The model outputs at the selected sample points are equated with the estimates from the series approximation, resulting in a set of linear equations with more equations than unknowns. This system of equations is then solved using the singular value decomposition method [163].

The regression method has the advantage that it is more robust than the collocation method, but the method requires a higher number of model simulations to obtain estimates of output uncertainty. The advantages of this method are demonstrated by comparing the results of this method with those of ECM for a the case study involving RPM-IV, an atmospheric photochemical plume model, described in Section 5.2.3.

Based on preliminary case studies (presented in Chapter 5), the regression based method is found to be robust and the method resulted in good approximations of model outputs. Hence this method is recommended for use in general. The higher computational burden, compared to the ECM, is offset by the robustness this method provides for many cases. Throughout the rest of this document, the ECM method is denoted as ``ECM'' and the regression based method is denoted simply as ``SRSM''. Unless otherwise specified, the term SRSM in the case studies refers to a regression based SRSM.