Abstract

In the context of experimental design, achieving accurate and robust predictions across a range of conditions is crucial. Traditional optimality criteria like D-optimality focus on minimizing the determinant of the covariance matrix of the parameter estimates, which is useful for precise estimation of model parameters. However, when the primary goal is to ensure that predictions made by the model are reliable across the entire design space, G-optimality becomes the criterion of choice. G-optimality aims to minimize the maximum prediction variance over the design space, thereby ensuring the worst-case prediction variance is as low as possible. The current study does a comparison of various second order rotatable designs (SORDS) constructed using trigonometric functions on their G-optimality criteria.

1. Introduction

Optimal designs are experimental setups created based on specific criteria to optimize particular statistical models. The exploration of optimality in experimental design dates back to 1918 when Smith first proposed a criterion for regression problems. In 1959, Kiefer developed valuable computational methods for finding optimal designs in regression problems related to statistical inference.

An optimality criterion measures the quality of a design. There are many such criteria, often referred to as alphabetical optimality criteria, which can be categorized into four types: information-based criteria, distance-based criteria, compound design criteria, and other criteria ¹.

When fitting a statistical model to experimental data, the predictions made by the model have an associated variance, indicating how much the predicted values are expected to fluctuate around the true values. G-optimality is a criterion used in the design of experiments to minimize the maximum prediction variance over the design space ². This criterion is crucial when the primary objective is to ensure accurate predictions across the entire range of experimental conditions. A design is considered G-optimal if it minimizes the maximum prediction variance within the design space, making it useful for reliable predictions across a broad range of conditions ³.

G-optimality is a well-established criterion in experimental design, extensively studied for its theoretical foundations, computational methods, and applications. The literature on G-optimality spans several decades and includes seminal works that have established its theoretical underpinnings, as well as recent studies focusing on computational techniques and practical applications.

Jack Kiefer's seminal paper, "Optimum Experimental Designs" ⁴, introduced the concept of optimal design theory, including G-optimality. Kiefer's work laid the foundation for developing various optimality criteria and their mathematical formulations, significantly advancing the field of experimental design. Building on this foundation, Fedorov's "Theory of Optimal Experiments" ⁵ provided a comprehensive treatment of optimal experimental design theory, including a detailed discussion of G-optimality and other criteria. Additionally, Silvey's book "Optimal Design" ⁶ covers the theory and applications of various optimality criteria, including G-optimality, and offers insights into their practical implementation.

In computational methods, Mitchell's "An Algorithm for the Construction of 'D-optimal' Experimental Designs" ⁷ introduced algorithms adaptable for G-optimal design construction, despite being primarily focused on D-optimality. Cook and Nachtsheim's "A Comparison of Algorithms for Constructing Exact D-optimal Designs" ⁶ compares various algorithms for constructing D-optimal designs and discusses their adaptation to G-optimal designs. Meyer and Nachtsheim's "The Coordinate-Exchange Algorithm for Constructing Exact Optimal Experimental Designs" ⁸ introduced the coordinate-exchange algorithm, a versatile method for constructing exact optimal designs, including G-optimal designs. This algorithm has been widely used in practical applications, demonstrating its efficacy and flexibility.

The applications of G-optimality are well-documented. Atkinson, Donev, and Tobias's "Optimum Experimental Designs, with SAS" ⁹ offers a modern treatment of optimal design theory with practical examples and implementations using SAS, including discussions on G-optimality. Borkowski's "Designs with Minimal Variance of Prediction for Response Surfaces" ¹⁰ focuses on the application of G-optimality in response surface methodology. Giovagnoli and Wynn's "G-optimality of Experimental Designs for Generalized Linear Models" ¹¹ extends the concept of G-optimality to generalized linear models. Dette and Pepelyshev's "Generalized E- and G-optimal Designs for General Regression Models" ¹² explores generalized versions of E- and G-optimality for a broader class of regression models. Lastly, Jones and Goos's "A Candidate-set-free Algorithm for Generating D-optimal Designs" ¹³ introduces a novel algorithm that can be adapted for G-optimal design generation, highlighting the ongoing development of computational methods in the field.

The literature on G-optimality encompasses foundational theoretical works, computational methods, and practical applications. Seminal contributions by researchers such as Kiefer, Fedorov, and Silvey have established the theoretical basis for G-optimality, while recent studies have advanced computational techniques and explored new applications. G-optimal designs continue to be a valuable tool in ensuring uniform prediction accuracy in various scientific and engineering fields, reflecting their importance and relevance in modern experimental design. The goal of this paper is to compare five second-order rotatable designs in three dimensions constructed by Cornelious ^{14, 15, 16} on G-Optimality and efficiencies.

2. Methodology

A second-order rotatable design is achieved if sets of points are combined in such a way that the following conditions established by Box and Hunter ¹⁷ are met:

The sum of the squares of each design variable (for all design points ) is equal to

(1)

The sum of the fourth powers of each design variable and three times the product of the squares of different design variables and (for all design points ) are equal to

(2)

All other sums of powers and products up to and including order four are zero.

All summations are over to

The point set is said to form a second-order rotatable design if the conditions above are satisfied and a certain matrix used in a consequent least squares estimation is non-singular.

Box and Hunter ¹⁷ show that the necessary and sufficient condition for this to be achieved is:

(3)

The aim of G-optimality criterion; is to minimize the maximum prediction variance in the

design region. This minimization is given by; 𝑜𝑝𝑡 = 𝑚𝑖𝑛 [ ] (4)

Where ) (5) (6)

The information matrix from (7) is defined as;

Where,

And the and and in (8) are as explained in (1) and (2)

The G- efficiency of a design is defined as

(9)

Where is the number of parameters in the model and is maximum scaled variance of prediction.

The variance of the function at according to ⁸ is,

(10)

Where is the scaled prediction variance for any point in the design region,

Thus (11)

The vector is the row vector of the design matrix associated with the design point .

G- efficiency thus examines the maximum value of within the design region with respect to its theoretical minimum variance Therefore , a G optimality and the corresponding G-Efficiency emphasize the use of designs for which the maximum in the region of the design is not too large.

3. Results

Substituting the values of and for the 35 points on (8) and (4) gives,

For the set S (p, p,0), where the value of ;

For the set S (p, p,0),

For the set (r cos t𝛼, 𝑟 𝑠𝑖𝑛 𝑡𝛼, 0),

Of the three values, 90.9972 is the maximum, thus the G-criterion for the 35 points SORD is given by 90.9972.

from (6),

Substituting the values of and for the 36 points on (8) and (4) gives,

For the set ;

For the set S (p, p, 0);

Of the two values, 240.8 is the maximum, thus the G-criterion for the 36 points SORD is given by 240.8.

from (6),

Substituting the values of and for the 39 points on (8) and (4) gives,

+0.0002)*10^6

For the set );

For the set ;

Of the two values, 1400 is the maximum, thus the G-criterion for the 39 points SORD is given by 1400

From (6),

Substituting the values of and for the 42 points on (8) and (4) gives,

For the set ;

For the set ;

Of the two values, 93.3889 is the maximum, thus the G-criterion for the 42 points SORD is given by 93.3889. From (6),

Substituting the values of and for the 45 points on (8) and (4) gives,

For the set S (p, q,0)

for the set

Of the two values, 710 is the maximum, thus the G-criterion for the 45 points SORD is given by 710. From (6),

4. Discussion

The study of five designs with varying numbers of design points, ranging from 35 to 45, focused on assessing their performance based on the G-Criterion, which aims to minimize maximum prediction variance across experimental designs. Among these designs, the 42-point configuration emerged with the most favorable outcome, exhibiting the lowest maximum prediction variance. Conversely, the design comprising 39 points displayed the highest prediction variance among the group. Detailed data on prediction variances are provided in Table 1, illustrating these comparative results.

One notable finding from the analysis is that the prediction variance observed in these designs does not strictly correlate with the total number of points used but rather hinges on the specific arrangement and combination of these points within the second-order rotatable design framework. This underscores the critical role of design structure and point distribution in influencing prediction accuracy and variability.

Efficiency assessments further elucidated the performance disparities across the designs. The 35-point second-order rotatable design was identified as the most efficient, achieving an efficiency score of 10.99%. In contrast, the 39-point design exhibited the lowest efficiency at 0.71%, indicating its relative ineffectiveness in minimizing prediction variance compared to the other designs. The 42-point design closely followed as the second most efficient at 10.71%, while the 45-point and 36-point designs yielded efficiencies of 1.4% and 4.15%, respectively.

In conclusion, the study underscores the nuanced relationship between design point allocation, prediction variance, and efficiency in second-order rotatable designs. While the number of points plays a role, the configuration and selection of these points within the design framework significantly impact the overall predictive performance. These insights are crucial for optimizing experimental setups to ensure robust and reliable statistical inference across diverse scientific and engineering applications.

5. Conclusion

In practice, achieving G- optimality involves computational techniques to evaluate and optimize the design, ensuring that the chosen set of experimental runs provides the most robust predictions across the entire design space. This often requires iterative algorithms and can be computationally intensive, especially for large and complex design spaces.

G-Optimality is a crucial concept in experimental design that helps researchers and engineers develop experiments that provide reliable and accurate predictions across the full range of conditions being studied.

In this study, it is evident that G-optimality criteria and efficiency does not depend on the number of points (number of runs of the experiment.), it rather depends on the sets of points combined to form the SORDS.

References