DYNAMIC OPTIMISATION OF TILT-A-WHIRL

Christopher Iliffe Sprague


ABSTRACT

This paper examines the design optimization of a Tilt-A-Whirl ride, as an extension of Kautz’s Chaos at the amusement park: Dynamics of the Tilt-A-Whirl [?]. In particular, the task of optimizing passenger experience is analyzed and solved. The objective, the passenger’s enjoyment, is approximated by the standard deviation of each car’s intrinsic angular velocity, found through numerical integration. A Gaussian process surrogate model is used to approximate the system’s time dependent objective, in order to save on computationally intensive evaluations of the true objective function. A hybrid particle swarm algorithm is used to optimize the ride’s design parameters in order to maximize the passenger’s enjoyment. It is shown that a final objective value of over 200% better than the nominal, yielded from the parameters presented in [?], is achievable.

Index TermsAmusement Ride, Design Optimization, Particle Swarm, Gaussian Process

Contents

1 Background
 1.1 Parameters
  1.1.1 Constants
  1.1.2 Design
 1.2 Equation of Motion
 1.3 Numerical Integration
2 Optimization
 2.1 True Objective Function
 2.2 Problem Statement
 2.3 Surrogate model
  2.3.1 Sampling
  2.3.2 Gaussian Process
 2.4 Particle Swarm
3 Results
4 Source Code

List of Figures

Overhead
Three dimensional
Tangential
Radial
Views of the Tilt-A-Whirl model.
True Objective Function Sampling σ(ω)
ˆσ(ω) with r21 nominal.
ˆσ(r2) with ω,α1 nominal.
ˆσ(α1) with ω,r2 nominal.
Single Variable ˆ σ
ˆσ(r21) with ω nominal.
ˆσ(ω,α1) with r2 nominal.
ˆσ(ω,r2) with α1 nominal.
Nominal Multivariate ˆσ Topology
Hybrid Particle Swarm Convergence of ˆ σ with Ns = 100
Hybrid Particle Swarm Convergence of ˆ σ with Ns = 600
10 Sampling Resolution and Surrogate Objective Returns
ˆσ(r21) with ω optimal.
ˆσ(ω,α1) with r2 optimal.
ˆσ(ω,r2) with α1 optimal.
12 Optimal Multivariate ˆ σ Topology

Listings

1 Background
 1.1 Parameters
  1.1.1 Constants
  1.1.2 Design
 1.2 Equation of Motion
 1.3 Numerical Integration
2 Optimization
 2.1 True Objective Function
 2.2 Problem Statement
 2.3 Surrogate model
  2.3.1 Sampling
  2.3.2 Gaussian Process
 2.4 Particle Swarm
3 Results
4 Source Code

1.

1 Background

Following the work of [?], the dynamical model of a Tilta-A-Whirl is revisited. Like many other dynamical systems governed by deterministic equations, the Tilt-A-Whirl is found to exhibit chaotic behavior. In the dynamical model being examined, each of the Tilt-A-Whirl cars are free to pivot about the center of their own circular platform. The platforms move along a hilly circular track that tilts the platforms in all possible directions. While the translation and orientation of the platforms are completely predictable, each car whirls around in an independent and chaotic fashion.

1.1.

1.1 Parameters

1.1.1.

1.1.1 Constants

For this paper’s application, there are three design variables which are taken to be constant at the nominal values presented in [?], namely: the beam length from the center of the ride to the center of each platform r1 = 4.3 m, the radial incline offset of the platform α0 = 0.036 rad, and the parameter Q0 = (m ∕ρ)  ---
∘ gr32 = 20, where m is the mass of the car, ρ is a damping coefficient to account for friction, and g is the acceleration due to gravity.

1.1.2.

1.1.2 Design

For the design optimization problem analyzed in this paper, three parameters are chosen to be manipulable, namely: the extrinsic angular velocity of the platform ω = ˙
θ [0.3142,0.8378] rad∕s, the length of the radial arm from the center of each platform to the approximate center of gravity of the car r2 [0.1,1.5] m, and the maximum/minimum incline angle of the beam from the center of the ride to the platform α1 [0,0.3] rad. The nominal values of the design variables are given as {ω,r21}0 = {6.5 × (2π∕60) rad∕s,0.8 m,0.058 rad}. Both the constant parameters and variable design parameters can be seen within a clearer context in Figure 2. In an object-orientated manner, a Tilt-A-Whirl car can be programmatically instantiated with these nominal parameters and bounds, as shown in Listing 1. The remaining parameters are discussed in latter sections.

Listing 1: Nominal Tilt-A-Whirl Car
% Instantiate car with 
% nominal values. 
TW = Car(              ... 
    4.3,               ... r1 m 
    0.036,             ... alpha0 rad 
    20,                ... Q0 
    0.68,              ... omega rad/s 
    0.8,               ... r2 m 
    0.058,             ... alpha1 rad 
    [3, 8].⋆(2⋆pi/60), ... omega bounds rad/s 
    [0.1, 1.5],        ... r2 bounds m 
    [0,0.3],           ... alpha1 bounds rad 
    2000,              ... tau duration 
    100,               ... # of samples 
    0.05               ... noise level 
);


PIC Overhead

PIC Three dimensional

PIC Tangential

PIC Radial

Fig. 2: Views of the Tilt-A-Whirl model.

[?]


1.2.

1.2 Equation of Motion

From [?], the second order differential equation of motion governing the Tilt-A-Whirl is defined as

d2ϕ + -γ-dϕ-+ (ϵ - γ2α)sin (ϕ) + γ2β cos(ϕ) = 0
dτ2   Q0 dτ
(1)

, where the dimensionless parameters are defined as

     τ = 3ωt
       1-∘ -g-
   γ = 3ω  r2
         r1
     ϵ = 9r2

α = α0 - α1 cos(τ)
  β = 3α1sin (τ)
(2)

(3)

(4)

(5)

(6)
Programmatically, these equations of motion are transcribed as a first order system in Listing 2.
Listing 2: Equations of Motion
function dphi = Rate(self, tau, phi) 
    % Describes the Tilt-A-Whirl's 
    % second order ordinary differential 
    % equations of motion. 
    p         = phi(1); % p 
    dp        = phi(2); % p' 
    dphi(1,1) = dp;     % p'' 
    % The system dynamics from 
    % Eq. (27) of "Chaos at the amusement 
    % park: Dynamics of the Tilt-A-Whirl 
    % (Kautz and Huggard): 
    g     = 9.802; 
    gamma = 1/(3⋆self.omega); 
    gamma = sqrt(g/self.r2)gamma; 
    eps   = self.r1/(9⋆self.r2); 
    alpha = -self.alpha1cos(tau); 
    alpha = alpha+self.alpha0; 
    beta  = 3⋆self.alpha1sin(tau); 
    d2p   = -(eps-(gamma^2)alpha)sin(p); 
    d2p   = d2p-(gamma/self.Q0)dp; 
    d2p   = d2p-gamma^2⋆betacos(p); 
    dphi(2,1) = d2p; 
end

1.3.

1.3 Numerical Integration

Of course, because Equation 1 is rather nonlinear and chaotic, an analytical closed form solution does not exist. Thus, in order to observe the behavior of the system over time, it is necessary to numerically integrate the equation of motion. From a programmatic approach, one can implement Matlab’s ode45 [?], using the widely popular Runge-Kutta method of fourth and fifth order. From the object oriented appoach taken in this paper, this process manifests itself in the form of Listing 3.

Listing 3: Numerical Integration
function [phi, t] = Propogate(self) 
    % Propogates dynamics over over 
    % dimensionless time. 
    T        = self.T; 
    [t, phi] = ode45(@self.Rate,[0,T],[0,0]); 
end

2.

2 Optimization

It is supposed that the unpredictability of each car’s intrinsic angular velocity ˙ϕ is directly correlated to passenger enjoyment. Thus the objective measure of design performance is defined as the standard deviation of each car’s intrinsic angular velocity σ(dϕ)
 dt, where dϕ
dt is obtained through numerical integration, as discussed in Section 1.3.

2.1.

2.1 True Objective Function

Herein, the true objective function σ(   )
  dϕ-
  dt is referred to as a function of design parameters σ(ω, r,α )
   2  1, or less verbosely as σ. After numerical integration of the system’s dynamics, a large list of ϕ˙ data points are obtained, from which the computation of σ draws from. However, because the system’s dynamics, described in Equation 1, are with respect to the non-dimensional parameter τ, it is necessary to redimensionalize when computing the standard deviation of the car’s intrinsic angular velocity. The programmatic transcription of this objective function is seen in Listing 4.

Listing 4: True Objective Function
 function [stdp, phi, t] = StdPhi(self) 
    % Returns standard deviations of dphi/dt. 
    [phi,t] = self.Propogate(); 
    % Sample terminal dynamics only. 
    el      = length(phi);  % End index 
    tl      = round(el/3);  % Term index 
    y       = phi(tl:el,2); % Term sample 
    yb      = mean(y); 
    stdp    = mean((y-yb).^2); 
    stdp    = 3⋆self.omegasqrt(stdp); 
end

Note that only the last two thirds of the ϕ˙ data points are taken into account. This is done to disregard the start-up of the system, and only take heed to the terminal dynamics, so as to compute a purer standard deviation.

The nature of the true objective function σ can be seen by keeping α1 and r2 constant, while manipulating ω, as shown in Figure 3. Of course, the evaluation of this objective function is quite computationally expensive, as each evaluation elicits the numerical integration of the system’s equation of motion. It can also be seen that the true objective function is rather noisy, not lending itself well to gradient based optimizers, such as fmincon [?], at all. A conventional optimization algorithm would indeed become trapped within the multiple local extrema that are exhibited.

2.2.

2.2 Problem Statement

Taking note of the nominal constant parameter values and design parameter bounds described in Section 1.1, the statement of the optimization problem is more formally formulated as

miωn,imr2,αiz1e  σ (ω, r2,α1)
              60
subjectto  3 ≤ ω---≤ 8 rpm
              2π
         0.1 ≤ r2 ≤ 1.5m
         0 ≤ α1 ≤ 0.3rad
(7)


PIC

Fig. 3: True Objective Function Sampling σ(ω)


2.3.

2.3 Surrogate model

Because the true objective function σ is so noisy, chaotic, and computationally costly to evaluate, the appropriate course of action is to model the system’s time-dependent behavior with a surrogate model.

2.3.1.

2.3.1 Sampling

The true objective function is sampled at various design parameter values using Latin hypercube sampling, with the use of lhsdesign [?]. Latin hypercube sampling allows for near random sampling of the design space, to allow for the proper construction of the surrogate model. Programmatically this sampling is transcribed as shown in Listing 5.

Listing 5: Latin Hypercube Sampling
function [stphi, x] = LHS_Sampling(self) 
  % Define sample points with latin 
  % hyper cube sampling. 
  % # Variables 
  n = length(self.Design_Vector()); 
  s = self.samps; % # of samples 
  x = lhsdesign(s,n); 
  % Variable bounds 
  xl = [self.omegab(1); 
        self.r2b(1); 
        self.alpha1b(1)]; 
  xu = [self.omegab(2); 
        self.r2b(2); 
        self.alpha1b(2)]; 
  % Scaling according to bounds 
  for i = 1:n 
    x(:,i) = x(:,i).⋆(xu(i) - xl(i)); 
    x(:,i) = x(:,i) + xl(i); 
  end 
  % Evaluate objective function 
  % at LHS samples 
  ndes = length(x); % # designs 
  for i = 1:ndes 
    self.omega  = x(i,1); 
    self.r2     = x(i,2); 
    self.alpha1 = x(i,3); 
    stphi(i,1)  = self.StdPhi(); 
  end 
end

It is important to note that, initially, the sample values returned by lhsdesign are not scaled to the problem’s parameters; scaling must be done according the scale length of the problem’s bounds. The function shown in Listing 5 returns both the sample design parameter values and the true objective function values at those parameters. The reader is encouraged to consult [?] for a rigorous analysis of Latin hypercube sampling.

2.3.2.

2.3.2 Gaussian Process

Using the work of Rasmussen [?], a Gaussian process surrogate model is constructed to approximate the computationally expensive true objective function. The Gaussian process library is a machine learning library that addresses the supervised learning problem of mapping inputs to outputs, with previous knowledge of labelled training data. Using the Latin hypercube sampling described in Section 2.3.1, the surrogate model training process is instantiated in Listing 6, from which the Gaussian process hyperparameters are returned.

Using the true objective function samples, the surrogate model effectively learns the topology of the design space, through the training occurring during hyp = minimize(hyp, @gp, -100,@infExact,[],@covSEiso, @likGauss, xs, ys), in which the Gaussian process hyperparameters are minimized. Note that, in Listing 6, the hyperparameters and sampling points are saved, and used later unless the Gaussian process parameters have changed. This allows for significantly faster work-flow when testing different optimization algorithms.

Listing 6: Gaussian Process Hyperparameters
function [hyp, xs, ys] = GP_Hyper(self) 
  % Hyperparameters exist? 
  if exist('Hyper.mat','file') == 2 
    load('Hyper.mat'); 
    % Want a different sampling? 
    if (self.samps ~= length(xs) ... 
        || self.noise ~= ns      ... 
        || self.T ~= T) 
      disp('Differentsampling...'); 
      need = true; % Then sample again 
    else 
      need = false; % Otherwise don't 
    end 
  else 
    disp('Needsampling...'); 
    need = true; % No such file :( 
  end 
  if need == true 
    % Latin hyper cube sampling 
    [ys, xs]  = self.LHS_Sampling(); 
    % Set Matern covariance function 
    covfunc   = {@covMaterniso, 1}; 
    % hyp.cov = [log(1); lfilefileog(sigma)]; 
    hyp.cov   = log([1/4;1.0]); 
    % Gaussian likelihood function 
    likfunc   = @likGauss; 
    % Noise 
    ns = self.noise; 
    hyp.lik   = log(ns); 
    % Minimise negative log 
    % likelihood function 
    disp('Minimisingnegativeloglikelihoodfunction...'); 
    hyp = minimize(... 
      hyp, @gp, -100, @infExact, ... 
      [], @covSEiso, @likGauss, xs, ys ... 
    ); 
    T = self.T; 
    % Save for later 
    save('Hyper','hyp','xs','ys','ns','T') 
  end 
end

From the Gaussian process hyperparameters returned from Listing 6, the surrogate model in Listing 7 acts as a callable function, herein refereed to as ˆσ, that can be inexpensively evaluated in the optimization process. One major advantage of the surrogate objective function ˆσ in Listing 7 over the true objective function σ in Listing 4, is that it is smooth and is much nicer to optimize, especially with gradient based methods.

Listing 7: Surrogate Objective Function
function [fval,s2] = GP_Surrogate(self, x) 
  [hyp, xs, ys] = self.GP_Hyper(); 
  % Evaluate mean at variables 
  [fval, s2] = gp(                 ... 
    hyp, @infExact, [], @covSEiso, ... 
    @likGauss, xs, ys, x           ... 
  ); 
end

With the Gaussian process surrogate model transcribed in Listing 7, the single varriable design space topologies can be seen with 95% confidence bounds in Figure 5, with 200 Latin hypercube samples. Note the vastly smoother topology in Figure 5 compared that of Figure 3.


PIC ˆσ(ω) with r21 nominal.

PIC ˆσ(r2) with ω,α1 nominal.

PIC ˆσ(α1) with ω,r2 nominal.

Fig. 5: Single Variable ˆσ


With the use of function [X,Y,F] = Surface(self, cvar, np), the topology of the design space within the neighborhood of the nominal parameters can be seen in Figure 7. It can be seen that both ˆσ(r21) and ˆσ(ω,α1) are fairly nice look, while σˆ(ω,r2) seems rather treacherous.


PIC ˆσ(r21) with ω nominal.

PIC ˆσ(ω,α1) with r2 nominal.

PIC ˆσ(ω,r2) with α1 nominal.

Fig. 7: Nominal Multivariate ˆσ Topology


2.4.

2.4 Particle Swarm

Particle swarm is a population-based stochastic optimization algorithm. It solves an optimization problem by a population of possible solutions, otherwise known as particles, and moving the particles around in the design search space, according to the particles’ positions and velocities. Every particle’s movement is influenced by its best known local solution. However, the particles are also directed toward the best known positions in the design search space among other particles. In effect, this influences the swarm of particles to congregate upon the best solution. This optimization technique was inspired by the social behavior of bird flocking and fish school. The reader is encouraged to consult [?] for more information on this algorithm.

In order to implement this, Matlab’s particleswarm [?] function is used to optimize the surrogate function. This optimization algorithm is made particularly effective when parallelized on multiple CPU cores and augmented with the terminal use of a gradient based optimizer, such as fmincon [?]. Essentially the particle swarm optimization coarsely hones in on the global optimum, at which point a gradient based optimizer pinpoints the solution, assuming the design space is sufficiently convex within the vicinity of the solution.

The particle swarm optimization process can be intuitively commenced with [x,fval] = TW.Optimize('particle-swarm'). With a Gaussian process surrogate model constructed with 100 samples, an objective standard deviation value of ˆσ100 = 5.37419 is obtained, as shown with convergence history in Figure 8. With 600 true objective samples, a better objective of ˆσ = 6.17409 is obtained, as seen in Figure 9.


PIC

Fig. 8: Hybrid Particle Swarm Convergence of ˆσ with Ns = 100



PIC

Fig. 9: Hybrid Particle Swarm Convergence of ˆσ with Ns = 600


Objective values returned by the optimizer tend to get better as the resolution of the true objective function sampling increases, that is the number of samples Ns. However, it has been found that returns tend to decline as the resolution becomes very large. This is because, as the resolution of the surrogate function becomes greater, it exposes more detail of the true design space. Small details which may have been disregarded at lower sample values, become apparent, as shown in Figure 10, giving surrogate objective values of Ns = {10,100,200,400,600,1000,2000}↦→ˆσ = {3.5879,4.0953,5.2154,6.2128,6.5701,6.2818,5.5852}. However, the accuracy of the surrogate model and particle swarm optimization have been verified, yielding objective values over 200% better than the nominal, for almost all values of Ns. The Gaussian process surrogate model’s optimal design space topology can be seen in Figure 12.


PIC

Fig. 10: Sampling Resolution and Surrogate Objective Returns


3.

3 Results

Through the particle swarm optimization process outlined in Section 2.4, with Ns = 600, an objective surrogate function value of ˆσ = 6.1741 is obtained with design parameters {ω*,r2*1*} = {0.7703,0.1000,0.2716}. Evaluating the true objective function σ at these design parameters yields σ(ω*,r2*1*) = 6.6079, thereby further guaranteeing the accuracy of the Gaussian process surrogate model implemented, and demonstrating a colossal 312.99% improvement over the nominal σnom = 1.6. Interestingly, the optimal design parameter r2* tends to converge to its lower bound.


PIC ˆσ(r21) with ω optimal.

PIC ˆσ(ω,α1) with r2 optimal.

PIC ˆσ(ω,r2) with α1 optimal.

Fig. 12: Optimal Multivariate ˆσ Topology


4.

4 Source Code


Listing 8: Example Car Optimization
 
clear all; close all; 
% Instantiate car with 
% nominal values. 
TW = Car(              ... 
    4.3,               ... r1 m 
    0.036,             ... alpha0 rad 
    20,                ... Q0 
    0.68,              ... omega rad/s 
    0.8,               ... r2 m 
    0.058,             ... alpha1 rad 
    [3, 8].⋆(2⋆pi/60), ... omega bounds rad/s 
    [0.1, 1.5],        ... r2 bounds m 
    [0,0.3],           ... alpha1 bounds rad 
    2000,              ... tau 
    600,               ... # of samples 
    0.05               ... noise 
); 
 
[x,fval] = TW.Optimize('particle-swarm')

Listing 9: Car Class
 
classdef Car 
    properties 
        % Fixed parameters 
        r1; alpha0; Q0; 
        % Design parameters 
        omega; r2; alpha1; 
        % Bounds 
        omegab; r2b; alpha1b; 
        % Extra 
        T; samps; noise; 
    end 
    methods 
        function self = Car(      ... 
            r1, alpha0, Q0,       ... 
            omega, r2, alpha1,    ... 
            omegab, r2b, alpha1b, ... 
            T, samps, noise       ... 
        ) 
            % Fixed parameters 
            self.r1      = r1; 
            self.alpha0  = alpha0; 
            self.Q0      = Q0; 
 
            % Design parameters 
            self.omega   = omega; 
            self.r2      = r2; 
            self.alpha1  = alpha1; 
 
            % Bounds 
            self.omegab  = omegab; 
            self.r2b     = r2b; 
            self.alpha1b = alpha1b; 
 
            % Integration period 
            self.T       = T; 
            % # True function samples 
            self.samps   = samps; 
            % Amount of noise 
            self.noise   = noise; 
        end 
        function D = Design_Vector(self) 
            D = [self.omega, ... 
                 self.r2,    ... 
                 self.alpha1]; 
        end 
        function B = Bounds(self) 
          B = [self.omegab; ... 
               self.r2b;    ... 
               self.alpha1b]; 
        end 
        function dphi = Rate(self, tau, phi) 
            % Described the Tilt-A-Whirl's 
            % second order ordinary differential 
            % equations of motion. 
            p         = phi(1); % p 
            dp        = phi(2); % p' 
            dphi(1,1) = dp; % p'' 
            % The system dynamics from 
            % Eq. (27) of "Chaos at the amusement 
            % park: Dynamics of the Tilt-A-Whirl 
            % (Kautz and Huggard): 
            g     = 9.802; 
            gamma = 1/(3⋆self.omega); 
            gamma = sqrt(g/self.r2)gamma; 
            eps   = self.r1/(9⋆self.r2); 
            alpha = -self.alpha1cos(tau); 
            alpha = alpha+self.alpha0; 
            beta  = 3⋆self.alpha1sin(tau); 
            d2p   = -(eps-(gamma^2)alpha)sin(p); 
            d2p   = d2p-(gamma/self.Q0)dp; 
            d2p   = d2p-gamma^2⋆betacos(p); 
            dphi(2,1) = d2p; 
        end 
        function [phi, t] = Propogate(self) 
            % Propogates dynamics over over 
            % dimensionless time. 
            T        = self.T; 
            [t, phi] = ode45(@self.Rate,[0,T],[0,0]); 
        end 
        function [stdp, phi, t] = StdPhi(self) 
            % Returns standard deviations of dphi/dt. 
            [phi,t] = self.Propogate(); 
            % Sample terminal dynamics only. 
            el      = length(phi);  % End index 
            tl      = round(el/3);  % Term index 
            y       = phi(tl:el,2); % Term sample 
            yb      = mean(y); 
            stdp    = mean((y-yb).^2); 
            stdp    = 3⋆self.omegasqrt(stdp); 
        end 
        function [fval,s2] = GP_Surrogate(self, x) 
          [hyp, xs, ys] = self.GP_Hyper(); 
          % Evaluate mean at variables 
          [fval, s2] = gp(                 ... 
            hyp, @infExact, [], @covSEiso, ... 
            @likGauss, xs, ys, x           ... 
          ); 
        end 
        function [x,fval] = Optimize(self, alg) 
          fun = @(x) -self.GP_Surrogate(x); 
          % Initial guess 
          x0  = self.Design_Vector(); 
          nv = length(x0); 
          % Lower bounds 
          lb  = [self.omegab(1), ... 
                 self.r2b(1),    ... 
                 self.alpha1b(1)]; 
          % Upper bounds 
          ub  = [self.omegab(2), ... 
                 self.r2b(2),    ... 
                 self.alpha1b(2)]; 
          % Extraneous constraints 
          A   = []; b   = []; 
          Aeq = []; beq = []; nonlcon = []; 
          % Optimize the objective 
          disp('OptimisingGuassiansurrogate...') 
 
          if strcmp(alg,'simulated-annealing') 
            options = optimoptions(@simulannealbnd, ... 
              'PlotFcn',{@saplotbestf,@saplottemperature, ... 
                         @saplotf,@saplotstopping}); 
            [x,fval] = simulannealbnd(fun,x0,lb,ub, options); 
 
          elseif strcmp(alg,'particle-swarm') 
            options = optimoptions(@particleswarm, ... 
              'UseParallel', true, ... 
              'HybridFcn', @fmincon, ... 
              'PlotFcn', @pswplotbestf, ... 
              'Display', 'iter' ... 
            ); 
            [x,fval] = particleswarm(fun,nv,lb,ub, options); 
 
          elseif strcmp(alg,'genetic') 
            options = optimoptions(@ga, ... 
              'PlotFcn', @gaplotbestf, ... 
              'Display', 'iter', ... 
              'UseParallel', true ... 
            ); 
            [x,fval] = ga(fun,nv,A,b,Aeq,beq,lb,ub,nonlcon,options); 
          elseif strcmp(alg,'fmincon') 
            options = optimoptions(@fmincon, ... 
              'Display', 'iter', ... 
              'PlotFcn', {@optimplotfval,@optimplotx}, ... 
              'UseParallel', true, ... 
              'ScaleProblem', 'obj-and-constr' ... 
            ); 
            [x,fval] = fmincon(fun,x0,A,b,Aeq,beq,lb,ub,nonlcon,options); 
 
          end 
        end 
 
        function [hyp, xs, ys] = GP_Hyper(self) 
          % Hyperparameters exist? 
          if exist('Hyper.mat','file') == 2 
            load('Hyper.mat'); 
            % Want a different sampling? 
            if (self.samps ~= length(xs) ... 
                || self.noise ~= ns      ... 
                || self.T ~= T) 
              disp('Differentsampling...'); 
              need = true; % Then sample again 
            else 
              need = false; % Otherwise don't 
            end 
          else 
            disp('Needsampling...'); 
            need = true; % No such file :( 
          end 
 
          if need == true 
            % Latin hyper cube sampling 
            [ys, xs]  = self.LHS_Sampling(); 
            % Set Matern covariance function 
            covfunc   = {@covMaterniso, 1}; 
            % hyp.cov = [log(1); lfilefileog(sigma)]; 
            hyp.cov   = log([1/4;1.0]); 
            % Gaussian likelihood function 
            likfunc   = @likGauss; 
            % Noise 
            ns = self.noise; 
            hyp.lik   = log(ns); 
            % Minimise negative log 
            % likelihood function 
            disp('Minimisingnegativeloglikelihoodfunction...'); 
            hyp = minimize(... 
              hyp, @gp, -100, @infExact, ... 
              [], @covSEiso, @likGauss, xs, ys ... 
            ); 
            T = self.T; 
            % Save for later 
            save('Hyper','hyp','xs','ys','ns','T') 
          end 
        end 
        function [stphi, x] = LHS_Sampling(self) 
          % Define sample points with latin 
          % hyper cube sampling. 
          % # Variables 
          n = length(self.Design_Vector()); 
          s = self.samps; % # of samples 
          x = lhsdesign(s,n); 
          % Variable bounds 
          xl = [self.omegab(1); 
                self.r2b(1); 
                self.alpha1b(1)]; 
          xu = [self.omegab(2); 
                self.r2b(2); 
                self.alpha1b(2)]; 
          % Scaling according to bounds 
          for i = 1:n 
            x(:,i) = x(:,i).⋆(xu(i) - xl(i)); 
            x(:,i) = x(:,i) + xl(i); 
          end 
          % Evaluate objective function 
          % at LHS samples 
          ndes = length(x); % # designs 
          for i = 1:ndes 
            self.omega  = x(i,1); 
            self.r2     = x(i,2); 
            self.alpha1 = x(i,3); 
            stphi(i,1)  = self.StdPhi(); 
          end 
        end 
        function [X,Y,F] = Surface(self, cvar, np) 
          % The constant variable index 
          if strcmp(cvar,'omega') 
            ci = 1; 
          elseif strcmp(cvar,'r2') 
            ci = 2; 
          elseif strcmp(cvar,'alpha1') 
            ci = 3; 
          end 
          % Design variable bounds 
          B     = self.Bounds(); 
          ind   = [1,2,3]; 
          % Indicies of variables 
          ind   = ind(ind~=ci); 
          ix    = ind(1); 
          iy    = ind(2); 
          % Bounds of variables 
          B     = [B(ix,:); B(iy,:)]; 
          % Axes labels 
          axst  = {'$\omega$', '$r_2$', '$\alpha_1$'}; 
          axstx = axst(ix); 
          axsty = axst(iy); 
          % Generate grid 
          x     = linspace(B(1,1),B(1,2),np); 
          y     = linspace(B(2,1),B(2,2),np); 
          [X,Y] = meshgrid(x,y); 
          % Constant variable 
          vars = self.Design_Vector(); 
          z = vars(ci); 
          % Objective values 
          dim = size(X); 
          for j = 1:dim(2) 
            x = X(:,j); 
            y = Y(:,j); 
            z = ones(size(x)).⋆z; 
            d(:,ix) = x; 
            d(:,iy) = y; 
            d(:,ci) = z; 
            F(:,j) = self.GP_Surrogate(d); 
          end 
          surfc(X,Y,F); 
          xlabel(axstx,'Interpreter','LaTex'); 
          ylabel(axsty,'Interpreter','LaTex'); 
          zlabel('$\sigma(\frac{d\phi}{dt})$', ... 
            'Interpreter', 'LaTex') 
        end 
    end 
end