API

BayesianUDE

Basic data structure used to the model structure, parameters and data for Bayesian UDE and NODE models. ...

Elements

• times: a vector of times for each observation
• data: a matrix of observations at each time point
• X: a DataFrame with any covariates used by the model
• data_frame: a DataFrame with columns for the time of each observation and values of the state variables
• parameters: a ComponentArray that stores model parameters
• loss_function: the loss function used to fit the model
• process_model: a Julia mutable struct used to define model predictions
• process_loss: a Julia mutable struct used to measure the performance of model predictions
• observation_model: a Julia mutable struct used to predict observations given state variable estimates
• observation_loss: a Julia mutable struct used to measure the performance of the observation model
• process_regularization: a Julia mutable struct used to store data needed for process model regularization
• observation_regularization: a Julia mutable struct used to store data needed for observation model regularization
• constructor: A function that initializes a UDE model with identical structure.

...

LossFunction

A Julia mutable struct that stores the loss function and parameters. ...

Elements

• parameters: ComponentArray
• loss: Function

...

MultiProcessModel

A Julia mutable struct that stores the functions and parameters for the process model. ...

Elements

• parameters: ComponentArray
• predict: Function that predicts one time step ahead
• forecast: Function that is a modified version of predict to improve performace when extrapolating
• covariates: Function that returns the values of the covariates at each point in time
• right_hand_side: Function that returns the right-hand side of a differential equation (i.e., the relationships between state variables and parameters)

...

MultiTimeSeriesNetwork(inputs,series,outputs; kwargs...)

Builds a neural network object and returns randomly initialized parameter values. The neural network object can be evaluated as a function that takes three arguments: a vector x, an integer i, and a named tuple with the neural network weights and biases parameters.

# kwargs

• hidden: the number of neurons in the hidden layer. The default is 10.
• nonlinearity: the activation function used in the neural network. The default is the hyperbolic tangent function tanh.
• seed : random number generator seed for initializing the neural network weights.
• distance : determines the level of difference between the functions estimated for each time series. The default is 1.0.

ProcessModel

A Julia mutable struct that stores the functions and parameters for the process model. ...

Elements

• parameters: ComponentArray
• predict: Function the predict one time step ahead
• forecast: Function, a modified version of predict to improve performance when extrapolating
• covariates: Function that returns the value of the covariates at each point in time.

...

Regularization

A Julia mutable struct that stores the loss function and parameters. ...

Elements

• reg_parameters: ComponentArray
• loss: Function

...

SimpleNeuralNetwork(inputs,outputs; kwargs ...)

Builds a neural network object and returns randomly initialized parameter values. The neural network object can be evaluated as a function that takes two arguments: a vector x and a named tuple with the neural network weights and biases parameters.

# kwargs

• hidden: the number of neurons in the hidden layer. The default is 10.
• nonlinearity: the activation function used in the neural network. The default is the hyperbolic tangent function tanh.
• seed: random number generator seed for initializing the neural network weights.

UDE

Basic data structure used to the model structure, parameters and data for UDE and NODE models. ...

Elements

• times: a vector of times for each observation
• data: a matrix of observations at each time point
• X: a DataFrame with any covariates used by the model
• data_frame: a DataFrame with columns for the time of each observation and values of the state variables
• parameters: a ComponentArray that stores model parameters
• loss_function: the loss function used to fit the model
• process_model: a Julia mutable struct used to define model predictions
• process_loss: a Julia mutable struct used to measure the performance of model predictions
• observation_model: a Julia mutable struct used to predict observations given state variable estimates
• observation_loss: a Julia mutable struct used to measure the performance of the observation model
• process_regularization: a Julia mutable struct used to store data needed for process model regularization
• observation_regularization: a Julia mutable struct used to store data needed for observation model regularization
• constructor: A function that initializes a UDE model with identical structure.
• timecolumnname: A string with the name of the column used for
• weights
• variablecolumnname
• valuecolumnname

...

 BFGS!(UDE, kwargs ...)

minimizes the loss function of the UDE model using the BFGS algorithm is the inital step norm equal to initial_step_norm. The funciton will print the value fo the loss function after each iteration when verbose is true.

kwargs

• initial_step_norm: Initial step norm for BFGS algorithm. Default is 0.01.
• verbose: Should the training loss values be printed?. Default is false.

BayesianCustomDerivatives(data::DataFrame,X,derivs!::Function,initial_parameters;kwargs ... )

When a data frame X is supplied the model will run with covariates. The argument X should have a column for time t with the value for time in the remaining columns. The values in X will be interpolated with a linear spline for value of time not included in the data frame.

When X is provided the derivs function must have the form derivs!(du,u,x,p,t) where x is a vector with the value of the covariates at time t.

kwargs

• time_column_name: Name of column in data and X that corresponds to time. Default is "time".
• variable_column_name: Name of column in X that corresponds to the variables. Default is nothing.
• value_column_name: Name of column in X that corresponds to the covariates. Default is nothing.
• hidden_units: Number of neurons in hidden layer. Default is 10.
• seed: Fixed random seed for repeatable results. Default is 1.
• proc_weight: Weight of process error $omega_{proc}$. Default is 1.0.
• obs_weight: Weight of observation error $omega_{obs}$. Default is 1.0.
• reg_weight: Weight of regularization error $omega_{reg}$. Default is 10^-6.
• reg_type: Type of regularization, whether "L1" or "L2" regularization. Default is "L2".
• l: Extrapolation parameter for forecasting. Default is 0.25.
• extrap_rho: Extrapolation parameter for forecasting. Default is 0.0.

BayesianCustomDerivatives(data::DataFrame,derivs!::Function,initial_parameters;kwargs ... )

Constructs a Bayesian UDE model for the data set data based on user defined derivatives derivs. An initial guess of model parameters are supplied with the initial_parameters argument.

...

Arguments

• data: a DataFrame object with the time of observations in a column labeled t and the remaining columns the value of the state variables at each time point.
• derivs: a Function of the form derivs!(du,u,p,t) where u is the value of the state variables, p are the model parameters, t is time, and du is updated with the value of the derivatives
• init_parameters: A NamedTuple with the model parameters. Neural network parameters must be listed under the key NN.

kwargs

• time_column_name: Name of column in data that corresponds to time. Default is "time".
• hidden_units: Number of neurons in hidden layer. Default is 10.
• seed: Fixed random seed for repeatable results. Default is 1.
• proc_weight: Weight of process error $omega_{proc}$. Default is 1.0.
• obs_weight: Weight of observation error $omega_{obs}$. Default is 1.0.
• reg_weight: Weight of regularization error $omega_{reg}$. Default is 10^-6.
• reg_type: Type of regularization, whether "L1" or "L2" regularization. Default is "L2".
• l: Extrapolation parameter for forecasting. Default is 0.25.
• extrap_rho: Extrapolation parameter for forecasting. Default is 0.0.

...

BayesianNODE(data,X;kwargs ... )

When a data frame X is supplied the model will run with covariates. The argument X should have a column for time t with the value for time in the remaining columns. The values in X will be interpolated with a linear spline for values of time not included in the data frame.

kwargs

• time_column_name: Name of column in data and X that corresponds to time. Default is "time".
• variable_column_name: Name of column in X that corresponds to the variables. Default is nothing.
• value_column_name: Name of column in X that corresponds to the covariates. Default is nothing.
• hidden_units: Number of neurons in hidden layer. Default is 10.
• seed: Fixed random seed for repeatable results. Default is 1.
• proc_weight: Weight of process error $omega_{proc}$. Default is 1.0.
• obs_weight: Weight of observation error $omega_{obs}$. Default is 1.0.
• reg_weight: Weight of regularization error $omega_{reg}$. Default is 10^-6.
• reg_type: Type of regularization, whether "L1" or "L2" regularization. Default is "L2".
• l: Extrapolation parameter for forecasting. Default is 0.25.
• extrap_rho: Extrapolation parameter for forecasting. Default is 0.0.

BayesianNODE(data;kwargs ... )

Constructs a Bayesian continuous-time model for the data set data using a single layer neural network to represent the system's dynamics.

kwargs

• time_column_name: Name of column in data that corresponds to time. Default is "time".
• hidden_units: Number of neurons in hidden layer. Default is 10.
• seed: Fixed random seed for repeatable results. Default is 1.
• proc_weight: Weight of process error $omega_{proc}$. Default is 1.0.
• obs_weight: Weight of observation error $omega_{obs}$. Default is 1.0.
• reg_weight: Weight of regularization error $omega_{reg}$. Default is 10^-6.
• reg_type: Type of regularization, whether "L1" or "L2" regularization. Default is "L2".
• l: Extrapolation parameter for forecasting. Default is 0.25.
• extrap_rho: Extrapolation parameter for forecasting. Default is 0.0.

CustomDerivatives(data,derivs!,initial_parameters;kwargs ... )

Constructs a UDE model for the data set data based on user-defined derivatives derivs. An initial guess of model parameters are supplied with the initial_parameters argument.

...

Arguments

• data: a DataFrame object with the time of observations in a column labeled t and the remaining columns the value of the state variables at each time point.
• derivs: a Function of the form derivs!(du,u,p,t) where u is the value of the state variables, p are the model parameters, t is time, and du is updated with the value of the derivatives
• init_parameters: A NamedTuple with the model parameters. Neural network parameters must be listed under the key NN.

kwargs

• time_column_name: Name of column in data that corresponds to time. Default is "time".
• proc_weight: Weight of process error $omega_{proc}$. Default is 1.0.
• obs_weight: Weight of observation error $omega_{obs}$. Default is 1.0.
• reg_weight: Weight of regularization error $omega_{reg}$. Default is 10^-6.
• reg_type: Type of regularization, whether "L1" or "L2" regularization. Default is "L2".
• l: Extrapolation parameter for forecasting. Default is 0.25.
• extrap_rho: Extrapolation parameter for forecasting. Default is 0.0.

...

CustomDerivatives(data::DataFrame,X::DataFrame,derivs!::Function,initial_parameters;kwargs ... )

When a data frame X is supplied the model will run with covariates. The argument X should have a column for time t with the value for time in the remaining columns. The values in X will be interpolated with a linear spline for values of time not included in the data frame.

When X is provided the derivs function must have the form derivs!(du,u,x,p,t) where x is a vector with the value of the covariates at time t.

kwargs

• time_column_name: Name of column in data and X that corresponds to time. Default is "time".
• variable_column_name: Name of column in X that corresponds to the variables. Default is nothing.
• value_column_name: Name of column in X that corresponds to the covariates. Default is nothing.
• proc_weight: Weight of process error $omega_{proc}$. Default is 1.0.
• obs_weight: Weight of observation error $omega_{obs}$. Default is 1.0.
• reg_weight: Weight of regularization error $omega_{reg}$. Default is 10^-6.
• reg_type: Type of regularization, whether "L1" or "L2" regularization. Default is "L2".
• l: Extrapolation parameter for forecasting. Default is 0.25.
• extrap_rho: Extrapolation parameter for forecasting. Default is 0.0.

CustomDerivatives(data::DataFrame,derivs!::Function,initial_parameters,priors::Function;kwargs ... )

When a function's priors is supplied its value will be added to the loss function as a penalty term for user specified parameters. It should take the a single NamedTuple p as an argument penalties for each paramter should be calculated by accessing p with the period operator.

The prior function can be used to nudge the fitted model toward prior expectations for a parameter value. For example, the following function increases the loss when a parameter p.r has a value other than 1.5, nad a second parameter p.beta is greater than zeros.

function priors(p)
    l = 0.01*(p.r - 1.5)^2
    l += 0.01*(p.beta)^2
    return l
end

kwargs

• time_column_name: Name of column in data that corresponds to time. Default is "time".
• proc_weight: Weight of process error $omega_{proc}$. Default is 1.0.
• obs_weight: Weight of observation error $omega_{obs}$. Default is 1.0.
• reg_weight: Weight of regularization error $omega_{reg}$. Default is 10^-6.
• reg_type: Type of regularization, whether "L1" or "L2" regularization. Default is "L2".
• l: Extrapolation parameter for forecasting. Default is 0.25.
• extrap_rho: Extrapolation parameter for forecasting. Default is 0.0.

CustomDifference(data,step,initial_parameters;kwrags...)

Constructs a UDE model for the data set data based on user defined difference equation step. An initial guess of model parameters are supplied with the initial_parameters argument. ...

Arguments

• data: a DataFrame object with the time of observations in a column labeled t and the remaining columns the value of the state variables at each time point.
• step: a Function of the form step(u,t,p) where u is the value of the state variables, p are the model parameters.
• init_parameters: A NamedTuple with the model parameters. Neural network parameters must be listed under the key NN.

kwargs

• time_column_name: Name of column in data that corresponds to time. Default is "time".
• proc_weight: Weight of process error $omega_{proc}$. Default is 1.0.
• obs_weight: Weight of observation error $omega_{obs}$. Default is 1.0.
• reg_weight: Weight of regularization error $omega_{reg}$. Default is 10^-6.
• reg_type: Type of regularization, whether "L1" or "L2" regularization. Default is "L2".
• l: Extrapolation parameter for forecasting. Default is 0.25.
• extrap_rho: Extrapolation parameter for forecasting. Default is 0.0.

...

CustomDifference(data::DataFrame,X,step,initial_parameters;kwargs ... )

When a data frame X is supplied the model will run with covariates. The argument X should have a column for time t with the value for time in the remaining columns. The values in X will be interpolated with a linear spline for value of time not included in the data frame.

When X is provided the step function must have the form step(u,x,t,p) where x is a vector with the value of the covariates at time t.

# kwargs

• time_column_name: Name of column in data and X that corresponds to time. Default is "time".
• variable_column_name: Name of column in X that corresponds to the variables. Default is nothing.
• value_column_name: Name of column in X that corresponds to the covariates. Default is nothing.
• proc_weight: Weight of process error $omega_{proc}$. Default is 1.0.
• obs_weight: Weight of observation error $omega_{obs}$. Default is 1.0.
• reg_weight: Weight of regularization error $omega_{reg}$. Default is 10^-6.
• reg_type: Type of regularization, whether "L1" or "L2" regularization. Default is "L2".
• l: Extrapolation parameter for forecasting. Default is 0.25.
• extrap_rho: Extrapolation parameter for forecasting. Default is 0.0.

CustomDifference(data::DataFrame,step,initial_parameters,priors::Function;kwargs ... )

When a function's priors is supplied its value will be added to the loss function as a penalty term for user-specified parameters. It should take the a single NamedTuple p as an argument penalties for each parameter should be calcualted by accessing p with the period operator.

function priors(p)
    l = 0.01*(p.r - 1.5)^2
    l += 0.01*(p.beta)^2
    return l
end

kwargs

• time_column_name: Name of column in data that corresponds to time. Default is "time".
• proc_weight: Weight of process error $omega_{proc}$. Default is 1.0.
• obs_weight: Weight of observation error $omega_{obs}$. Default is 1.0.
• reg_weight: Weight of regularization error $omega_{reg}$. Default is 10^-6.
• reg_type: Type of regularization, whether "L1" or "L2" regularization. Default is "L2".
• l: Extrapolation parameter for forecasting. Default is 0.25.
• extrap_rho: Extrapolation parameter for forecasting. Default is 0.0.

CustomModel(data::DataFrame,X::DataFrame, derivs!::Function, initial_parameters; kwargs ...)

Constructs a UDE model from a DataFrame data a DataFrame with covariates X a function derivs and an initial guess of the paramter values inital_parameters. The modle strucutre can be further modified by the key word arguments to specify the relationship between the state variables and observations and the loss function. These areguments are discussed individuals below.

kwargs

link - A function that takes the value of the state variable u and paramters p and retuns and estiamte of the obseration y linkparams - parameters for the link function, can be an empty NamedTuple if no paramters are used observationloss - loss function that describes the distance betwen the observed and estimated states observationparams - parameters for the obseraiton loss function - can be an empty named tuple of no paramters are needed processloss - loss funciton that describes the distance betwen the observed and predicted state tranistions. processlossparams - parameters for the process loss. statevariabletransform - a function that maps from the variables used in the optimizer to states variables used by the observaiton and prediction funitons. logpriors - prior probabilities for the model paramters + nerual network regularization timecolumnname - column that indexes time in the data frames valuecolumnname - the column that indicates the variabe in long formate covariates data sets variablecolumnname = the column that indicates the value of the variables in long formate covariates data sets regweight - weight given to regualrizing the neural network reg_type - funcrional form of regualrization "L1" or "L2"

EasyNODE(data,X;kwargs ... )

When a data frame X is supplied the model will run with covariates. The argument X should have a column for time t with the value for time in the remaining columns. The values in X will be interpolated with a linear spline for values of time not included in the data frame.

kwargs

• time_column_name: Name of column in data and X that corresponds to time. Default is "time".
• variable_column_name: Name of column in X that corresponds to the variables. Default is nothing.
• value_column_name: Name of column in X that corresponds to the covariates. Default is nothing.
• hidden_units: Number of neurons in hidden layer. Default is 10.
• seed: Fixed random seed for repeatable results. Default is 1.
• proc_weight: Weight of process error $omega_{proc}$. Default is 1.0.
• obs_weight: Weight of observation error $omega_{obs}$. Default is 1.0.
• reg_weight: Weight of regularization error $omega_{reg}$. Default is 10^-6.
• reg_type: Type of regularization, whether "L1" or "L2" regularization. Default is "L2".
• l: Extrapolation parameter for forecasting. Default is 0.25.
• extrap_rho: Extrapolation parameter for forecasting. Default is 0.0.
• step_size: Step size for ADAM optimizer. Default is 0.05.
• maxiter: Maximum number of iterations in gradient descent algorithm. Default is 500.
• verbose: Should the training loss values be printed?. Default is false.

EasyNODE(data;kwargs ... )

Constructs a pretrained continuous-time model for the data set data using a single layer neural network to represent the system's dynamics.

kwargs

• time_column_name: Name of column in data that corresponds to time. Default is "time".
• hidden_units: Number of neurons in hidden layer. Default is 10.
• seed: Fixed random seed for repeatable results. Default is 1.
• proc_weight: Weight of process error $omega_{proc}$. Default is 1.0.
• obs_weight: Weight of observation error $omega_{obs}$. Default is 1.0.
• reg_weight: Weight of regularization error $omega_{reg}$. Default is 10^-6.
• reg_type: Type of regularization, whether "L1" or "L2" regularization. Default is "L2".
• l: Extrapolation parameter for forecasting. Default is 0.25.
• extrap_rho: Extrapolation parameter for forecasting. Default is 0.0.
• step_size: Step size for ADAM optimizer. Default is 0.05.
• maxiter: Maximum number of iterations in gradient descent algorithm. Default is 500.
• verbose: Should the training loss values be printed?. Default is false.

EasyUDE(data,derivs!,initial_parameters;kwargs ... )

Constructs a pretrained UDE model for the data set data based on user defined derivatives derivs. An initial guess of model parameters are supplied with the initial_parameters argument.

kwargs

• time_column_name: Name of column in data that corresponds to time. Default is "time".
• hidden_units: Number of neurons in hidden layer. Default is 10.
• seed: Fixed random seed for repeatable results. Default is 1.
• proc_weight: Weight of process error $omega_{proc}$. Default is 1.0.
• obs_weight: Weight of observation error $omega_{obs}$. Default is 1.0.
• reg_weight: Weight of regularization error $omega_{reg}$. Default is 10^-6.
• reg_type: Type of regularization, whether "L1" or "L2" regularization. Default is "L2".
• l: Extrapolation parameter for forecasting. Default is 0.25.
• extrap_rho: Extrapolation parameter for forecasting. Default is 0.0.
• step_size: Step size for ADAM optimizer. Default is 0.05.
• maxiter: Maximum number of iterations in gradient descent algorithm. Default is 500.
• verbose: Should the training loss values be printed?. Default is false.

EasyUDE(data::DataFrame,X,derivs!::Function,initial_parameters;kwargs ... )

When a data frame X is supplied the model will run with covariates. The argument X should have a column for time t with the value for time in the remaining columns. The values in X will be interpolated with a linear spline for value of time not included in the data frame. When X is provided the derivs function must have the form derivs!(du,u,x,p,t) where x is a vector with the value of the covariates at time t.

# kwargs

• time_column_name: Name of column in data and X that corresponds to time. Default is "time".
• variable_column_name: Name of column in X that corresponds to the variables. Default is "variable".
• value_column_name: Name of column in X that corresponds to the covariates. Default is "value".
• hidden_units: Number of neurons in hidden layer. Default is 10.
• seed: Fixed random seed for repeatable results. Default is 1.
• proc_weight: Weight of process error $omega_{proc}$. Default is 1.0.
• obs_weight: Weight of observation error $omega_{obs}$. Default is 1.0.
• reg_weight: Weight of regularization error $omega_{reg}$. Default is 10^-6.
• reg_type: Type of regularization, whether "L1" or "L2" regularization. Default is "L2".
• l: Extrapolation parameter for forecasting. Default is 0.25.
• extrap_rho: Extrapolation parameter for forecasting. Default is 0.0.
• step_size: Step size for ADAM optimizer. Default is 0.05.
• maxiter: Maximum number of iterations in gradient descent algorithm. Default is 500.
• verbose: Should the training loss values be printed?. Default is false.

LorenzLotkaVolterra(;kwargs)

Create a sample dataset using the Lorenz Lotka-Volterra model as its process model:

```math
rac{dx}{dt} = rx(1-rac{x}{K}) - lpha xy + gz\
rac{dy}{dt} = 	hetalpha xy - my\
rac{dz}{dt} = l(w-z)\
rac{dw}{dt} = z(
ho-s) 0 w\
rac{ds}{dt} = zw-eta s
```

and an observation error following a normal distribution with mean 0 and standard deviation σ_{obs}.

# kwargs
- `plot`: Does the function return a plot? Default is `true`.
- `seed`: Seed for observation error to create repeatable examples. Default is `123`.
- `datasize`: Number of time steps generated. Default is `60`.
- `T`: Maximum timespan. Default is `3.0`.
- `sigma`: Standard deviation of observation error. Default is `0.075`.

LotkaVolterra(;kwargs)

Create a sample dataset using the Lotka-Volterra predator-prey model as its process model:

```math
rac{dN}{dt} = rN - lpha NP \
rac{dP}{dt} = 	hetalpha NP - mP
```

and an observation error following a normal distribution with mean 0 and standard deviation σ.

# kwargs
- `plot`: Does the function return a plot? Default is `true`.
- `seed`: Seed for observation error to create repeatable examples. Default is `123`.
- `datasize`: Number of time steps generated. Default is `60`.
- `T`: Maximum timespan. Default is `3.0`.
- `sigma`: Standard deviation of observation error. Default is `0.075`.

MultiCustomDerivatives(data,derivs!,initial_parameters;kwargs...)

Builds a UDE model that can be trianed on multiple time series simultaniously. The user defined derivatives functions must allow for an extra argument i that indexes over the time seris in the data set (e.g. derivs!(du,u,i,)). data is a DataFrame object with time arguments placed in a column labeled t and a second column with a unique index for each time series. The remaining columns have observations of the state variables at each point in time and for each time series.

kwargs

• time_column_name: Name of column in data that corresponds to time. Default is "time".
• series_column_name: Name of column in data that corresponds to time. Default is "series".
• variable_column_name: Name of column in data that corresponds to the variables. Default is "variable".
• value_column_name: Name of column in data that corresponds to the covariates. Default is "value".
• proc_weight: Weight of process error omega_{proc}. Default is 1.0.
• obs_weight: Weight of observation error omega_{obs}. Default is 1.0.
• reg_weight: Weight of regularization error omega_{reg}. Default is 10^-6.
• reg_type: Type of regularization, whether "L1" or "L2" regularization. Default is "L2".
• l: Extrapolation parameter for forecasting. Default is 0.25.
• extrap_rho: Extrapolation parameter for forecasting. Default is 0.0.

MultiNODE(data,X;kwargs...)

When a dataframe X is supplied the model will run with covariates. the argument X should have a column for time t with the value for time in the remaining columns. The values in X will be interpolated with a linear spline for values of time not included in the data frame.

kwargs

• time_column_name: Name of column in data that corresponds to time. Default is "time".
• series_column_name: Name of column in data that corresponds to time. Default is "series".
• variable_column_name: Name of column in data that corresponds to the covariates. Default is "variable".
• value_column_name: Name of column in data that corresponds to the covariates. Default is "value".
• hidden_units: Number of neurons in hidden layer. Default is 10.
• seed: Fixed random seed for repeatable results. Default is 1.
• proc_weight: Weight of process error omega_{proc}. Default is 1.0.
• obs_weight: Weight of observation error omega_{obs}. Default is 1.0.
• reg_weight: Weight of regularization error omega_{reg}. Default is 10^-6.
• reg_type: Type of regularization, whether "L1" or "L2" regularization. Default is "L2".
• l: Extrapolation parameter for forecasting. Default is 0.25.
• extrap_rho: Extrapolation parameter for forecasting. Default is 0.0.

MultiNODE(data;kwargs...)

builds a NODE model to fit to the data. data is a DataFrame object with time arguments placed in a column labed t and a second column with a unique index for each time series. The remaining columns have observations of the state variables at each point in time and for each time series.

kwargs

• time_column_name: Name of column in data that corresponds to time. Default is "time".
• series_column_name: Name of column in data that corresponds to time. Default is "series".
• hidden_units: Number of neurons in hidden layer. Default is 10.
• seed: Fixed random seed for repeatable results. Default is 1.
• proc_weight: Weight of process error omega_{proc}. Default is 1.0.
• obs_weight: Weight of observation error omega_{obs}. Default is 1.0.
• reg_weight: Weight of regularization error omega_{reg}. Default is 10^-6.
• reg_type: Type of regularization, whether "L1" or "L2" regularization. Default is "L2".
• l: Extrapolation parameter for forecasting. Default is 0.25.
• extrap_rho: Extrapolation parameter for forecasting. Default is 0.0.

NNDE(data;kwargs ...)

Constructs a nonparametric discrete-time model for the data set data using a single layer neural network to represent the system's dynamics.

# kwargs

• time_column_name: Name of column in data that corresponds to time. Default is "time".
• hidden_units: Number of neurons in hidden layer. Default is 10.
• seed: Fixed random seed for repeatable results. Default is 1.
• proc_weight: Weight of process error $omega_{proc}$. Default is 1.0.
• obs_weight: Weight of observation error $omega_{obs}$. Default is 1.0.
• reg_weight: Weight of regularization error $omega_{reg}$. Default is 10^-6.
• reg_type: Type of regularization, whether "L1" or "L2" regularization. Default is "L2".
• l: Extrapolation parameter for forecasting. Default is 0.25.
• extrap_rho: Extrapolation parameter for forecasting. Default is 0.0.

NODE(data,X;kwargs ... )

When a data frame X is supplied the model will run with covariates. The argument X should have a column for time t with the value for time in the remaining columns. The values in X will be interpolated with a linear spline for values of time not included in the data frame.

• time_column_name: Name of column in data and X that corresponds to time. Default is "time".
• variable_column_name: Name of column in X that corresponds to the variables. Default is nothing.
• value_column_name: Name of column in X that corresponds to the covariates. Default is nothing.
• hidden_units: Number of neurons in hidden layer. Default is 10.
• seed: Fixed random seed for repeatable results. Default is 1.
• proc_weight: Weight of process error $omega_{proc}$. Default is 1.0.
• obs_weight: Weight of observation error $omega_{obs}$. Default is 1.0.
• reg_weight: Weight of regularization error $omega_{reg}$. Default is 10^-6.
• reg_type: Type of regularization, whether "L1" or "L2" regularization. Default is "L2".
• l: Extrapolation parameter for forecasting. Default is 0.25.
• extrap_rho: Extrapolation parameter for forecasting. Default is 0.0.

NODE(data;kwargs ... )

Constructs a nonparametric continuous-time model for the data set data using a single layer neural network to represent the system's dynamics.

# kwargs

• time_column_name: Name of column in data that corresponds to time. Default is "time".
• hidden_units: Number of neurons in hidden layer. Default is 10.
• seed: Fixed random seed for repeatable results. Default is 1.
• proc_weight: Weight of process error $omega_{proc}$. Default is 1.0.
• obs_weight: Weight of observation error $omega_{obs}$. Default is 1.0.
• reg_weight: Weight of regularization error $omega_{reg}$. Default is 10^-6.
• reg_type: Type of regularization, whether "L1" or "L2" regularization. Default is "L2".
• l: Extrapolation parameter for forecasting. Default is 0.25.
• extrap_rho: Extrapolation parameter for forecasting. Default is 0.0.

 NUTS!(UDE, kwargs ...)

performs Bayesian estimation on the parameters of an UDE using the NUTS sampling algorithm.

kwargs

• delta: Step size used in NUTS adaptor. Default is 0.45.
• samples: Number of parameters sampled. Default is 500.
• burnin: Number of samples used as burnin of Bayesian algorithm. Default is samples/10.
• verbose: Should the training loss values be printed?. Default is false.

 SGLD!(UDE, kwargs ...)

Performs Bayesian estimation on the parameters of an UDE using the SGLD sampling algorithm. At each step t, the stochastic update is provided by a random variable ε with mean 0 and variance:

math a(b + t-1)^γ

kwargs

• a: Default is 10.0.
• b: Default is 1000.
• γ: Default is 0.9.
• samples: Number of parameters sampled. Default is 500.
• burnin: Number of samples used as burnin of Bayesian algorithm. Default is samples/10.
• verbose: Should the training loss values be printed?. Default is false.

bifurcation_data(model::MultiUDE;N=25)

Calcualtes the equilibrium values of the state variabels $y_t$ as a function of the covariates X_t and return the value in a data frame. The funciton calcualtes the equilibrium values on a grid of $N$ evenly spaced point for each covariate. The calcualtion are repeated for each time series $i$ included in the training data set.

bifurcation_data(model::UDE;N=25)

Calcualtes the equilibrium values of the state variabels $y_t$ as a function of the covariates X_t and return the value in a data frame. The funciton calcualtes the equilibrium values on a grid of $N$ evenly spaced point for each covariate.

crossvalidationkfold(model::UDE; kwagrs...)

This funciton approximates model performance on out of sample data by leaving blocks of consequtive observaitons out of the training data. The model is trained on the remiaining observations and the and the one step ahead prediction accuracy is calcualted on the testing data set. This procedure is repeated k times. ...

Arguments

k = 10 - the number of testing data sets BFGS = false - weather or not to train the models with the BFGS algorithm stepsize = 0.05 - the step size for the first round of gradient descent optimization maxiter = 500 - the number of iterations for the first round of gradinet descent stepsize2 = 0.05 - the step size for the second round of gradient descent maxiter2 = 500 - the number of iterations for the second round of gradient descent N = 1000 - the number of particle to use in the particle filter algorithm that estiamtes the states in the out of sample data nugget = 10^-10 - a small number to added variance terms in the particle filter algorith to improve numerical stability ...

derivative_matching!(UDE::UDE; kwargs ...)

Trains the UDE models using a two step process. First a smooth curve is fit to the data set using funcitons from DataInterpolations.jl (Bhagavan et al. 2024). The derivatives of the soothing function are then compared to the derivative rpeodicted by the right hand side of the UDE model using the mean squared error. This approach signifcantly increases the speed of training becuse it does not require the UDE model to be integrated by a ODE solver. Unfortunately, this method also relies on the accuracy of the smoothing algorithm. We suggest using this method to get close to the optial parameter sets and then applying a differnt more accurate method to finish the training procedure.

Note hat some of the smoothing curves will lose accuracy near the begining and end of the time series. The key word arguemnt remove_ends allows the user to specify the number of data point to leave out to remove these edge effects.

kwargs

• step_size: Step size for ADAM optimizer. Default is 0.05.
• maxiter: Maximum number of iterations in gradient descent algorithm. Default is 500.
• verbose: Should the training loss values be printed?. Default is false.
• d: the number of grid points used by the data interpolation algorithm. Default is 12.
• alg: The algorithm from DataInterpolations.jl used to fit the smoothing curve. Default is :gcv_svd.
• remove_ends: Number of data points to leave off to remove edge effects while training. Defualt is 2.

equilibrium_and_stability(UDE,X,lower,upper;t=0,Ntrials=100,tol=10^-3)

Attempts to find all the equilibrium points for the UDE model between the upper and lower bounds, and then returns the real component of the dominant eigenvalue to analyze stability.

...

kwargs

• t = 0: The point in time where the UDE model is evaluated, only relevant for time aware UDEs.
• Ntrials = 100: the number of initializations of the root finding algorithm.
• tol = 10^-3: The threshold Euclidean distance between points beyond which a new equilibrium is sufficiently different to be retained.

...

equilibrium_and_stability(UDE,lower,upper;t=0,Ntrials=100,tol=10^-3)

Attempts to find all the equilibrium points for the UDE model between the upper and lower bounds, and then returns the real component of the dominant eigenvalue to analyze stability.

...

kwargs

• t = 0: The point in time where the UDE model is evaluated, only relevant for time aware UDEs.
• Ntrials = 100: the number of initializations of the root finding algorithm.
• tol = 10^-3: The threshold Euclidean distance between points beyond which a new equilibrium is sufficiently different to be retained.

...

equilibrium_and_stability(UDE::MultiUDE,site,X,lower,upper;t=0,Ntrials=100,tol=10^-3)

Attempts to find all the equilibrium points for the UDE model between the upper and lower bounds, and then returns the real component of the dominant eigenvalue to analyze stability.

...

kwargs

• t = 0: The point in time where the UDE model is evaluated, only relevant for time aware UDEs.
• Ntrials = 100: the number of initializations of the root finding algorithm.
• tol = 10^-3: The threshold Euclidean distance between points beyond which a new equilibrium is sufficiently different to be retained.

...

forecast(UDE::BayesianUDE, u0::AbstractVector{}, times::AbstractVector{}; summarize = true, ci = 95)

predictions from the trained model UDE starting at u0 saving values at times at each individual sampled parameter. Assumes u0 is the value at time times[1]

If summarize is true, this function returns the median prediction as well as the ci% lower and upper confidence intervals. Othwerise, it returns all the individual predictions for each sampled parameter.

forecast(UDE::BayesianUDE, u0::AbstractVector{}, t0::Real, times::AbstractVector{}; summarize = true, ci = 95)

predictions from the trained model UDE starting at u0 saving values at times at each individual sampled parameter. Assumes u0 occurs at time t0 and times are all larger than t0.

If summarize is true, this function returns the median prediction as well as the ci% lower and upper confidence intervals. Othwerise, it returns all the individual predictions for each sampled parameter.

forecast(UDE::UDE, u0::AbstractVector{}, times::AbstractVector{})

Predictions from the trained UDE model starting at u0 and saving values at times. Assumes u0 is the value at initial time times[1]

forecast(UDE::UDE, u0::AbstractVector{}, t0::Real, times::AbstractVector{})

predictions from the trained model UDE starting at u0 saving values at times. Assumes u0 occurs at time t0 and times are all larger than t0.

get_NN_parameters(UDE::UDE)

Returns the values of the weights and biases of the neural network.

get_parameters(UDE::UDE)

Returns model parameters.

get_right_hand_side(UDE::UDE)

Returns the right-hand side of the differential equation (or difference equation) used to build the process model.

The function will take the state vector u and time t if the model does not include covariates. If covariates are included, then the arguments are the state vector u , covariates vector x, and time t.

 gradient_descent!(UDE, kwargs ...)

Minimizes the loss function of the UDE model with the gradient descent algorithm with a step size of step_size and a maximum number of iterations of maxiter. Prints the value of the loss function after each iteration when maxiter is true.

kwargs

• step_size: Step size for ADAM optimizer. Default is 0.05.
• maxiter: Maximum number of iterations in gradient descent algorithm. Default is 500.
• verbose: Should the training loss values be printed?. Default is false.

kfold_cv(model::UDE; kwargs ...)

Runs K fold leave future out cross validation and returns the mean squared forecasting error and a plot to visualize the model fits.

...

Arguments

model - a UDE model object k=10 - the number of testing and taining data sets to build leaveout=5 - the number of data points to leave out in each testingdata set BFGS = false - use the BFGS algorithm to train the model stepsize = 0.05 - step size for first run of the ADAM algorithm maxiter = 500 - maximum iterations for first trial of ADAM step_size2 = 0.01 - step size for second iteration of ADAM, only used of BFGS is false maxiter2 = 500 - step size for second iteration of ADAM, only used of BFGS is false ...

leave_future_out_cv(model::UDE; kwargs ...)

Runs K fold leave future out cross validation and returns the mean squared forecasting error and a plot to visualize the model fits.

...

Arguments

model - the UDE model to test forecastlength = 5 - The number of data points to forecast forecastnumber = 10 - The number of trainin gnad testing data sets to create spacing = 2 - The number of data points to skip between the end of each new triaining data set BFGS=false - use BFGS algorithm to train the model stepsize = 0.05 - step size for first run of the ADAM algorithm maxiter = 500 - maximum iterations for first trial of ADAM stepsize2 = 0.01 - step size for second iteration of ADAM, only used of BFGS is false maxiter2 = 500 - step size for second iteration of ADAM, only used of BFGS is false ...

mini_batching!(UDE::UDE, kwargs...)

Trains the UDE model using the mini batching algorithm. Mini batching breaks the data set up into blocks of consequtive observaitons of length pred_length. The model predicts the value at each time point in the block by solving the ODE using the first data point as the initial conditions up to the time of the final data point in the next block. The loss is calcualted by comparing the predicted and observed values using the mean squared error.

Longer block lengths may increase the speed of training by allowing the ODE solvers to find efficent integation schemes. However. long step sizes can create local minima in the loss funciton on data sets with oscilations or orther forms of variability.

kwargs

• pred_length: Number of observations in each block. Default is 10.
• step_size: Step size for ADAM optimizer. Default is 0.05.
• maxiter: Maximum number of iterations in gradient descent algorithm. Default is 500.
• verbose: Should the training loss values be printed?. Default is false.
• ode_solver: Algorithm for solving the ODE solver from DiffEqFlux. Default is Tsit5().
• ad_method: Automatic differntialion algorithm for the ODE solver from DiffEqFlux. Default is ForwardDiffSensitivity().

observation_error_correlations(UDE)

The first differnce plot of the observation errors $psilon_t$. This allows the user to check for autocorrelation in the observation errors.

onestepahead!(UDE::UDE, kwargs...)

Trains the model UDE using a modified version of the loss function where the estimated value of the state variables are fixed at the value fo the observations uhat = y. The model is then trined to minimized the differnce between the prediced and observed changes in the data sets using the ADAM gradient descent algorithm.

kwargs

• step_size: Step size for ADAM optimizer. Default is 0.05.
• maxiter: Maximum number of iterations in gradient descent algorithm. Default is 500.
• verbose: Should the training loss values be printed?. Default is false.

phase_plane(UDE::UDE, u0s::AbstractArray; idx=[1,2],T = 100)

Plots the trajectory of state variables as forecasted by the model. Runs a forecast for each provided initial condition out to T timesteps. Change the state variables that are plotted by changing idx such that it equals the indexes of the desired state variables as they appear in the data.

phase_plane(UDE::UDE; idx=[1,2], u1s=-5:0.25:5, u2s=-5:0.25:5, u3s = 0, T = 100)

Plots the trajectory of state variables as forecasted by the model. Runs a forecast for each permutation of u1 and u2 out to T timesteps. Change the state variables that are plotted by changing idx such that it equals the indexes of the desired state variables as they appear in the data.

phase_plane_3d(UDE::UDE; idx=[1,2,3], u1s=-5:0.25:5, u2s=-5:0.25:5, u3s=-5:0.25:5, T = 100)

The same as phase_plane(), but displays three dimensions/state variables instead of two.

plot_bifurcation_diagram(model::UDE, xvariable; N = 25, color_variable= nothing, conditional_variable = nothing, size= (600, 400))

This function returns a plot of the equilibrium values of the state varaibles $y_t$ as a funciton of the covariates $X_t$. The arguemnt xvariable determines the covariate plotted on the x-axis. Additional variables can be visualized in sperate panel by specifying the conditional_variable key word argument or visualized by the color scheme using the color_variable argument.

The time sereis are treated as an additional covariate that can be visualized by setting the color_variable or conditional_variable equal to "series" or the series column name in the training data.

The key word arguent size controls the dimensions of the final plot.

plot_bifurcation_diagram(model::UDE, xvariable; N = 25, color_variable= nothing, conditional_variable = nothing, size= (600, 400))

This function returns a plot of the equilibrium values of the state varaibles $y_t$ as a funciton of the covariates $X_t$. The arguemnt xvariable determines the covariate plotted on the x-axis. Additional variables can be visualized in sperate panel by specifying the conditional_variable key word argument or visualized by the color scheme using the color_variable argument.

The key word arguent size controls the dimensions of the final plot.

plot_forecast(UDE::BayesianUDE, test_data::DataFrame)

Plots the model's forecast over the range of the test_data along with the value of the test data including the median prediction as well as the ci% lower and upper confidence intervals.

plot_forecast(UDE::BayesianUDE, T::Int)

Plots the models forecast up to T time steps into the future from the last observation including the median prediction as well as the ci% lower and upper confidence intervals.

plot_forecast(UDE::UDE, test_data::DataFrame)

Plots the model's forecast over the range of the test data along with the value of the test data.

plot_forecast(UDE::UDE, T::Int)

Plots the model's forecast up to T time steps into the future from the last observation.

plot_predictions(UDE::BayesianUDE;ci=95)

Plots the correspondence between the observed state transitons and the predicitons for the model UDE with a confidence interval ci.

plot_predictions(UDE::UDE, test_data::DataFrame)

Plots the correspondence between the observed state transitions in test data and the predictions from the UDE model.

plot_predictions(UDE::UDE)

Plots the correspondence between the observed state transitions and the predictions from the UDE model.

plot_state_estimates(UDE::UDE)

Plots the values of the state variables estimated by the UDE model.

predict(UDE::BayesianUDE,test_data::DataFrame;summarize = true,ci = 95,df = true)

Uses the Bayesian UDE UDE to predict the state of the data test_data for each of the sampled parameters in training.

If summarize is true, this function returns the median prediction as well as the ci% lower and upper confidence intervals. Othwerise, it returns all the individual predictions for each sampled parameter.

If df is true, this function returns a DataFrame object. Otherwise, it returns an Array with the predictions.

print_parameter_estimates(UDE::UDE)

Prints the values of the known dynamics parameters estimated by the UDE model.

simulate_coral_data()

Generates synthetic coral reef time series data used in the time dependent UDE model example.

vectorfieldandnullclines(UDE; kwargs)

Calculate the vector field and nullclines of the 2D UDE model and returns their plot.

kwargs

-t: Time step t at which the vector field and nullclines are calculated. Default is 0. -n: Number of elements per axes to evaluate vector field at. Default is 15. -lower: Lower limits of vector field and nullclines. Default is [0.0,0.0]. -upper: Upper limits of vector field and nullclines. Default is [1.0,1.0]. -arrowlength: Arrow size of vector field plot. Default is 2. -arrow_color: Arrow color of vector field plot. Default is grey. -xlabel: X-label of vector field plot. Default is u1. -ylabel: Y-label of vector field plot. Default is u2. -title: Plot title. Default is Vector field. -color_u1: Color of nullcline in x-axis. Default is "red". -color_u2: Color of nullcline in y-axis. Default is "black". -legend: Position of legends of nullcines in plot. Default is :outerright.