GitHub

Fitting a model to multiple time series

UniversalDiffEq.jl provides a set of functions to fit models to multiple time series. The functions for this mirror the functions for fitting NODEs and UDEs to single time series with the prefix Multi. For example, to build a NODE model for multiple time series you would use the MultiNODE function. The functions for model fitting, testing, and visualization have the same names. The other important difference is the data format: a column with a unique index for each time series must be included.

Table: Example data frame with multiple time series

timeseries$x_1$$x_2$
11$x_{1,1,t}$$x_{1,2,t}$
21$x_{1,1,t}$$x_{1,2,t}$
31$x_{1,1,t}$$x_{1,2,t}$
12$x_{2,1,t}$$x_{2,2,t}$
22$x_{2,1,t}$$x_{2,2,t}$
32$x_{2,1,t}$$x_{2,2,t}$
UniversalDiffEq.MultiNODE — Method
MultiNODE(data;kwargs...)

builds a NODE model to fit to the data. data is a DataFrame object with time arguments placed in a column labed t and a second column with a unique index for each time series. The remaining columns have observations of the state variables at each point in time and for each time series.

kwargs

  • time_column_name: Name of column in data that corresponds to time. Default is "time".
  • series_column_name: Name of column in data that corresponds to time. Default is "series".
  • hidden_units: Number of neurons in hidden layer. Default is 10.
  • seed: Fixed random seed for repeatable results. Default is 1.
  • proc_weight: Weight of process error omega_{proc}. Default is 1.0.
  • obs_weight: Weight of observation error omega_{obs}. Default is 1.0.
  • reg_weight: Weight of regularization error omega_{reg}. Default is 10^-6.
  • reg_type: Type of regularization, whether "L1" or "L2" regularization. Default is "L2".
  • l: Extrapolation parameter for forecasting. Default is 0.25.
  • extrap_rho: Extrapolation parameter for forecasting. Default is 0.0.
source
UniversalDiffEq.MultiNODE — Method
MultiNODE(data,X;kwargs...)

When a dataframe X is supplied the model will run with covariates. the argument X should have a column for time t with the value for time in the remaining columns. The values in X will be interpolated with a linear spline for values of time not included in the data frame.

kwargs

  • time_column_name: Name of column in data that corresponds to time. Default is "time".
  • series_column_name: Name of column in data that corresponds to time. Default is "series".
  • variable_column_name: Name of column in data that corresponds to the covariates. Default is "variable".
  • value_column_name: Name of column in data that corresponds to the covariates. Default is "value".
  • hidden_units: Number of neurons in hidden layer. Default is 10.
  • seed: Fixed random seed for repeatable results. Default is 1.
  • proc_weight: Weight of process error omega_{proc}. Default is 1.0.
  • obs_weight: Weight of observation error omega_{obs}. Default is 1.0.
  • reg_weight: Weight of regularization error omega_{reg}. Default is 10^-6.
  • reg_type: Type of regularization, whether "L1" or "L2" regularization. Default is "L2".
  • l: Extrapolation parameter for forecasting. Default is 0.25.
  • extrap_rho: Extrapolation parameter for forecasting. Default is 0.0.
source

Multiple time series custom models

Custom models can be trained on multiple time series using the MultiCustomDerivatives function. The user-defined function that builds these models requires an additional argument, i added as the third argument (e.g., derivs!(du,u,i,X,p,t), derivs!(du,u,i,p,t)). The UniversalDiffEq.jl library will use this argument to pass a unique index for each time series. These indices can then be used to estimate different parameter values for each time series as illustrated in the following examples.

UniversalDiffEq.MultiCustomDerivatives — Method
MultiCustomDerivatives(data,derivs!,initial_parameters;kwargs...)

Builds a UDE model that can be trianed on multiple time series simultaniously. The user defined derivatives functions must allow for an extra argument i that indexes over the time seris in the data set (e.g. derivs!(du,u,i,)). data is a DataFrame object with time arguments placed in a column labeled t and a second column with a unique index for each time series. The remaining columns have observations of the state variables at each point in time and for each time series.

kwargs

  • time_column_name: Name of column in data that corresponds to time. Default is "time".
  • series_column_name: Name of column in data that corresponds to time. Default is "series".
  • variable_column_name: Name of column in data that corresponds to the variables. Default is "variable".
  • value_column_name: Name of column in data that corresponds to the covariates. Default is "value".
  • proc_weight: Weight of process error omega_{proc}. Default is 1.0.
  • obs_weight: Weight of observation error omega_{obs}. Default is 1.0.
  • reg_weight: Weight of regularization error omega_{reg}. Default is 10^-6.
  • reg_type: Type of regularization, whether "L1" or "L2" regularization. Default is "L2".
  • l: Extrapolation parameter for forecasting. Default is 0.25.
  • extrap_rho: Extrapolation parameter for forecasting. Default is 0.0.
source

Example 1: Estimating unique growth rates for population time series

To illustrate how unique parameters can be estimated for separate time series, we build a generalized logistic model that uses a neural network to describe the density dependence of the populations and estimates unique growth rates for each time series

\[\frac{du_i}{dt} = r_i u_i NN(u_i).\]

To build this model, we use the argument i of the derivs! function to index it into a vector of growth rate parameters. Notice that we need to transform i to be an integer for the indexing operation by calling round(Int,i) and we initialize the growth rate parameter r.

# set up neural network
using Lux
dims_in = 1
hidden_units = 10
nonlinearity = tanh
dims_out = 1
NN = Lux.Chain(Lux.Dense(dims_in,hidden_units,nonlinearity),Lux.Dense(hidden_units,dims_out))

# initialize parameters
using Random
rng = Random.default_rng()
NNparameters, NNstates = Lux.setup(rng,NN)

function derivs!(du,u,i,p,t)
    index = round(Int,i)
    du .= p.r[index].* u .* NN(u,p.NN, NNstates)[1]

end

m = 2 # number of time series
init_parameters = (NN = NNparameters, r = zeros(m), )

model = MultiCustomDerivatives(training_data,derivs!,init_parameters;proc_weight=2.0,obs_weight=0.5,reg_weight=10^-4)

Example 2: Allowing a neural network to vary between time series

In some cases, we may want to allow the neural networks to vary between time series so that they can estimate unknown functions specific to each time series. This can be achieved by passing an indicator variable to the neural network that encodes the time series being fit using one-hot encoding. This method allows the model to learn unique functions for each time series if appropriate, while also sharing information about the unknown function between time series.

To illustrate this, define a NODE model that takes the indicator variable as an additional argument.

# set up neural network
m = 10 # number of time series
using Lux
dims_in = 2+m #two inputs for the state variable plus m inputs for the one-hot encoding
hidden_units = 10
nonlinearity = tanh
dims_out = 2
NN = Lux.Chain(Lux.Dense(dims_in,hidden_units,nonlinearity),Lux.Dense(hidden_units,dims_out))

# initialize parameters
using Random
rng = Random.default_rng()
NNparameters, NNstates = Lux.setup(rng,NN)

function derivs!(du,u,i,X,p,t)
    index = round(Int,i)
    one_hot = zeros(m)
    one_hot[index] = 1
    du .=  NN(vcat(u,one_hot) ,p.NN, NNstates)[1]
end

init_parameters = (NN = NNparameters, )

model = MultiCustomDerivatives(training_data,derivs!,init_parameters;proc_weight=2.0,obs_weight=0.5,reg_weight=10^-4)
nothing