Data Preparation

This page explains how to format your time-series data for use with NeurEco Dynamic models, whether for training or evaluation.

Data Structure Overview

Dynamic models work with two arrays per time series, and each array begins with a time column (column 0):

1. Excitation (input) array: column 0 is time, the remaining columns are the input signals driving the system

2. Output array: column 0 is time, the remaining columns are the target system responses

Both arrays of a sequence must carry the same time column. Multiple sequences (experiments, simulations, or recordings) can be combined for training.

Note

Deprecated separate-time form. Passing the time as a separate array (train_time_list and its valid_/test_ counterparts, with the excitation/output arrays then holding pure excitations/outputs) is still supported for backward compatibility but emits a DeprecatedParameterWarning. Prefer placing the time in the first column of each excitation and output array.

Critical Requirements

Warning

Violating these requirements will cause errors or incorrect results.

Requirements for All Dynamic Models

1. Matching row counts: For each sequence, the excitation and output arrays must have the same number of rows and share the same time column (column 0)

2. Consistent feature counts: All excitation arrays must have the same number of columns; all output arrays must have the same number of columns (the leading time column included)

3. Corresponding indices: Lists must maintain correspondence — the i-th excitation and output arrays must belong to the same sequence

Template-Specific Requirements

Requirement	ODE	Convolutional Dynamic
Time stamp ordering	Required — the time column (column 0) must be sorted in strictly increasing order (no repeated time stamps)	Required — the time column (column 0) must be sorted in strictly increasing order (no repeated time stamps)
Time step uniformity	Not required — can handle non-uniform time steps (as long as they are strictly increasing)	Required — must be uniform within and across sequences
Time step consistency	Can vary between sequences	Must be identical across all sequences

What Can Vary Between Sequences

• Duration (number of time steps)

• Start time

• Time step spacing (ODE only)

What Must Be Constant

• Number of input features

• Number of output features

• Time step (Convolutional Dynamic only)

Data Formats by Interface

GUIPython APICLI

File Formats

The GUI accepts data in the following formats:

• CSV with ; or , separator

• NumPy .npy files

• MATLAB .mat files

Allowed data types: int, float, double

Input (Excitation) Files

Each input file contains a table where:

• First column: Time variable (uniformly spaced for Convolutional Dynamic; for ODE any spacing is allowed, provided the values are sorted in strictly increasing order)

• Remaining columns: Input features (excitations)

• Each row: Time point and corresponding input values

• CSV files may include a header row

Example CSV structure:

time;force;temperature;pressure
0.00;1.2;25.3;101.5
0.01;1.4;25.4;101.6
0.02;1.3;25.5;101.7
...

Output Files

Each output file contains a table where:

• First column: Time variable (must match the corresponding input file)

• Remaining columns: Output features

• Each row: Time point and corresponding output values

• CSV files may include a header row

Example CSV structure:

time;position;velocity
0.00;0.0;0.0
0.01;0.012;0.24
0.02;0.048;0.47
...

Multiple Sequences

When your data comes from multiple experiments:

• Provide multiple input and output file pairs

• Maintain correspondence between input and output files

• For Convolutional Dynamic: all files must use the same time step

• For ODE: time steps can vary between sequences

• Different sequences can have different durations and start times

Data Normalization

NeurEco automatically normalizes your data before training. Understanding normalization options helps you get better results.

How Normalization Works

NeurEco applies the formula:

\[x_{normalized} = \frac{x - shift}{scale}\]

Where shift and scale are computed based on your chosen methods.

Note

ODE and Convolutional Dynamic templates have different default normalization options optimized for their respective use cases.

Shift Methods

Method	Description	Default for
mean	Shift by the mean value (centers data around 0)	ODE
min	Shift by the minimum value	Convolutional Dynamic
none	No shifting applied	—

Scale Methods

Method	Description	Default for
l2	Scale by the L2 norm	ODE
minmax	Scale to [0, 1] range based on min/max values	Convolutional Dynamic
max	Scale by the maximum absolute value	—
std	Scale by the standard deviation	—
none	No scaling applied	—

Per-Feature Normalization

The normalize per feature option controls whether normalization is applied independently to each feature or globally across all features.

Set to True when:

• Features represent different physical quantities (e.g., temperature and pressure)

• Features have vastly different scales or units

• You want each feature to contribute equally

Set to False when:

• Features represent the same quantity (e.g., temperatures at different locations)

• You’re working with field data where relative magnitudes matter

GUIPython API (ODE)Python API (Convolutional Dynamic)CLI (ODE)CLI (Convolutional Dynamic)

In the Build panel, under Normalization:

• Select shift and scale methods from dropdowns

• Check/uncheck Normalize per feature for inputs and outputs

Tip

If the default normalization doesn’t suit your needs, you can normalize your data yourself before passing it to NeurEco and set normalization to none.

Complete Python Example

import numpy as np
from NeurEco import NeurEcoDynamic as Dynamic

# Define common time step
dt = 0.001  # 1 ms

# Experiment 1: 5 seconds, 3 inputs, 2 outputs (time in column 0)
t1 = np.arange(0, 5, dt)
x1 = np.column_stack([
    t1,                             # Column 0: time
    np.sin(2 * np.pi * 1.0 * t1),   # 1 Hz sine wave
    np.sin(2 * np.pi * 2.5 * t1),   # 2.5 Hz sine wave
    np.random.randn(len(t1)) * 0.1  # Noise
])
y1 = np.column_stack([
    t1,                                  # Column 0: time
    np.sin(2 * np.pi * 1.0 * t1 + 0.5),  # Phase-shifted response
    np.cos(2 * np.pi * 2.5 * t1)          # Another response
])

# Experiment 2: 3 seconds
t2 = np.arange(0, 3, dt)
x2 = np.column_stack([
    t2,                             # Column 0: time
    np.sin(2 * np.pi * 1.5 * t2),
    np.sin(2 * np.pi * 3.0 * t2),
    np.random.randn(len(t2)) * 0.1
])
y2 = np.column_stack([
    t2,                                  # Column 0: time
    np.sin(2 * np.pi * 1.5 * t2 + 0.5),
    np.cos(2 * np.pi * 3.0 * t2)
])

# Each array: column 0 = time, remaining columns = excitations / outputs.
train_exc_list = [x1, x2]
train_out_list = [y1, y2]

# Verify shapes
for i, (x, y) in enumerate(zip(train_exc_list, train_out_list)):
    print(f"Sequence {i}: excitation={x.shape}, output={y.shape}")

# Output:
# Sequence 0: excitation=(5000, 4), output=(5000, 3)
# Sequence 1: excitation=(3000, 4), output=(3000, 3)

# Now ready to use with NeurEco ODE or Convolutional Dynamic
model = Dynamic.ODE()
config = Dynamic.ODEBuildConfig(
    train_exc_list=train_exc_list,
    train_out_list=train_out_list,
    # ... other configuration options
)

Next Steps

• Build ODE models

• Build Convolutional Dynamic models