Advanced Features

This page covers advanced features and techniques for NeurEco Dynamic models.

Model Inspection

Understanding what your trained model has learned helps validate its behavior and identify potential issues.

ODE Model Inspection

ODE models provide several methods to inspect their structure and learned parameters:

from NeurEco import NeurEcoDynamic as Dynamic

model = Dynamic.ODE()
model.load("my_ode_model.ernn")

# Basic model information
print(f"Number of inputs: {model.get_input_count()}")
print(f"Number of outputs: {model.get_output_count()}")
print(f"Number of hidden states: {model.get_state_count()}")
print(f"Initialization steps required: {model.get_number_of_init_tsteps()}")

# Steady-state values
steady_inputs = model.get_steady_input()
steady_outputs = model.get_steady_output()
print(f"Steady-state inputs: {steady_inputs}")
print(f"Steady-state outputs: {steady_outputs}")

Convolutional Dynamic Model Inspection

from NeurEco import NeurEcoDynamic as Dynamic

model = Dynamic.ConvolutiveDynamic()
model.load("my_convolutive_model.ecnn")

# Basic model information
print(f"Number of inputs: {model.get_input_count()}")
print(f"Number of outputs: {model.get_output_count()}")
print(f"Window size: {model.get_window_size()}")

Sensitivity Analysis

Sensitivity analysis reveals how model outputs respond to changes in inputs, helping identify which inputs have the strongest influence on system behavior.

Computing Sensitivity

import numpy as np
from NeurEco import NeurEcoDynamic as Dynamic

model = Dynamic.ODE()
model.load("my_ode_model.ernn")

# Prepare the evaluation point. Note: unlike evaluate(), sensitivity() takes the
# time as a SEPARATE argument (it does not use the combined time-first form), and
# it is available for ODE models only (not Convolutional Dynamic).
dt = 0.01
time = np.arange(0, 5, dt)
excitations = np.random.randn(len(time), 3)

# Sensitivity of output 0 to each input -> array of shape (n_inputs,)
sens_output0 = model.sensitivity(time, excitations, id_output=0)
print(f"Sensitivity of output 0 to each input: {sens_output0}")

# Build the full (n_outputs, n_inputs) sensitivity matrix
sensitivity_matrix = np.vstack([
    model.sensitivity(time, excitations, id_output=k)
    for k in range(model.get_output_count())
])
print("Sensitivity matrix (outputs x inputs):")
print(sensitivity_matrix)

Interpreting Sensitivity

High sensitivity values indicate:

• Strong coupling between input and output

• Inputs that significantly affect system behavior

• Potential areas for control optimization

Low sensitivity values suggest:

• Weak or no coupling

• Inputs that may be candidates for removal

• System behavior is robust to that input

Checkpoint Management

Checkpoints save intermediate models throughout the build, so you can evaluate or export a model before training finishes.

Saving Checkpoints

Checkpoints are saved automatically during training:

config = Dynamic.ODEBuildConfig(
    # Training data...
    checkpoint_address="./checkpoints/my_model",
    write_model_to="./model/final_model.ernn",
)

model.build(config)

Resuming from Checkpoint

A build that stops before the end (it was stopped, it crashed, or the session was closed) can be resumed from its checkpoint instead of starting again:

Template	GUI	Python API	CLI
Convolutional Dynamic	RESUME	resume=True	"resume": true
ODE	RESUME	resume=True	"resume": true

What resume does:

• NeurEco reads the checkpoint at checkpoint_address, the file that the stopped build wrote, and continues the same build. The models already in the checkpoint are kept, and the new checkpoints are added after them.

• Convolutional Dynamic: the build continues at the stage recorded in the last checkpoint (first convolutional layer, added layers or Tabular stage, see Convolutional Dynamic Template).

• ODE: the build continues where it stopped. A resumed build gives the same models as a build without pause. If the build had already ended, nothing is left to build, and the build log says so.

• ODE: a build that cannot continue exactly stops with an error, for example when the checkpoint was written by an older NeurEco version. Start a new build instead.

• NeurEco resumes from the last checkpoint that was completely written, even if the build stopped during a save.

• The build log confirms the resume with Resume building from Checkpoint N (ODE) or Resume building from checkpoint N (Convolutional Dynamic), where N is the number of the last checkpoint.

• If the checkpoint cannot be read (it is missing, empty or damaged), NeurEco logs a warning and starts a new build from scratch. A checkpoint of another model type (for example a Tabular checkpoint) stops the build with an error.

Constraints:

• Use the same data and the same settings as the stopped build.

  • ODE: NeurEco checks them. If they differ, the resume stops with an error.

  • Convolutional Dynamic: NeurEco does not check them. Different data or settings can make the resumed build fail, or mix models built with different settings in one checkpoint.

• Keep checkpoint_address unchanged: the same file is read and then written.

• To use an intermediate model without building further, load it with load_model_from_checkpoint(path, model_index). Indices start at 0: “Checkpoint N” in the build log and the GUI is model_index N - 1.

GUIPython APICLI

Stop the build with STOP. While the checkpoint holds at least one model, the build button then reads RESUME: click it to continue the build where it stopped. Do not change the data or the settings in between.

Resuming is also available for Tabular models, with different options (see Tabular advanced features).

Custom Steady-State Constraints (ODE)

ODE models can be constrained to satisfy specific steady-state conditions, which improves physical consistency.

Specifying Steady State

# Each array: column 0 = time, remaining columns = excitations / outputs.
config = Dynamic.ODEBuildConfig(
    train_exc_list=train_exc_list,
    train_out_list=train_out_list,

    # Constrain steady-state values
    steady_state_exc=np.array([0.0, 0.0, 0.0]),  # Steady-state inputs
    steady_state_out=np.array([1.0, 0.0]),       # Steady-state outputs

    write_model_to="./model/constrained_model.ernn",
)

model.build(config)

When to Use Steady-State Constraints

The constraint gives the model exactly one equilibrium: under the constant excitations steady_state_exc the model rests and outputs steady_state_out (see Choosing the steady state).

Use constraints when:

• You know a physical equilibrium of the system, for example the system at rest

• The model must rest there when the excitations are held at their steady values (an evaluation without initialization data or an FMU run whose excitations start at these values starts there; one that starts elsewhere starts from the equilibrium the model learned for its first excitation values)

Avoid constraints when:

• The system has no natural equilibrium, or you do not know one

• The values are only a guess: a pair the system never rests at forces a false equilibrium into the model

Without constraints, get_steady_input() returns the input normalization shift and get_steady_output() the output the model settles to under it.

Multi-Sequence Training

Training with multiple sequences (experiments) improves model generalization.

Organizing Multiple Sequences

import numpy as np

dt = 0.01

# Different operating conditions
sequences = []
for amplitude in [0.5, 1.0, 2.0]:
    for frequency in [1.0, 2.0, 5.0]:
        t = np.arange(0, 10, dt)
        x = amplitude * np.sin(2 * np.pi * frequency * t)
        # Excitation: column 0 = time, remaining columns = excitations
        exc = np.column_stack([t, x])
        # Simulate or load corresponding outputs (column 0 = time)
        out = np.column_stack([t, np.random.randn(len(t), 2)])  # Replace with actual data

        sequences.append((exc, out))

# Each array: column 0 = time, remaining columns = excitations / outputs.
train_exc_list = [s[0] for s in sequences]
train_out_list = [s[1] for s in sequences]

Benefits of Multiple Sequences

• Better coverage: Different operating conditions expose more system behavior

• Improved generalization: Model learns to handle varied inputs

• Reduced overfitting: More diverse training data prevents memorization

Handling Long Time Series

For very long time series, consider these strategies:

Splitting Long Sequences

def split_sequence(exc, out, max_length=10000):
    """Split a long sequence into smaller chunks.

    ``exc`` and ``out`` are time-first arrays (column 0 = time); the time
    column is carried along with each chunk.
    """
    n_samples = len(exc)
    n_chunks = (n_samples + max_length - 1) // max_length

    exc_list = []
    out_list = []

    for i in range(n_chunks):
        start = i * max_length
        end = min((i + 1) * max_length, n_samples)

        exc_list.append(exc[start:end])
        out_list.append(out[start:end])

    return exc_list, out_list

Memory Management

For large datasets:

# Use float32 instead of float64 to reduce memory
train_exc_list = [exc.astype(np.float32) for exc in train_exc_list]
train_out_list = [out.astype(np.float32) for out in train_out_list]

Model Comparison

Compare different models or configurations:

import numpy as np
from NeurEco import NeurEcoDynamic as Dynamic

# Load models to compare
model_a = Dynamic.ODE()
model_a.load("model_a.ernn")

model_b = Dynamic.ODE()
model_b.load("model_b.ernn")

# Prepare test data (time in column 0)
test_time = np.arange(0, 10, 0.01)
test_exc = np.column_stack([test_time, np.random.randn(len(test_time), 3)])
test_out = np.column_stack([test_time, np.random.randn(len(test_time), 2)])  # Ground truth

# Evaluate both models
# test_exc has time in its first column; predictions are returned time-first.
pred_a = model_a.evaluate(test_exc)
pred_b = model_b.evaluate(test_exc)

# Compare errors (drop the leading time column; compute_error takes outputs only)
error_a = model_a.compute_error(pred_a[:, 1:], test_out[:, 1:])
error_b = model_b.compute_error(pred_b[:, 1:], test_out[:, 1:])

print(f"Model A L2 relative error: {error_a:.6f}")
print(f"Model B L2 relative error: {error_b:.6f}")

# Visualization
import matplotlib.pyplot as plt

fig, axes = plt.subplots(2, 1, figsize=(12, 8))

for i, ax in enumerate(axes):
    # Column 0 is time, so output i lives in column i + 1
    ax.plot(test_time, test_out[:, i + 1], 'k-', label='Reference', alpha=0.7)
    ax.plot(test_time, pred_a[:, i + 1], 'b--', label='Model A')
    ax.plot(test_time, pred_b[:, i + 1], 'r--', label='Model B')
    ax.set_xlabel('Time')
    ax.set_ylabel(f'Output {i}')
    ax.legend()
    ax.grid(True)

plt.tight_layout()
plt.savefig('model_comparison.png')
plt.show()

Best Practices Summary

1. Data quality: Ensure consistent feature counts across all sequences. Convolutional Dynamic additionally requires uniform time steps; ODE only requires each time array to be sorted in strictly increasing order

2. Steady-state constraints: Use them when you know the physical equilibrium of your system

3. Multiple sequences: Train with varied operating conditions for better generalization

4. Checkpoints: Always enable checkpoints for long training runs

5. Validation: Use separate validation data to monitor overfitting during training

6. Time step matching: Always use the same time step for training, validation, and deployment

7. Model inspection: Verify model properties (steady state, hidden states) match physical expectations

Next Steps

• Dynamic examples

• Export Dynamic models