.. _ode-evaluating:

=====================
Evaluating ODE Models
=====================

This page explains how to run inference (evaluation) on trained NeurEco ODE models. ODE models are dynamical systems that require understanding of initialization to produce accurate simulations.

.. contents:: On this page
   :local:
   :depth: 2

Understanding ODE Initialization
================================

Unlike static models, ODE models maintain internal state and produce trajectories that evolve over time. To simulate a trajectory, the model needs to know where to start — this is called **initialization**.

NeurEco ODE supports two initialization modes:

1. **Explicit initialization** (Recommended): You provide the initial state of the system
2. **Steady-state initialization** (Default): The model starts from its own steady state for the first excitation values

Initialization Modes
====================

Mode 1: Explicit Initialization (Recommended)
---------------------------------------------

Provide initial conditions to start the simulation from a known state. This mode is recommended because:

- It gives you control over the starting point
- It allows evaluation from any operating point
- It produces more accurate results when the initial state is known

**Required data**, over the same initialization window, normally the
``model.get_number_of_init_tsteps()`` samples that end where the prediction starts:

- ``init_outputs``: The output values over the window, with time in the first column
- ``init_excitations``: The input values over the window, with time in the first column

.. note::

   The initialization time is read from the first column of ``init_excitations``
   and ``init_outputs``. The separate ``init_time`` keyword is deprecated (kept
   for backward compatibility, emits a ``DeprecatedParameterWarning``).

.. important::

   **Where the evaluation starts.** The model estimates its state at the **last** time of
   the initialization window and starts the evaluation from it. The evaluation excitations
   must therefore start at that last initialization time: with the first ``n_init`` samples
   of a trajectory as the window, evaluate from index ``n_init - 1`` (``exc[n_init - 1:]``),
   not ``n_init``.

   - The first predicted sample is the model's value at the last initialization time: the
     window and the prediction share that time step. To join them into one trajectory, drop
     the last sample of the window.
   - An evaluation that starts one sample later is not rejected, but the whole prediction is
     then one time step late.
   - A longer window is accepted, and only its last ``get_number_of_init_tsteps()`` samples
     are used. A shorter one is padded at its start, as if the system were at rest before it,
     and a warning is logged.

Mode 2: Steady-State Initialization (Default)
---------------------------------------------

If no initialization is provided, NeurEco computes the steady state (equilibrium) of the
learned model for the first excitation values of the sequence, and the simulation
starts from it.

This equilibrium is found for the excitation values at the first time of the sequence. It is
the steady state given at the build only when these values are that steady state:

- **First excitation values equal to** ``steady_state_exc``: the simulation starts from the
  given steady state, and the first predicted outputs equal ``steady_state_out``.
- **Other first excitation values:** NeurEco searches the model's equilibrium for them,
  starting from the given steady state (or, for a model built without one, from its
  equilibrium at the input normalization shift). This equilibrium was learned, not given at
  the build: the first predicted outputs are the model's, and nothing guarantees they match
  the real system.

This mode:

- Is simpler (no initialization data needed)
- Assumes the system starts at rest, in the steady state that matches its first excitation values
- Starts from a known point only when the first excitation values are the steady state given at the build
  (see :ref:`ode-steady-state`)

If no steady state can be computed for the first excitation values, which can happen with
or without a steady state given at the build, the evaluation starts from the initial state
learned for the training trajectory whose first excitation values are closest. The log says so, as an
information message when the model was built without a steady state and as a warning
otherwise. To set the starting point yourself, use explicit initialization
(``init_outputs``).

Evaluating an ODE Model
=======================

.. tabs::

   .. tab:: GUI

      **Loading a Model**

      1. Open your project or load a saved model
      2. Switch to the **Evaluation** panel

      **Providing Evaluation Data**

      1. Click **+** next to **Evaluation files**
      2. Select your input (excitation) file
      3. Optionally select an output file (for comparison/metrics)

      **Initialization Options**

      *Using steady-state initialization:*

      - Simply run evaluation without providing initialization data
      - The model starts from its own steady state for the first excitation values (see Mode 2 above)

      *Using explicit initialization:*

      1. Provide an output file that includes initial values
      2. Use the **Initialization samples** slider to specify how many initial time steps to use
      3. The model uses these points to initialize, then simulates forward. The prediction
         starts at the last sample of the initialization interval (the interval and the
         prediction share that sample) and runs to the end of the file.

      **Running Evaluation**

      1. Click **Evaluate**
      2. View results in the plot panel
      3. Use the **Sample** slider to browse through sequences
      4. Compare predictions (blue) with reference (orange) if provided

      **Metrics**

      Switch to the **Metrics** panel to see:

      - L2 relative error for each output
      - Per-sequence error breakdown
      - Training vs. validation vs. test performance

      Each file of the **Metrics** list offers two initializations. **Default Initialization**, the default,
      initializes every checkpoint from the first samples of the file, with the checkpoint's own number of
      initialization time steps (the **Time steps for initialization** editor shown over the file changes this
      number). **Steady State Initialization** starts from the model's steady state for the first excitation
      values (Mode 2 above).

   .. tab:: Python API

      **Loading a Model**

      .. code-block:: python

         from NeurEco import NeurEcoDynamic as Dynamic

         model = Dynamic.ODE()
         model.load("path/to/model.ernn")

      **Evaluation with Steady-State Initialization**

      .. code-block:: python

         import numpy as np

         # Prepare evaluation data (time in column 0)
         dt = 0.01  # Must match training time step
         time = np.arange(0, 5, dt)
         eval_exc = np.column_stack([time, np.random.randn(len(time), 3)])  # time + 3 inputs

         # Evaluate using steady-state initialization
         # eval_exc has time in its first column; predictions are returned time-first.
         outputs = model.evaluate(eval_exc)

         print(f"Output shape: {outputs.shape}")

      **Evaluation with Explicit Initialization**

      .. code-block:: python

         import numpy as np

         # Get number of initialization steps required
         n_init_steps = model.get_number_of_init_tsteps()
         print(f"Model requires {n_init_steps} initialization steps")

         dt = 0.01

         # Prepare initialization data (time in column 0)
         init_time = np.arange(0, n_init_steps * dt, dt)
         init_exc = np.column_stack([init_time, np.random.randn(n_init_steps, 3)])
         init_out = np.column_stack([init_time, np.random.randn(n_init_steps, 2)])  # Known initial state

         # Prepare evaluation data (time in column 0)
         # Note: Start from (n_init_steps - 1) * dt because the last init point
         # is also the first prediction point (they overlap)
         eval_time = np.arange((n_init_steps - 1) * dt, 5, dt)
         eval_exc = np.column_stack([eval_time, np.random.randn(len(eval_time), 3)])

         # Evaluate with explicit initialization
         # eval_exc / init_exc / init_out all carry time in their first column;
         # predictions are returned time-first.
         outputs = model.evaluate(
             eval_exc,
             init_excitations=init_exc,
             init_outputs=init_out,
         )

      **Evaluating Multiple Sequences**

      .. code-block:: python

         # Multiple sequences of different lengths (each array: time in column 0)
         times = [
             np.arange(0, 3, dt),
             np.arange(0, 5, dt),
             np.arange(0, 2, dt),
         ]

         eval_exc_list = [
             np.column_stack([times[0], np.random.randn(len(times[0]), 3)]),
             np.column_stack([times[1], np.random.randn(len(times[1]), 3)]),
             np.column_stack([times[2], np.random.randn(len(times[2]), 3)]),
         ]

         # Evaluate all sequences
         # Each array has time in its first column; predictions are returned time-first.
         output_list = model.evaluate(eval_exc_list)

         for i, out in enumerate(output_list):
             print(f"Sequence {i}: output shape {out.shape}")

      **Computing Error**

      .. code-block:: python

         # Compare predictions to ground truth
         # eval_exc has time in its first column; predictions are returned time-first.
         predictions = model.evaluate(eval_exc)
         # compute_error takes outputs only, so drop the leading time column from both.
         error = model.compute_error(predictions[:, 1:], reference_outputs[:, 1:])
         print(f"L2 relative error: {error:.6f}")

   .. tab:: CLI

      **Configuration File**

      .. code-block:: json

         {
             "neurecoODE_evaluate": {
                 "ernn_filename": "./model/ode_model.ernn",
                 "exc_filenames": ["data/test/test_exc_1.csv"],
                 "write_model_output_to_directory": "./results/"
             }
         }

      **Running Evaluation**

      .. code-block:: shell

         neureco ODE evaluate config.json

      Results are saved to the specified output directory, one file per excitation file,
      named after it with ``_ernn.csv`` added to its stem (``test_exc_1_ernn.csv`` here),
      with the time in the first column. Without initialization files, the evaluation starts
      from the steady state (Mode 2 above).

      **Explicit Initialization**

      Add the initialization files, one excitation file and one output file per evaluated
      file, in the same order:

      .. code-block:: json

         {
             "neurecoODE_evaluate": {
                 "ernn_filename": "./model/ode_model.ernn",
                 "exc_filenames": ["data/test/test_exc_1.csv"],
                 "init_exc_filenames": ["data/test/init_exc_1.csv"],
                 "init_output_filenames": ["data/test/init_out_1.csv"],
                 "write_model_output_to_directory": "./results/"
             }
         }

      - Give both lists, with as many entries as ``exc_filenames``.
      - The initialization files hold the initialization window (Mode 1 above), with the
        time in the first column. Both files must have the same time values.
      - The excitation file must start at the last time of the initialization window, and
        the result file starts there too.

Method Reference
================

.. code-block:: python

   predictions = model.evaluate(
       excitations=None,        # Input array(s), time in column 0
       init_excitations=None,   # Initialization inputs (required for explicit init), time in column 0
       init_outputs=None,       # Initialization outputs (required for explicit init), time in column 0
       *,
       time=None,               # Deprecated: separate time array(s)
       init_time=None,          # Deprecated: separate initialization time
   )

**Parameters:**

.. list-table::
   :header-rows: 1
   :widths: 20 15 65

   * - Parameter
     - Required
     - Description
   * - ``excitations``
     - Yes
     - Input array(s) with time in the first column. Shape ``(n_samples, 1 + n_inputs)`` or a list of such arrays.
   * - ``init_excitations``
     - No
     - Initialization input values, time in the first column (required for explicit init)
   * - ``init_outputs``
     - No
     - Initialization output values, time in the first column (required for explicit init)
   * - ``time``
     - No
     - **Deprecated** (keyword-only). Separate time array(s); emits a ``DeprecatedParameterWarning`` and returns predictions without a leading time column.
   * - ``init_time``
     - No
     - **Deprecated** (keyword-only). Separate initialization time values.

**Returns:**

- Single sequence: NumPy array of shape ``(n_samples, 1 + n_outputs)``, with time in the first column
- Multiple sequences: List of such NumPy arrays

.. note::

   When ``excitations`` carries the time in its first column, the returned
   predictions are **time-first** (time in column 0). The deprecated
   separate-``time`` form returns predictions **without** a leading time column.

   Scripts written for NeurEco 4.x that call
   ``evaluate(time, excitations, init_time, init_excitations, init_outputs)`` with
   positional arguments still run: this form emits a ``DeprecatedParameterWarning``
   and also returns predictions without a leading time column.

Using Known Initial Conditions from Data
========================================

When you have reference data and want to evaluate starting from a known point:

.. code-block:: python

   import numpy as np
   from NeurEco import NeurEcoDynamic as Dynamic

   # Load model
   model = Dynamic.ODE()
   model.load("my_ode_model.ernn")

   # Reference data (time in column 0)
   time_full = np.linspace(0, 10, 1000)
   exc_full = np.column_stack([time_full, np.random.randn(1000, 3)])
   out_full = np.column_stack([time_full, np.random.randn(1000, 2)])  # Known ground truth

   # Use first N points as initialization
   n_init = model.get_number_of_init_tsteps()

   # Both init arrays keep their time in the first column
   init_exc = exc_full[:n_init]
   init_out = out_full[:n_init]

   # Evaluate from the last initialization point onward
   # Note: The last init point is the first prediction point (overlap)
   eval_exc = exc_full[n_init - 1:]

   # eval_exc / init_exc / init_out all carry time in their first column;
   # predictions are returned time-first.
   outputs = model.evaluate(
       eval_exc,
       init_excitations=init_exc,
       init_outputs=init_out,
   )

   # Compare to ground truth (starting from the same point, also time-first).
   # compute_error takes outputs only, so drop the leading time column from both.
   ground_truth = out_full[n_init - 1:]
   error = model.compute_error(outputs[:, 1:], ground_truth[:, 1:])
   print(f"L2 relative error: {error:.6f}")

Retrieving Steady-State Values
==============================

To inspect the steady-state values learned by the model:

.. code-block:: python

   steady_inputs = model.get_steady_input()
   steady_outputs = model.get_steady_output()

   print(f"Steady-state inputs: {steady_inputs}")
   print(f"Steady-state outputs: {steady_outputs}")

Model Information
=================

.. code-block:: python

   # Number of inputs and outputs
   print(f"Inputs: {model.get_input_count()}")
   print(f"Outputs: {model.get_output_count()}")

   # Number of hidden states
   print(f"Hidden states: {model.get_state_count()}")

   # Initialization steps required
   print(f"Init steps: {model.get_number_of_init_tsteps()}")

Best Practices
==============

1. **Match time step**: The evaluation time step must match the training time step

2. **Use explicit initialization when possible**: For best accuracy, provide known initial conditions

3. **Verify initialization length**: Use ``get_number_of_init_tsteps()`` to determine how many initialization points are needed

4. **Check steady state**: If using steady-state initialization, verify the learned steady state matches your expectations

Next Steps
==========

- :ref:`Export Dynamic models <dynamic-exporting>`
- :ref:`Dynamic advanced features <dynamic-advanced-features>`