.. _convolutive-building: ===================================== Building Convolutional Dynamic Models ===================================== This page explains how to train NeurEco Convolutional Dynamic models using the GUI, Python API, and CLI. .. contents:: On this page :local: :depth: 2 Prerequisites ============= Before building, ensure your data is properly formatted: - Time-series data with uniform time steps - Excitation (input) and output arrays - See :ref:`dynamic-data-preparation` for details Building a Convolutional Dynamic Model ====================================== .. tabs:: .. tab:: GUI **Starting a New Project** 1. Launch NeurEco 2. Select **Dynamic** > **CNN Dynamic** 3. Click **New Project** **Loading Data** *Training data:* 1. In the **Build** panel, click **+** next to **Training files** 2. Select your input (excitation) file 3. Select the corresponding output file 4. Repeat for additional sequences/experiments *Validation data (optional):* - Add files under **Validation files** - If not provided, NeurEco uses a percentage of training data *Test data (optional):* - Add files under **Test files** - Used only for final evaluation, not during training **Build Parameters** The **CNN Dynamic** panel shows the following settings. *Hardware:* - **Use GPU**: train on a CUDA GPU (shown only when one is available); **GPU** selects which one *Normalization:* - **Input normalization** and **Output normalization**: each has a **Shift type**, a **Scale type** and **normalize per feature** (when on, each feature is normalized separately; when off, all features are normalized together) *Training data settings:* - **Validation data percentage** (default: 30): the percentage of the training data that NeurEco keeps for its internal validation; ignored when validation data is set - **Window stride** (default: 1): the step between two consecutive training windows. 1 uses every possible window; a larger value keeps fewer windows, so the build is faster but learns from less data. It only affects the training data: the model still predicts every time step *Convolutional network:* - **Window size** (default: 32, minimum: 3): the number of consecutive time steps the network reads to make one prediction, that is, the memory length of the model and the input length of the network. It cannot be longer than the shortest trajectory - **Last layer grid number** (default: 0): the number of kernels of the last convolutional layer, the one that feeds the Tabular regression; 0 lets NeurEco choose (up to 20) - **Tolerance last layer** (default: 0.1): only used when **Last layer grid number** is 0. A new kernel of the last layer is kept only if it divides the validation error by at least 1 + this value - **Layer multiplier** (default: 2): each convolutional layer has this many times more kernels than the layer after it - **Hidden layer number** (default: 2): the maximum number of convolutional layers - **Padding**: how each convolutional layer extends the window at its two edges so that its kernels can be applied there. **Mode** is Constant (fill with **Value**), Reflect (mirror the signal around the edge) or Replicate (repeat the edge value) See :ref:`convolutive-window-stride` and the Parameter Reference below for more details. **Building** 1. Click **BUILD** 2. Monitor training progress in the **Training** panel 3. Checkpoints are saved automatically 4. Training completes when convergence is reached .. tab:: Python API **Setup** .. code-block:: python from NeurEco import NeurEcoDynamic as Dynamic import numpy as np # Initialize model model = Dynamic.ConvolutiveDynamic() **Build Configuration** Create a ``ConvolutiveDynamicBuildConfig`` with your parameters: .. code-block:: python # Each array: column 0 = time, remaining columns = excitations / outputs. config = Dynamic.ConvolutiveDynamicBuildConfig( # Required: Training data train_exc_list=train_exc_list, train_out_list=train_out_list, # Optional: Validation data valid_exc_list=None, valid_out_list=None, # Optional: Test data test_exc_list=None, test_out_list=None, # Validation split (if validation data not provided) valid_percentage=30.0, # Output paths write_model_to="./model/convolutive_model.ecnn", checkpoint_address="./model/convolutive_checkpoint", # Normalization inputs_scaling="minmax", inputs_shifting="min", outputs_scaling="minmax", outputs_shifting="min", inputs_normalize_per_feature=True, outputs_normalize_per_feature=True, # Training windows (training data only) window_stride=1, # Convolution parameters window_size=32, activation_function="Tanh", padding_mode="CONSTANT", padding_value=0.0, # Network architecture hidden_layer_number=2, layer_multiplier=2.0, last_layer_grid_number=0, tol_last_layer=0.1, ) **Building the Model** .. code-block:: python status = model.build(config) if status == 0: print("Build completed successfully!") print(f"Window size: {model.get_window_size()}") else: print(f"Build failed with status code: {status}") .. tab:: CLI **Configuration File** Create a JSON configuration file: .. code-block:: json { "neurecoConvolutiveDynamic_build": { "data": { "training": { "inputs": [ "data/train/train_exc_1.csv", "data/train/train_exc_2.csv" ], "targets": [ "data/train/train_out_1.csv", "data/train/train_out_2.csv" ] }, "validation": { "inputs": ["data/valid/valid_exc_1.csv"], "targets": ["data/valid/valid_out_1.csv"] }, "test": { "inputs": ["data/test/test_exc_1.csv"], "targets": ["data/test/test_out_1.csv"] }, "validation_percentage": 30 }, "normalizers": { "inputs": { "shift_type": "min", "scale_type": "minmax", "normalize_per_feature": true }, "targets": { "shift_type": "min", "scale_type": "minmax", "normalize_per_feature": true } }, "settings": { "window_size": 32, "window_stride": 1, "hidden_layer_number": 2, "layer_multiplier": 2.0, "last_layer_grid_number": 0, "tol_last_layer": 0.1, "activation_function": "Tanh", "padding_mode": "CONSTANT", "padding_value": 0.0 }, "write_model_to": "./model/convolutive_model.ecnn", "checkpoint_address": "./model/convolutive_checkpoint.checkpoint", "write_model_output_to_directory": "./model/BuildingResults", "resume": false } } The data files sit under ``data``, as the ``inputs`` (excitations) and ``targets`` (outputs) of the ``training``, ``validation`` and ``test`` sets, the normalization under ``normalizers``, and the other build parameters under ``settings``. The file names follow the layout of ``TutorialCases/NonLinearOscillator``. See :ref:`cli-reference` for all the fields. **Running the Build** .. code-block:: shell neureco ConvolutiveDynamic build config.json Parameter Reference =================== Training Data ------------- .. list-table:: :header-rows: 1 :widths: 25 15 60 * - Parameter - Required - Description * - ``train_exc_list`` - Yes - List of 2-D arrays; column 0 is time, remaining columns are input signals * - ``train_out_list`` - Yes - List of 2-D arrays; column 0 is time, remaining columns are target outputs Validation/Test Data -------------------- .. list-table:: :header-rows: 1 :widths: 25 15 60 * - Parameter - Default - Description * - ``valid_exc_list`` - ``None`` - Validation input arrays (time in the first column) * - ``valid_out_list`` - ``None`` - Validation output arrays (time in the first column) * - ``test_exc_list`` - ``None`` - Test input arrays (time in the first column) * - ``test_out_list`` - ``None`` - Test output arrays (time in the first column) * - ``valid_percentage`` - ``30.0`` - Percentage of training data for validation .. note:: **Deprecated separate-time parameters.** ``train_time_list``, ``valid_time_list`` and ``test_time_list`` (lists of 1-D time arrays passed alongside pure excitation/output arrays) are still accepted for backward compatibility but emit a ``DeprecatedParameterWarning``. Prefer placing the time in the first column of each excitation and output array. Training Windows ---------------- .. list-table:: :header-rows: 1 :widths: 25 15 60 * - Parameter - Default - Description * - ``window_stride`` - ``1`` - Training data only: step between two consecutive training windows; ``1`` uses every window. The model still predicts every time step (see :ref:`convolutive-window-stride`) Output Paths ------------ .. list-table:: :header-rows: 1 :widths: 25 15 60 * - Parameter - Default - Description * - ``write_model_to`` - ``"model"`` - Path for final trained model * - ``checkpoint_address`` - ``"ckpt"`` - Path for intermediate checkpoints * - ``resume`` - ``False`` - ``True``: continue the build stored in ``checkpoint_address`` (see :ref:`convolutive-resume`) Normalization ------------- .. list-table:: :header-rows: 1 :widths: 30 15 55 * - Parameter - Default - Description * - ``inputs_scaling`` - ``"minmax"`` - Options: ``"none"``, ``"max"``, ``"minmax"``, ``"std"`` * - ``inputs_shifting`` - ``"min"`` - Options: ``"none"``, ``"mean"``, ``"min"`` * - ``outputs_scaling`` - ``"minmax"`` - Options: ``"none"``, ``"max"``, ``"minmax"``, ``"std"`` * - ``outputs_shifting`` - ``"min"`` - Options: ``"none"``, ``"mean"``, ``"min"`` * - ``inputs_normalize_per_feature`` - ``True`` - Normalize each input feature independently * - ``outputs_normalize_per_feature`` - ``True`` - Normalize each output feature independently Convolution Parameters ---------------------- .. list-table:: :header-rows: 1 :widths: 25 15 60 * - Parameter - Default - Description * - ``window_size`` - ``32`` - Number of past time steps seen per prediction, the model's memory length; at least ``3`` (see :ref:`convolutive-window-stride`) * - ``activation_function`` - ``"Tanh"`` - Activation function of the convolutional layers * - ``padding_mode`` - ``"CONSTANT"`` - How each convolutional layer extends the window at its two edges so that its kernels can be applied there. One of ``"CONSTANT"``, ``"REFLECT"``, ``"REPLICATE"`` (see the note below) * - ``padding_value`` - ``0.0`` - The value used when ``padding_mode`` is ``"CONSTANT"`` (ignored for the other modes) .. note:: **Padding.** Near the two edges of the window (its first and last time steps), a convolution kernel would reach past the window. Before each convolutional layer, NeurEco extends the layer input at both edges, and ``padding_mode`` sets how the extra values are filled: - ``"CONSTANT"``: fill with the fixed ``padding_value`` (default ``0.0``); - ``"REFLECT"``: mirror the values around the edge; - ``"REPLICATE"``: repeat the edge value. Padding acts inside each window only. It does not let the model predict before the first full window: the first ``window_size - 1`` time steps of a sequence are not predicted. Network Architecture -------------------- .. list-table:: :header-rows: 1 :widths: 25 15 60 * - Parameter - Default - Description * - ``hidden_layer_number`` - ``2`` - Maximum number of convolutional layers; at least ``1`` * - ``layer_multiplier`` - ``2.0`` - Each convolutional layer has this many times more kernels than the layer after it * - ``last_layer_grid_number`` - ``0`` - Number of kernels of the last convolutional layer, the one that feeds the Tabular regression; ``0`` lets NeurEco choose the number automatically (up to 20) * - ``tol_last_layer`` - ``0.1`` - Automatic mode only: a new kernel of the last convolutional layer is kept only if it divides the validation error by at least ``1 + tol_last_layer`` (see the note below); must be greater than ``0`` .. note:: **Automatic last layer.** When ``last_layer_grid_number`` is ``0`` (the default), NeurEco builds the last convolutional layer one kernel at a time, up to 20 kernels. It keeps a new kernel only if it divides the validation error by at least ``1 + tol_last_layer`` (with the default ``0.1``, the error must drop by about 9%), and it also stops when the network becomes too large for the amount of training data. If you instead set ``last_layer_grid_number`` to a fixed value, NeurEco uses that many kernels and ``tol_last_layer`` has no effect. Choosing Window Size and Stride =============================== The window size determines how much input history the model considers (its memory length); the stride only sets how many training windows are cut from the build data. See :ref:`convolutive-window-stride` for the full explanation of both parameters. .. code-block:: python # Small window: fast training, may miss long-term patterns config = Dynamic.ConvolutiveDynamicBuildConfig( # ... window_size=16, ) # Large window: captures more history, slower training config = Dynamic.ConvolutiveDynamicBuildConfig( # ... window_size=128, ) **Guidelines:** - Start with ``window_size=32`` or ``64`` - Increase if validation error is high and you suspect longer-term dependencies - Decrease if training is too slow or data has short-term correlations - Keep ``window_stride=1`` (the default) to train on every window; increase it only to speed up the build on long trajectories, at the cost of fewer training windows (the model still predicts every time step) .. _convolutive-resume: Resuming a Build ================ If a Convolutional Dynamic build stops before the end, resume it from its checkpoint instead of starting again. Keep the same data, the same settings and the same ``checkpoint_address``: NeurEco reads that checkpoint, keeps the models it holds and continues the build at the stage recorded in its last model. See :ref:`dynamic-resume` for what happens when the checkpoint cannot be read. .. tabs:: .. tab:: GUI Stop the build with **STOP**. While the checkpoint holds at least one model, the build button then reads **RESUME**: click it to continue the build. A build stopped before its first checkpoint has nothing to resume: the button reads **BUILD** and starts a new build. .. tab:: Python API .. code-block:: python config = Dynamic.ConvolutiveDynamicBuildConfig( train_exc_list=train_exc_list, train_out_list=train_out_list, write_model_to="./model/convolutive_model.ecnn", checkpoint_address="./model/convolutive_checkpoint", window_size=32, hidden_layer_number=2, ) model = Dynamic.ConvolutiveDynamic() try: model.build(config) # Ctrl+C stops the build except KeyboardInterrupt: print("Build stopped: the checkpoint keeps the models built so far") # Same data, same settings, same checkpoint_address config.resume = True model = Dynamic.ConvolutiveDynamic() model.build(config) .. tab:: CLI Set ``"resume": true`` in the configuration file of the stopped build and run the same command again: .. code-block:: shell neureco ConvolutiveDynamic build config.json Complete Example ================ .. code-block:: python import numpy as np from NeurEco import NeurEcoDynamic as Dynamic # Prepare training data (time in column 0) dt = 0.01 t_train = np.arange(0, 10, dt) train_exc_list = [np.column_stack([t_train, np.random.randn(len(t_train), 3)])] train_out_list = [np.column_stack([t_train, np.random.randn(len(t_train), 2)])] # Prepare validation data (time in column 0) t_valid = np.arange(0, 5, dt) valid_exc_list = [np.column_stack([t_valid, np.random.randn(len(t_valid), 3)])] valid_out_list = [np.column_stack([t_valid, np.random.randn(len(t_valid), 2)])] # Initialize model model = Dynamic.ConvolutiveDynamic() # Configure build # Each array: column 0 = time, remaining columns = excitations / outputs. config = Dynamic.ConvolutiveDynamicBuildConfig( train_exc_list=train_exc_list, train_out_list=train_out_list, valid_exc_list=valid_exc_list, valid_out_list=valid_out_list, write_model_to="./model/my_convolutive_model.ecnn", checkpoint_address="./model/my_convolutive_checkpoint", window_size=64, hidden_layer_number=3, ) # Build the model status = model.build(config) if status == 0: print("Model trained successfully!") print(f"Window size: {model.get_window_size()}") Next Steps ========== - :ref:`Evaluate Convolutional Dynamic models ` - :ref:`Export Dynamic models `