.. _pfs-workflow:

======================
PFS Complete Workflow
======================

This page covers the complete workflow for NeurEco Parametric Frequency Sweep (PFS) models: data preparation, building, evaluating, and exporting.

.. contents:: On this page
   :local:
   :depth: 2

Data Preparation
================

Data Structure
--------------

PFS models require:

- **Input data**: Frequency and parameters — shape ``(n_samples, n_inputs)`` where ``n_inputs >= 2``
- **Output data**: Complex frequency responses — shape ``(n_samples, n_outputs)``

.. warning::

   **The first column of input data must ALWAYS be the frequency.**

   This requirement applies in all cases:

   - During training
   - During evaluation
   - Even when evaluating at a single frequency (the frequency column must still be present)

   The model expects frequency as input index 0 — this cannot be changed or omitted.

Data Types
----------

.. list-table::
   :header-rows: 1
   :widths: 20 40 40

   * - Data
     - Python/NumPy
     - CSV
   * - Inputs
     - ``float64``
     - ``float`` or ``double``
   * - Outputs
     - ``complex128``
     - Complex notation (see below)

**Complex Number Formats in CSV:**

- ``0.1+0.1j`` — Standard Python notation
- ``(0.1+0.1j)`` — With parentheses
- ``(0.1,0.1)`` — Tuple notation (real, imag)

File Formats
------------

.. tabs::

   .. tab:: Python API

      Use NumPy arrays or lists of arrays:

      .. code-block:: python

         import numpy as np

         # Load from .npy files
         x_train = np.load('inputs_train.npy')
         y_train = np.load('targets_train.npy')

         # Or create programmatically
         # Column 0: frequency, Column 1+: parameters
         inputs = np.zeros((n_samples, n_inputs), dtype=np.float64)
         outputs = np.zeros((n_samples, n_outputs), dtype=np.complex128)

   .. tab:: GUI / CLI

      Supported formats:

      - **CSV** with ``;`` or ``,`` separator
      - **NumPy** ``.npy`` files
      - **MATLAB** ``.mat`` files

      **Input file structure:**

      .. code-block:: text

         frequency;param1;param2;param3
         1.0;0.5;1.2;0.8
         2.0;0.5;1.2;0.8
         3.0;0.5;1.2;0.8
         ...

      **Output file structure (complex values):**

      .. code-block:: text

         output1;output2
         0.1+0.2j;0.3+0.4j
         0.15+0.18j;0.32+0.41j
         ...

Data Normalization
------------------

NeurEco automatically normalizes PFS data:

**Input normalization** (per feature):

.. math::

   x_{normalized} = \frac{x - \min(x)}{\max(x) - \min(x)}

**Output normalization** (global):

.. math::

   y_{normalized} = \frac{y}{\max(|y|)}

Building a Model
================

.. tabs::

   .. tab:: GUI

      **Starting a New Project**

      1. Launch NeurEco
      2. Select **Parametric Frequency Sweep**
      3. Click **New Project**

      **Loading Data**

      1. In the **Build** panel, click **+** next to **Training files**
      2. Select your input file (frequency + parameters)
      3. Select your output file (complex responses)
      4. Repeat for validation and test files if available

      **Build Parameters**

      .. list-table::
         :header-rows: 1
         :widths: 30 70

         * - Parameter
           - Description
         * - Valid percentage
           - Percentage of training data for validation (if no validation files provided)
         * - Reduced space dimension
           - Dimension of compressed output space (controls internal complexity)
         * - Min enrichment steps
           - Minimum number of enrichment iterations
         * - Max enrichment steps
           - Maximum number of enrichment iterations (upper limit)
         * - Unsuccessful steps
           - Consecutive failures before early stopping
         * - Enrichment rate
           - Rate of parameter addition per step (0 < rate <= 1)

      **Building**

      1. Click **BUILD**
      2. Monitor progress in the **Training** panel
      3. Checkpoints are saved automatically

   .. tab:: Python API

      **Basic Build**

      .. code-block:: python

         from NeurEco import NeurEcoFrequential as Frequential
         import numpy as np

         # Load data
         x_train = np.load('inputs_train.npy')
         y_train = np.load('targets_train.npy')
         x_valid = np.load('inputs_valid.npy')
         y_valid = np.load('targets_valid.npy')

         # Initialize model
         model = Frequential.PFS()

         # Configure build
         config = Frequential.PFSBuildConfig(
             # Required: training data
             input_data=x_train,
             output_data=y_train,
             # Optional: validation data
             validation_input_data=x_valid,
             validation_output_data=y_valid,
             # Output paths
             write_model_to="./model/pfs_model.efnn",
             checkpoint_address="./model/pfs_checkpoint",
         )

         # Build
         status = model.build(config)

         if status == 0:
             print("Build completed successfully!")

      **Advanced Build Configuration**

      .. code-block:: python

         config = Frequential.PFSBuildConfig(
             # Training data
             input_data=x_train,
             output_data=y_train,
             # Validation data
             validation_input_data=x_valid,
             validation_output_data=y_valid,
             # Test data (for final evaluation only)
             test_input_data=x_test,
             test_output_data=y_test,
             # Validation split (used only if validation data not provided)
             valid_percentage=33.33,
             # Model architecture
             reduced_space_dimension=5,
             # Training control
             min_enrichment_steps=10,
             max_enrichment_steps=200,
             unsuccessful_enrichment_steps=4,
             enrichment_rate=0.2,
             # Output paths
             write_model_to="./model/pfs_model.efnn",
             checkpoint_address="./model/pfs_checkpoint",
         )

   .. tab:: CLI

      **Configuration File**

      .. code-block:: json

         {
             "neurecoPFS_build": {
                 "input_filenames": ["data/inputs_train.npy"],
                 "output_filenames": ["data/targets_train.npy"],
                 "validation_input_filenames": ["data/inputs_valid.npy"],
                 "validation_output_filenames": ["data/targets_valid.npy"],
                 "settings": {
                     "validation_percentage": 33.33,
                     "compressed_space_size": 5,
                     "min_number_of_enrichments": 10,
                     "max_number_of_enrichments": 200,
                     "unsuccessful_enrichments": 4,
                     "enrichment_rate": 0.2
                 },
                 "write_model_to": "./model/pfs_model.efnn",
                 "checkpoint_address": "./model/pfs_checkpoint.checkpoint",
                 "write_model_output_to_directory": "./model/BuildingResults"
             }
         }

      The configuration file names four settings differently from Python:
      ``compressed_space_size`` (``reduced_space_dimension``),
      ``min_number_of_enrichments`` (``min_enrichment_steps``),
      ``max_number_of_enrichments`` (``max_enrichment_steps``) and
      ``unsuccessful_enrichments`` (``unsuccessful_enrichment_steps``).

      **Running the Build**

      .. code-block:: shell

         neureco PFS build config.json

Parameter Reference
-------------------

.. list-table::
   :header-rows: 1
   :widths: 30 15 55

   * - Parameter
     - Default
     - Description
   * - ``input_data``
     - (required)
     - Training inputs. Shape: ``(n_samples, n_inputs)``, first column = frequency
   * - ``output_data``
     - (required)
     - Training outputs. Shape: ``(n_samples, n_outputs)``, dtype: complex
   * - ``validation_input_data``
     - ``None``
     - Validation inputs (optional)
   * - ``validation_output_data``
     - ``None``
     - Validation outputs (optional)
   * - ``valid_percentage``
     - ``33.33``
     - Validation split (1–50%, used if no validation data)
   * - ``reduced_space_dimension``
     - ``5``
     - Compressed output space dimension
   * - ``min_enrichment_steps``
     - ``10``
     - Minimum enrichment iterations
   * - ``max_enrichment_steps``
     - ``200``
     - Maximum enrichment iterations
   * - ``unsuccessful_enrichment_steps``
     - ``4``
     - Failures before early stopping
   * - ``enrichment_rate``
     - ``0.2``
     - Parameter addition rate (0 < rate <= 1)

Working with Checkpoints
------------------------

During training, NeurEco saves intermediate models. You can load and compare them:

.. code-block:: python

   model = Frequential.PFS()

   # Get number of checkpointed models
   n_models = model.get_number_of_networks_from_checkpoint("./model/pfs_checkpoint")
   print(f"Found {n_models} checkpointed models")

   # Evaluate each checkpoint
   for i in range(n_models):
       model.load_model_from_checkpoint("./model/pfs_checkpoint", i)
       outputs = model.evaluate(x_test)
       error = model.compute_error(outputs, y_test)
       denom_size = model.get_denominator_size()
       print(f"Model {i}: error={100*error:.2f}%, denominator_size={denom_size}")

Evaluating a Model
==================

.. tabs::

   .. tab:: GUI

      1. Go to the **Evaluation** panel
      2. Load evaluation data
      3. Click **Evaluate**
      4. View results in the plot panel
      5. Check **Metrics** panel for error statistics

   .. tab:: Python API

      **Loading and Evaluating**

      .. code-block:: python

         from NeurEco import NeurEcoFrequential as Frequential
         import numpy as np

         # Load model
         model = Frequential.PFS()
         model.load("./model/pfs_model.efnn")

         # Get model info
         print(f"Inputs: {model.get_input_count()}")
         print(f"Outputs: {model.get_output_count()}")
         print(f"Denominator size: {model.get_denominator_size()}")

         # Evaluate
         predictions = model.evaluate(x_test)
         print(f"Output shape: {predictions.shape}")
         print(f"Output dtype: {predictions.dtype}")  # complex128

         # Compute error
         error = model.compute_error(predictions, y_test)
         print(f"L2 relative error: {100 * error:.4f}%")

      **Visualizing Frequency Response**

      .. code-block:: python

         import matplotlib.pyplot as plt

         # Evaluate over frequency range
         frequencies = np.linspace(1, 500, 200)
         param_value = 0.05

         inputs = np.column_stack([frequencies, np.full(200, param_value)])
         outputs = model.evaluate(inputs)

         # Plot magnitude and phase
         fig, axes = plt.subplots(2, 1, figsize=(10, 6))

         # Magnitude (dB)
         magnitude_db = 20 * np.log10(np.abs(outputs[:, 0]))
         axes[0].plot(frequencies, magnitude_db)
         axes[0].set_xlabel("Frequency (Hz)")
         axes[0].set_ylabel("Magnitude (dB)")
         axes[0].grid(True)

         # Phase (degrees)
         phase_deg = np.angle(outputs[:, 0], deg=True)
         axes[1].plot(frequencies, phase_deg)
         axes[1].set_xlabel("Frequency (Hz)")
         axes[1].set_ylabel("Phase (degrees)")
         axes[1].grid(True)

         plt.tight_layout()
         plt.show()

   .. tab:: CLI

      **Configuration File**

      .. code-block:: json

         {
             "neurecoPFS_evaluate": {
                 "neureco_filename": "./model/pfs_model.efnn",
                 "input_filenames": ["data/inputs_test.npy"],
                 "write_model_output_to_directory": "./results/"
             }
         }

      **Running Evaluation**

      .. code-block:: shell

         neureco PFS evaluate config.json

Input Sweep Analysis
--------------------

Analyze how one input affects the output while keeping others constant:

.. code-block:: python

   # Define a reference input sample
   input_sample = np.array([100.0, 0.05, 1.0, 0.5])  # freq, param1, param2, param3

   # Sweep the frequency (input_id=0) from 1 to 500 Hz
   model.perform_input_sweep(
       input_feature=input_sample,
       input_id=0,              # Sweep frequency
       input_interval=[1, 500], # Range
       output_id=0,             # Which output to plot
       n_points=200,
       show=True,
       save_path="frequency_sweep.png"
   )

Exporting a Model
=================

.. tabs::

   .. tab:: Python API

      **Export to FMU**

      .. code-block:: python

         status = model.export_fmu("./model/pfs_model.fmu")

         if status == 0:
             print("Model exported to FMU successfully")

      .. note::

         FMU export requires a **NeurEco Embed PFS license**.

      **Save in Native Format**

      .. code-block:: python

         model.save("./model/pfs_model_copy.efnn")

   .. tab:: GUI

      1. Go to the **Export** panel
      2. Select export format (FMU)
      3. Choose destination path
      4. Click **Export**

   .. tab:: CLI

      .. code-block:: shell

         neureco PFS exportFMU ./model/pfs_model.efnn ./model/pfs_model.fmu 3

      The arguments are the model file, the ``.fmu`` file to write and the platform
      flag: use ``3`` (Windows and Linux).

Method Reference
================

.. list-table::
   :header-rows: 1
   :widths: 35 65

   * - Method
     - Description
   * - ``build(config)``
     - Train a PFS model with the given configuration
   * - ``load(path)``
     - Load a saved model from disk
   * - ``save(path)``
     - Save the model to disk
   * - ``evaluate(inputs)``
     - Run inference on input data, returns complex outputs
   * - ``compute_error(predictions, targets)``
     - Compute L2 relative error between predictions and targets
   * - ``get_input_count()``
     - Get number of input features
   * - ``get_output_count()``
     - Get number of output features
   * - ``get_denominator_size()``
     - Get denominator size of the rational approximation
   * - ``perform_input_sweep(...)``
     - Sweep one input and visualize the response
   * - ``export_fmu(path)``
     - Export model to FMU format
   * - ``get_number_of_networks_from_checkpoint(path)``
     - Get number of models in a checkpoint
   * - ``load_model_from_checkpoint(path, model_index)``
     - Load a specific model from checkpoint
   * - ``delete()``
     - Free memory (call when done)

Best Practices
==============

1. **Always put frequency first**: The first column of input data must be the frequency

2. **Use complex128 for outputs**: Ensure your output data is ``complex128`` dtype

3. **Explore checkpoints**: Compare intermediate models to find the best accuracy/complexity trade-off

4. **Monitor denominator size**: Larger denominators = more complex models, potentially better accuracy but slower inference

5. **Clean up**: Call ``model.delete()`` when finished to free memory

Next Steps
==========

- :ref:`PFS Examples <pfs-examples>`