.. _example-heaviside:

=====================
Heaviside Compression
=====================

This example demonstrates Tabular Compression using signal data with step functions (Heaviside-like patterns).

.. contents:: On this page
   :local:
   :depth: 2

Problem Description
===================

**Goal:** Compress and reconstruct signal data, discovering underlying patterns.

**Input:**

- Signal samples with step-like (Heaviside) patterns
- High-dimensional measurements per sample

**Output:**

- Compressed representation (learned automatically)
- Reconstruction of original signal

This example showcases:

- Dimensionality reduction
- Pattern discovery in latent space
- Autoencoder capabilities

Dataset
=======

The dataset contains:

- ``x_train.csv`` — Training signal samples
- No separate output file (compression learns to reconstruct input)

Building the Model
==================

.. tabs::

   .. tab:: GUI

      1. Launch NeurEco and select **Tabular Compression**
      2. Load ``x_train.csv`` as input (no output file needed)
      3. Set compression parameters:
         - **Compression tolerance:** 0.01 (1% error)
         - **Minimum compression coefficients:** 1
      4. Click **BUILD**
      5. Observe the learned compression dimension

   .. tab:: Python API

      .. code-block:: python

         from NeurEco import NeurEcoTabular as Tabular
         import numpy as np
         import matplotlib.pyplot as plt

         # Load data (no separate output needed)
         x_train = np.genfromtxt("x_train.csv", delimiter=";", skip_header=True)
         print(f"Input dimension: {x_train.shape[1]}")

         # Create and build model
         model = Tabular.Compressor()

         config = Tabular.CompressorBuildConfig(
             input_data=x_train,
             write_model_to="./heaviside_model/complete.ednn",
             write_compression_model_to="./heaviside_model/encoder.ednn",
             write_decompression_model_to="./heaviside_model/decoder.ednn",
             checkpoint_address="./heaviside_model/heaviside.checkpoint",
             compress_tolerance=0.01,
             minimum_compression_coefficients=1,
             valid_percentage=33.33,
             # Match the CLI configuration so both tabs build the same compressor
             inputs_shifting="min_centered",
             inputs_scaling="max_centered",
             inputs_normalize_per_feature=True,
         )

         model.build(config)

         # Test reconstruction
         reconstructed = model.evaluate(x_train[:10])
         error = model.compute_error(reconstructed, x_train[:10])
         print(f"Reconstruction L2 Relative Error: {error:.6f}")

         # Load encoder to check compression dimension
         encoder = Tabular.Compressor()
         encoder.load("./heaviside_model/encoder.ednn")

         compressed = encoder.evaluate(x_train)
         print(f"Compressed dimension: {compressed.shape[1]}")
         print(f"Compression ratio: {x_train.shape[1] / compressed.shape[1]:.1f}x")

         encoder.delete()
         model.delete()

   .. tab:: CLI

      Create ``build_heaviside.conf``:

      .. code-block:: json

         {
             "neurecoCompression_build": {
                 "DevSettings": {
                     "valid_percentage": 33.33
                 },
                 "compress_tolerance": 0.01,
                 "minimum_compression_coefficients": 1,
                 "exc_filenames": ["x_train.csv"],
                 "input_normalization": {
                     "normalize_per_feature": true,
                     "scale_type": "max_centered",
                     "shift_type": "min_centered"
                 },
                 "output_normalization": {
                     "normalize_per_feature": true,
                     "scale_type": "max_centered",
                     "shift_type": "min_centered"
                 },
                 "write_model_to": "./heaviside_model/complete.ednn",
                 "write_compression_model_to": "./heaviside_model/encoder.ednn",
                 "write_decompression_model_to": "./heaviside_model/decoder.ednn",
                 "checkpoint_address": "./heaviside_model/heaviside.checkpoint"
             }
         }

      Run:

      .. code-block:: shell

         neureco Compression build build_heaviside.conf

Analyzing Compression Coefficients
==================================

One of the most interesting aspects of compression is analyzing the latent space. With the settings above, NeurEco typically finds 2 compression coefficients for this dataset. The ``Compressor`` model provides a built-in method for visualization.

**Visualizing coefficient relationships:**

.. code-block:: python

   # Load the complete model (or encoder)
   model = Tabular.Compressor()
   model.load("./heaviside_model/complete.ednn")

   # 2D visualization of latent space
   model.plot_compression_coefficients(
       data_to_compress=x_train,
       neurons_ids=[0, 1],
       title="Latent Space Visualization",
       save_path="latent_space_2d.png"
   )

The latent space typically shows a sun-like pattern radiating from the center outward, revealing the underlying structure of the Heaviside signals.

**Coloring by signal amplitude:**

You can color points by data properties to reveal patterns:

.. code-block:: python

   # Compute amplitudes for each sample
   amplitudes = np.array([np.max(sample) for sample in x_train])

   # Create class labels based on amplitude threshold
   classes = np.where(amplitudes > 0.2, "high", "low")

   # Plot compression coefficients colored by amplitude class
   model.plot_compression_coefficients(
       data_to_compress=x_train,
       neurons_ids=[0, 1],
       data_labels=list(classes),
       title="Latent Space by Signal Amplitude"
   )

This visualization shows how signals with different amplitudes are distributed in the compressed space.

Visualizing Reconstruction Quality
==================================

.. code-block:: python

   # Select a few samples
   sample_indices = [0, 10, 20]

   fig, axes = plt.subplots(len(sample_indices), 1, figsize=(12, 8))

   for idx, sample_idx in enumerate(sample_indices):
       original = x_train[sample_idx]
       reconstructed = model.evaluate(x_train[sample_idx:sample_idx+1])[0]

       axes[idx].plot(original, 'b-', label='Original', alpha=0.7)
       axes[idx].plot(reconstructed, 'r--', label='Reconstructed', alpha=0.7)
       axes[idx].set_title(f'Sample {sample_idx}')
       axes[idx].legend()

   plt.tight_layout()
   plt.savefig('reconstruction_comparison.png')
   plt.show()

Expected Results
================

After training:

- **Significant compression** — a big reduction in dimensions depending on the build settings (compress_tolerance)
- **Low reconstruction error** — Within the specified tolerance

Key Takeaways
=============

This example demonstrates:

1. **Automatic dimensionality reduction** — NeurEco finds the optimal latent dimension
2. **Pattern discovery** — Latent space visualization reveals data structure
3. **Separate encoder/decoder** — Useful for IoT and distributed systems
4. **Reconstruction quality** — Controlled by compression tolerance

Variations to Try
=================

- Adjust ``compress_tolerance`` and observe the effect on latent dimension
- Try different ``compress_decompress_size_ratio`` for IoT scenarios
- Use clustering to analyze discovered patterns
- Compare reconstruction quality across different samples