Efficient on-device learning for resource-constrained devices

.gitignore

@@ -55,3 +55,4 @@ build-*
 .venv/
 uv.lock
 *.cbuild-pack.yml
+disasm_*

AGENTS.md

@@ -20,3 +20,11 @@
 - For each of the generated pickle file, generate missing C files with : `python processTests.py -gen . -p Patterns -d Parameters -f <pickle file> -e`
 - Switch to `cmsis_build` working directory
 - Run python script `python runall.py`
+
+## Testing instructions for dsppp autodiff
+- Do not use or install `pyocd` for autodiff tests.
+- Change working directory to the `dsppp` folder.
+- Set `AVH_FVP_PLUGINS` to an empty string before launching the FVP.
+- Prepend the CMSIS Toolbox `bin` directory and the Python directory containing `python3.dll` to `PATH`.
+- Build the generated project with `cbuild -O cprj test.csolution.yml --toolchain AC6 -c test.Release+VHT-Corstone-300`.
+- Run `FVP_Corstone_SSE-300_Ethos-U55.exe` directly with `-f fvp_configs\VHT-Corstone-300.txt -a cpu0=cprj\out\test\VHT-Corstone-300\Release\test.axf`.

Documentation/Doxygen/src/matrix.md

@@ -41,9 +41,23 @@ Once you have initialized matrixes, you can operate on them:
 Matrix<float32_t,ROWS,COLS> result = a * a + b;
 ```
 
-The operators `+` and `*` are merged into the loop. `*` is the element-wise multiply. For the vector / matrix products you should use the operator `dot`.
+The operators `+` and `*` are merged into the loop. `*` is the element-wise multiply. For eagerly evaluated vector / matrix products you should use the operator `dot`.
 
-Note that fusion of operators will not work with `dot(Matrix, Matrix`). It is only supported with vectors : `dot(Vector,Vector)` or `dot(Matrix,Vector)`.
+Note that fusion of operators will not work with `dot(Matrix, Matrix)` or
+`dot(Matrix, Vector)`, since both operations return an owning, eagerly
+evaluated result. Vector dot products return a scalar.
+
+For a matrix times vector product that must be fused with vector operations,
+use the lazy `matvec` operator:
+
+```cpp
+result += matvec(matrix, vector);
+```
+
+`matvec` returns a non-owning expression. In this example, the matrix-vector
+product is accumulated directly into `result` without allocating a temporary
+vector. The operands must remain alive until the complete expression is
+evaluated. Use `dot(matrix, vector)` when an owning result is required.
 
 ## VectorView
 
@@ -136,6 +150,18 @@ The compiler may use the move semantic to copy the temporary result of the `dot`
 
 In this case, no copy would occur and `result` after the assignment would be a vector allocated by `dot` so using the `TMP_ALLOC` .
 
+### matvec
+
+```cpp
+result = bias + matvec(matrix, vector);
+```
+
+Unlike `dot(matrix, vector)`, `matvec(matrix, vector)` is lazy. It participates
+in the normal one-dimensional fusion loop, so the matrix-vector result can be
+combined with additions, multiplications, or accumulation without first being
+stored in a temporary vector. Matrix-matrix products remain eager and continue
+to use `dot`.
+
 ### diagonal
 
 ```cpp

dsppp/.clangd

@@ -0,0 +1,18 @@
+CompileFlags:
+  CompilationDatabase: c:\Users\chrfav01\benchresults\cmsis\CMSIS-DSP\dsppp\out\example\VHT-Corstone-300\Release
+
+---
+If:
+  PathMatch: .*\.(c|C|h)$
+CompileFlags:
+  Add:
+    - -include
+    - c:\Users\chrfav01\benchresults\cmsis\CMSIS-DSP\dsppp\out\example\VHT-Corstone-300\Release\compile_macros_c.h
+
+---
+If:
+  PathMatch: .*\.(cpp|c\+\+|C\+\+|cxx|cc|CC|hpp)$
+CompileFlags:
+  Add:
+    - -include
+    - c:\Users\chrfav01\benchresults\cmsis\CMSIS-DSP\dsppp\out\example\VHT-Corstone-300\Release\compile_macros_cxx.h

dsppp/Examples/LICENSE-Iris.txt

@@ -0,0 +1,16 @@
+Iris dataset license and attribution
+====================================
+
+The data in iris_data.hpp is derived from the Iris dataset distributed by the
+UCI Machine Learning Repository. The dataset is licensed under the Creative
+Commons Attribution 4.0 International license (CC BY 4.0):
+
+https://creativecommons.org/licenses/by/4.0/legalcode
+
+Attribution:
+
+Fisher, R. A. (1936). Iris [Dataset]. UCI Machine Learning Repository.
+https://doi.org/10.24432/C56C76
+
+The CMSIS-DSP source code around the data remains covered by the CMSIS-DSP
+project license.

dsppp/Examples/README.md

@@ -0,0 +1,58 @@
+# CMSIS-DSP C++ examples
+
+The examples demonstrate the CMSIS-DSP C++ API and its autodiff extension.
+Each source file defines its own `main`, so enable only one example at a time
+in `dsppp/example.cproject.yml` by commenting and uncommenting its `file` line.
+
+## Expression examples
+
+- `dot_product.cpp` evaluates a fused vector expression ending in a dot
+  product.
+- `vector_op.cpp` demonstrates floating-point and fixed-point vector
+  expressions, vector views, and strided views.
+- `matrix_op.cpp` demonstrates matrix expressions and row and column views.
+
+## Autodiff examples
+
+- `autodiff_example.cpp` is a minimal fully connected layer followed by ReLU.
+  It runs one forward and backward pass and prints the resulting gradients.
+- `autodiff_regression.cpp` trains a cubic polynomial to approximate a sine
+  wave. It demonstrates RMSProp, reusable graphs, parameter freezing, and
+  saving model parameters.
+- `autodiff_lms.cpp` identifies an unknown FIR filter with a per-sample LMS
+  update expressed as quadratic error, reverse differentiation, and SGD. It is
+  an educational demonstration; the specialized CMSIS-DSP LMS functions are
+  more efficient for production filtering.
+- `autodiff_iris.cpp` trains a small classifier on the Iris flower dataset.
+  It uses Adam and reserves 30 of the 150 samples for a final test that is not
+  used during training.
+- `autodiff_fully_connected_qat.cpp` demonstrates quantization-aware training
+  of a fully connected layer for later deployment with CMSIS-NN or Ethos-U.
+
+### Iris classifier
+
+The Iris classifier contains two fully connected layers. Cross entropy is the
+training loss and is not part of the network used for inference.
+
+```mermaid
+flowchart LR
+    input["Input<br/>4 flower measurements"] --> fc1["Fully connected<br/>4 to 8"]
+    fc1 --> relu["ReLU<br/>8 values"]
+    relu --> fc2["Fully connected<br/>8 to 3"]
+    fc2 --> softmax["Softmax<br/>3 class probabilities"]
+    softmax --> loss["Cross entropy loss"]
+    target["One-hot target"] --> loss
+```
+
+The dataset is stored as floating-point measurements in `iris_data.hpp` and
+normalized when each sample is loaded. Its license and attribution are in
+`LICENSE-Iris.txt`.
+
+## Building an example
+
+After selecting one source in `example.cproject.yml`, the Cortex-M55 virtual
+target can be built with:
+
+```text
+cbuild test.csolution.yml --context example.Release+VHT-Corstone-300
+```

dsppp/Examples/autodiff_example.cpp

@@ -0,0 +1,62 @@
+#include <dsppp/autodiff/reverse.hpp>
+#include <dsppp/autodiff/operators/fully_connected.hpp>
+#include <dsppp/autodiff/operators/relu.hpp>
+
+#include <cstdio>
+
+using namespace arm_cmsis_dsp::autodiff;
+
+int main()
+{
+    Arena<2048> arena;
+    Tape<float> &tape = arena.tape();
+    tape.register_operator<FullyConnectedOperator<float>>();
+    tape.register_operator<ReluOperator<float>>();
+
+    float x_value[] = {2.0F, -1.0F};
+    float matrix_value[2][2] = {{1.0F, 2.0F}, {-3.0F, 1.0F}};
+    float bias_value[] = {1.0F, 0.0F};
+    float linear_value[2] = {};
+    float activation_value[2] = {};
+
+    BufferView x = tape.input(x_value);
+    MatrixView matrix = tape.parameter(matrix_value);
+    BufferView bias = tape.parameter(bias_value);
+    BufferView linear = tape.output(linear_value);
+    BufferView activation = tape.output(activation_value);
+
+    if (!tape.good())
+    {
+        std::printf("Autodiff setup failed (status=%u)\n",
+                    static_cast<unsigned>(tape.status()));
+        return 1;
+    }
+
+    linear = fully_connected(x, matrix, bias);
+    activation = relu(linear);
+
+    // Seed both ReLU outputs. The second neuron is negative, so ReLU blocks
+    // its gradient during the backward pass.
+    const float seed[] = {1.0F, 1.0F};
+    if (!tape.backward(activation, seed, 2))
+    {
+        return 1;
+    }
+
+    std::printf("linear = {%g, %g}\n", static_cast<double>(linear[0]),
+                static_cast<double>(linear[1]));
+    std::printf("relu = {%g, %g}\n", static_cast<double>(activation[0]),
+                static_cast<double>(activation[1]));
+    std::printf("db = {%g, %g}\n", static_cast<double>(bias.gradient(0)),
+                static_cast<double>(bias.gradient(1)));
+    for (std::size_t row = 0; row < matrix.rows(); ++row)
+    {
+        for (std::size_t column = 0; column < matrix.columns(); ++column)
+        {
+            std::printf("dm[%u][%u] = %g\n", static_cast<unsigned>(row),
+                        static_cast<unsigned>(column),
+                        static_cast<double>(matrix.gradient(row, column)));
+        }
+    }
+    return 0;
+}