Skip to content

Commit

Permalink
Merge pull request #541 from gizatechxyz/develop
Browse files Browse the repository at this point in the history
Merge develop into main
  • Loading branch information
raphaelDkhn authored Jan 12, 2024
2 parents 345d3bc + 4d34849 commit 3cfa610
Show file tree
Hide file tree
Showing 606 changed files with 28,628 additions and 1,222 deletions.
9 changes: 9 additions & 0 deletions .all-contributorsrc
Original file line number Diff line number Diff line change
Expand Up @@ -260,6 +260,15 @@
"contributions": [
"code"
]
},
{
"login": "Vid201",
"name": "Vid Kersic",
"avatar_url": "https://avatars.githubusercontent.com/u/38610409?v=4",
"profile": "https://github.com/Vid201",
"contributions": [
"code"
]
}
],
"contributorsPerLine": 7,
Expand Down
1 change: 1 addition & 0 deletions .tool-versions
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
scarb 2.4.0
5 changes: 4 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@

# Orion: An Open-source Framework for Validity and ZK ML ✨
<!-- ALL-CONTRIBUTORS-BADGE:START - Do not remove or modify this section -->
[![All Contributors](https://img.shields.io/badge/all_contributors-28-orange.svg?style=flat-square)](#contributors-)
[![All Contributors](https://img.shields.io/badge/all_contributors-29-orange.svg?style=flat-square)](#contributors-)
<!-- ALL-CONTRIBUTORS-BADGE:END -->

Orion is an open-source, community-driven framework dedicated to Provable Machine Learning. It provides essential components and a new ONNX runtime for building verifiable Machine Learning models using [STARKs](https://starkware.co/stark/).
Expand Down Expand Up @@ -104,6 +104,9 @@ Thanks goes to these wonderful people:
<td align="center" valign="top" width="14.28%"><a href="https://www.bilginkocak.com/"><img src="https://avatars.githubusercontent.com/u/30844607?v=4?s=100" width="100px;" alt="Bilgin Koçak"/><br /><sub><b>Bilgin Koçak</b></sub></a><br /><a href="https://github.com/gizatechxyz/orion/commits?author=bilgin-kocak" title="Code">💻</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://t.me/notaihe"><img src="https://avatars.githubusercontent.com/u/22559023?v=4?s=100" width="100px;" alt="akhercha"/><br /><sub><b>akhercha</b></sub></a><br /><a href="https://github.com/gizatechxyz/orion/commits?author=akhercha" title="Code">💻</a></td>
</tr>
<tr>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/Vid201"><img src="https://avatars.githubusercontent.com/u/38610409?v=4?s=100" width="100px;" alt="Vid Kersic"/><br /><sub><b>Vid Kersic</b></sub></a><br /><a href="https://github.com/gizatechxyz/orion/commits?author=Vid201" title="Code">💻</a></td>
</tr>
</tbody>
</table>

Expand Down
2 changes: 1 addition & 1 deletion Scarb.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "orion"
version = "0.1.9"
version = "0.2.0"
cairo-version = "2.4.0"
edition = "2023_10"
description = "ONNX Runtime in Cairo for verifiable ML inference using STARK"
Expand Down
24 changes: 24 additions & 0 deletions docgen/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,14 @@ fn main() {
doc_trait(trait_path, doc_path, label);
doc_functions(trait_path, doc_path, trait_name, label);

// SEQUENCE DOC
let trait_path = "src/operators/sequence/core.cairo";
let doc_path = "docs/framework/operators/sequence";
let label = "sequence";
let trait_name = "SequenceTrait";
doc_trait(trait_path, doc_path, label);
doc_functions(trait_path, doc_path, trait_name, label);

// FIXED POINT DOC
let trait_path = "src/numbers/fixed_point/core.cairo";
let doc_path = "docs/framework/numbers/fixed-point";
Expand Down Expand Up @@ -66,6 +74,22 @@ fn main() {
let trait_name: &str = "LinearRegressorTrait";
doc_trait(trait_path, doc_path, label);
doc_functions(trait_path, doc_path, trait_name, label);

// LINEAR CLASSIFIER DOC
let trait_path = "src/operators/ml/linear/linear_classifier.cairo";
let doc_path = "docs/framework/operators/machine-learning/linear-classifier";
let label = "linear_classifier";
let trait_name: &str = "LinearClassifierTrait";
doc_trait(trait_path, doc_path, label);
doc_functions(trait_path, doc_path, trait_name, label);

// SVM REGRESSOR DOC
let trait_path = "src/operators/ml/svm/svm_regressor.cairo";
let doc_path = "docs/framework/operators/machine-learning/svm-regressor";
let label = "svm_regressor";
let trait_name: &str = "SVMRegressorTrait";
doc_trait(trait_path, doc_path, label);
doc_functions(trait_path, doc_path, trait_name, label);
}

fn doc_trait(trait_path: &str, doc_path: &str, label: &str) {
Expand Down
4 changes: 4 additions & 0 deletions docs/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,11 @@ All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased] - 2023-12-25

## Added
- Compress Operator.

## [Unreleased] - 2023-12-14

## Added
Expand Down
29 changes: 22 additions & 7 deletions docs/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,7 @@
* [tensor.bitwise\_and](framework/operators/tensor/tensor.bitwise\_and.md)
* [tensor.bitwise\_xor](framework/operators/tensor/tensor.bitwise\_xor.md)
* [tensor.bitwise\_or](framework/operators/tensor/tensor.bitwise\_or.md)
* [tensor.resize](framework/operators/tensor/tensor.resize.md)
* [tensor.round](framework/operators/tensor/tensor.round.md)
* [tensor.scatter](framework/operators/tensor/tensor.scatter.md)
* [tensor.array\_feature\_extractor](framework/operators/tensor/tensor.array\_feature\_extractor.md)
Expand All @@ -110,28 +111,24 @@
* [tensor.reduce\_prod](framework/operators/tensor/tensor.reduce\_prod.md)
* [tensor.gather\_elements](framework/operators/tensor/tensor.gather\_elements.md)
* [tensor.gather\_nd](framework/operators/tensor/tensor.gather\_nd.md)
* [tensor.sequence\_length](framework/operators/tensor/tensor.sequence\_length.md)
* [tensor.sequence\_at](framework/operators/tensor/tensor.sequence\_at.md)
* [tensor.reduce\_min](framework/operators/tensor/tensor.reduce\_min.md)
* [tensor.sequence\_construct](framework/operators/tensor/tensor.sequence\_construct.md)
* [tensor.shrink](framework/operators/tensor/tensor.shrink.md)
* [tensor.sequence\_empty](framework/operators/tensor/tensor.sequence\_empty.md)
* [tensor.reduce\_mean](framework/operators/tensor/tensor.reduce\_mean.md)
* [tensor.pow](framework/operators/tensor/tensor.pow.md)
* [tensor.sequence\_erase](framework/operators/tensor/tensor.sequence\_erase.md)
* [tensor.sequence\_insert](framework/operators/tensor/tensor.sequence\_insert.md)
* [tensor.concat\_from\_sequence](framework/operators/tensor/tensor.concat\_from\_sequence.md)
* [tensor.is\_nan](framework/operators/tensor/tensor.is\_nan.md)
* [tensor.is\_inf](framework/operators/tensor/tensor.is\_inf.md)
* [tensor.not](framework/operators/tensor/tensor.not.md)
* [tensor.erf](framework/operators/tensor/tensor.erf.md)
* [tensor.reduce\_log\_sum](framework/operators/tensor/tensor.reduce\_log\_sum.md)
* [tensor.unique](framework/operators/tensor/tensor.unique.md)
* [tensor.compress](framework/operators/tensor/tensor.compress.md)
* [tensor.layer_normalization](framework/operators/tensor/tensor.layer_normalization.md)
* [Neural Network](framework/operators/neural-network/README.md)
* [nn.relu](framework/operators/neural-network/nn.relu.md)
* [nn.leaky\_relu](framework/operators/neural-network/nn.leaky\_relu.md)
* [nn.sigmoid](framework/operators/neural-network/nn.sigmoid.md)
* [nn.softmax](framework/operators/neural-network/nn.softmax.md)
* [nn.softmax_zero](framework/operators/neural-network/nn.softmax_zero.md)
* [nn.logsoftmax](framework/operators/neural-network/nn.logsoftmax.md)
* [nn.softsign](framework/operators/neural-network/nn.softsign.md)
* [nn.softplus](framework/operators/neural-network/nn.softplus.md)
Expand All @@ -142,6 +139,24 @@
* [Machine Learning](framework/operators/machine-learning/README.md)
* [Tree Ensemble Classifier](framework/operators/machine-learning/tree-ensemble-classifier/README.md)
* [tree\_ensemble\_classifier.predict](framework/operators/machine-learning/tree-ensemble-classifier/tree\_ensemble\_classifier.predict.md)
* [Tree Ensemble Regressor](framework/operators/machine-learning/tree-ensemble-regressor/README.md)
* [tree\_ensemble\_regressor.predict](framework/operators/machine-learning/tree-ensemble-regressor/tree\_ensemble\_regressor.predict.md)
* [Linear Classifier](framework/operators/machine-learning/linear-classifier/README.md)
* [linear\_classifier.predict](framework/operators/machine-learning/linear-classifier/linear\_classifier.predict.md)
* [Linear Regressor](framework/operators/machine-learning/linear-regressor/README.md)
* [linear\_regressor.predict](framework/operators/machine-learning/linear-regressor/linear\_regressor.predict.md)
* [SVM Regressor](framework/operators/machine-learning/svm-regressor/README.md)
* [svm\_regressor.predict](framework/operators/machine-learning/svm-regressor/svm\_regressor.predict.md)
* [Sequence](framework/operators/sequence/README.md)
* [sequence.sequence\_construct](framework/operators/sequence/sequence.sequence\_construct.md)
* [sequence.sequence\_empty](framework/operators/sequence/sequence.sequence\_empty.md)
* [sequence.sequence\_length](framework/operators/sequence/sequence.sequence\_length.md)
* [sequence.sequence\_at](framework/operators/sequence/sequence.sequence\_at.md)
* [sequence.sequence\_empty](framework/operators/sequence/sequence.sequence\_empty.md)
* [sequence.sequence\_erase](framework/operators/sequence/sequence.sequence\_erase.md)
* [sequence.sequence\_insert](framework/operators/sequence/sequence.sequence\_insert.md)
* [sequence.concat\_from\_sequence](framework/operators/sequence/sequence.concat\_from\_sequence.md)


## 🏛 Hub

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -205,7 +205,7 @@ X_min = np.min(X_original, axis=0)
X_max = np.max(X_original, axis=0)
X_range = X_max - X_min
df_forecast_data_normalized = (df_forecast_data - X_min) / X_range
# tranpose the matrix and add bias
# transpose the matrix and add bias
df_forecast_data_normalized_transposed= df_forecast_data_normalized.T
df_forecast_data_normalized_transposed_with_bias = np.vstack((df_forecast_data_normalized_transposed, np.ones(df_forecast_data_normalized_transposed.shape[1])))
#normalized forecasts
Expand Down Expand Up @@ -913,7 +913,7 @@ fn normalize_user_x_inputs(
shape: array![data_len].span(), data: x_range.span()
);

// for normalizing 2D user inputed feature vals
// for normalizing 2D user inputted feature vals
if x_inputs.shape.len() > 1 {
let mut j: u32 = 0;
loop {
Expand Down Expand Up @@ -1069,7 +1069,7 @@ let mut rescale_forecasts = rescale_predictions(forecast_results, main_y_vals);

Our model will get tested under the `multiple_linear_regression_test()` function which will follow these steps:

1. Data retrival: The function initiates by fetching the AAVE dataset's x and y values.
1. Data retrieval: The function initiates by fetching the AAVE dataset's x and y values.
2. Dataset construction and normalization: A new Dataset object gets initialized by passing the x and y variables. It is then normalized using the built-in `normalize_dataset()` method.
3. Model fitting: Using the `MultipleLinearRegression` function we fit the normalized dataset and compute the regression coefficients.
4. Computing accuracy of the model: To calculate the accuracy we utilize the `predict` method to compute the dot product between the model's regression coefficients and the x values. We then compute the R-squared score to measure the accuracy of our model.
Expand Down
22 changes: 13 additions & 9 deletions docs/framework/compatibility.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ You can see below the list of current supported ONNX Operators:
| [ThresholdedRelu](operators/neural-network/nn.thresholded\_relu.md) | :white\_check\_mark: |
| [Sigmoid](operators/neural-network/nn.sigmoid.md) | :white\_check\_mark: |
| [Softmax](operators/neural-network/nn.softmax.md) | :white\_check\_mark: |
| [Softmax_zero](operators/neural-network/nn.softmax_zero.md) | :white\_check\_mark: |
| [LogSoftmax](operators/neural-network/nn.logsoftmax.md) | :white\_check\_mark: |
| [Softsign](operators/neural-network/nn.softsign.md) | :white\_check\_mark: |
| [Softplus](operators/neural-network/nn.softplus.md) | :white\_check\_mark: |
Expand Down Expand Up @@ -77,6 +78,7 @@ You can see below the list of current supported ONNX Operators:
| [BitwiseAnd](operators/tensor/tensor.bitwise_and.md) | :white\_check\_mark: |
| [BitwiseOr](operators/tensor/tensor.bitwise_or.md) | :white\_check\_mark: |
| [BitwiseXor](operators/tensor/tensor.bitwise_xor.md) | :white\_check\_mark: |
| [Resize](operators/tensor/tensor.resize.md) | :white\_check\_mark: |
| [Round](operators/tensor/tensor.round.md) | :white\_check\_mark: |
| [MaxInTensor](operators/tensor/tensor.max\_in\_tensor.md) | :white\_check\_mark: |
| [Max](operators/tensor/tensor.max.md) | :white\_check\_mark: |
Expand All @@ -89,20 +91,22 @@ You can see below the list of current supported ONNX Operators:
| [ReduceL1](operators/tensor/tensor.reduce\_l1.md) | :white\_check\_mark: |
| [ReduceL2](operators/tensor/tensor.reduce\_l2.md) | :white\_check\_mark: |
| [GatherElements](operators/tensor/tensor.gather/_elements.md) | :white\_check\_mark: |
| [SequenceLength](operators/tensor/tensor.sequence\_length.md) | :white\_check\_mark: |
| [SequenceAt](operators/tensor/tensor.sequence\_at.md) | :white\_check\_mark: |
| [SequenceConstruct](operators/tensor/tensor.sequence\_construct.md) | :white\_check\_mark: |
| [SequenceLength](operators/sequence/sequence.sequence\_length.md) | :white\_check\_mark: |
| [SequenceAt](operators/sequence/sequence.sequence\_at.md) | :white\_check\_mark: |
| [SequenceConstruct](operators/sequence/sequence.sequence\_construct.md) | :white\_check\_mark: |
| [Shrink](operators/tensor/tensor.shrink.md) | :white\_check\_mark: |
| [SequenceEmpty](operators/tensor/tensor.sequence\_empty.md) | :white\_check\_mark: |
| [SequenceEmpty](operators/sequence/sequence.sequence\_empty.md) | :white\_check\_mark: |
| [ReduceL2](operators/tensor/tensor.reduce\_l2.md) | :white\_check\_mark: |
| [SequenceErase](operators/tensor/tensor.sequence\_erase.md) | :white\_check\_mark: |
| [SequenceInsert](operators/tensor/tensor.sequence\_insert.md) | :white\_check\_mark: |
| [ConcatFromSequence](operators/tensor/tensor.concat\_from\_sequence.md) | :white\_check\_mark: |
| [SequenceErase](operators/sequence/sequence.sequence\_erase.md) | :white\_check\_mark: |
| [SequenceInsert](operators/sequence/sequence.sequence\_insert.md) | :white\_check\_mark: |
| [ConcatFromSequence](operators/sequence/sequence.concat\_from\_sequence.md) | :white\_check\_mark: |
| [IsNaN](operators/tensor/tensor.is\_nan.md) | :white\_check\_mark: |
| [IsInf](operators/tensor/tensor.is\_inf.md) | :white\_check\_mark: |
| [Not](operators/tensor/tensor.not.md) | :white\_check\_mark: |
| [GatherND](operators/tensor/tensor.gather/_nd.md) | :white\_check\_mark: |
| [ReduceLogSum](operators/tensor/tensor.reduce\_log\_sum.md) | :white\_check\_mark: |
| [Erf](operators/tensor/tensor.erf.md) | :white\_check\_mark: |
| [Erf](operators/tensor/tensor.erf.md) | :white\_check\_mark: |
| [Compress](operators/tensor/tensor.compress.md) | :white\_check\_mark: |
| [Layer_normalization](operators/tensor/tensor.layer_normalization.md) | :white\_check\_mark: |

Current Operators support: **96/156 (62%)**
Current Operators support: **97/156 (62%)**
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# Linear Classifier

`LinearClassifierTrait` provides a trait definition for linear classification problem.

```rust
use orion::operators::ml::LinearClassificationTrait;
```

### Data types

Orion supports currently only fixed point data types for `LinearClassificationTrait`.

| Data type | dtype |
| -------------------- | ------------------------------------------------------------- |
| Fixed point (signed) | `LinearClassifierTrait<FP8x23 \| FP16x16 \| FP64x64 \| FP32x32>` |


***

| function | description |
| --- | --- |
| [`linear_classifier.predict`](linear_classifier.predict.md) | Performs the linear classification. |

Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
# LinearClassifierTrait::predict

```rust
fn predict(ref self: LinearClassifier<T>, X: Tensor<T>) -> Tensor<T>;
```

Linear Classifier. Performs the linear classification.

## Args

* `self`: LinearClassifier<T> - A LinearClassifier object.
* `X`: Input 2D tensor.

## Returns

* Tensor<T> containing the linear classification evaluation of the input X.

## Type Constraints

`LinearClassifier` and `X` must be fixed points

## Examples

```rust
use orion::numbers::FP16x16;
use orion::operators::tensor::{Tensor, TensorTrait, FP16x16Tensor, U32Tensor};

use orion::operators::ml::linear::linear_classifier::{
LinearClassifierTrait, POST_TRANSFORM, LinearClassifier
};

fn linear_classifier_helper(
post_transform: POST_TRANSFORM
) -> (LinearClassifier<FP16x16>, Tensor<FP16x16>) {

let classlabels: Span<usize> = array![0, 1, 2].span();
let classlabels = Option::Some(classlabels);

let classlabels_strings: Option<Span<FP16x16>> = Option::None;

let coefficients: Span<FP16x16> = array![
FP16x16 { mag: 38011, sign: true },
FP16x16 { mag: 19005, sign: true },
FP16x16 { mag: 5898, sign: true },
FP16x16 { mag: 38011, sign: false },
FP16x16 { mag: 19005, sign: false },
FP16x16 { mag: 5898, sign: false },
]
.span();

let intercepts: Span<FP16x16> = array![
FP16x16 { mag: 176947, sign: false },
FP16x16 { mag: 176947, sign: true },
FP16x16 { mag: 32768, sign: false },
]
.span();
let intercepts = Option::Some(intercepts);

let multi_class: usize = 0;

let mut classifier: LinearClassifier<FP16x16> = LinearClassifier {
classlabels,
coefficients,
intercepts,
multi_class,
post_transform
};

let mut X: Tensor<FP16x16> = TensorTrait::new(
array![3, 2].span(),
array![
FP16x16 { mag: 0, sign: false },
FP16x16 { mag: 65536, sign: false },
FP16x16 { mag: 131072, sign: false },
FP16x16 { mag: 196608, sign: false },
FP16x16 { mag: 262144, sign: false },
FP16x16 { mag: 327680, sign: false },
]
.span()
);

(classifier, X)
}

fn linear_classifier_multi_softmax() -> (Span<usize>, Tensor<FP16x16>) {
let (mut classifier, X) = linear_classifier_helper(POST_TRANSFORM::SOFTMAX);

let (labels, mut scores) = LinearClassifierTrait::predict(ref classifier, X);

(labels, scores)
}

>>>
([0, 2, 2],
[
[0.852656, 0.009192, 0.138152],
[0.318722, 0.05216, 0.629118],
[0.036323, 0.090237, 0.87344]
])
```
23 changes: 23 additions & 0 deletions docs/framework/operators/machine-learning/svm-regressor/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# SVM Regressor

`SVMRegressorTrait` provides a trait definition for svm regression problem.

```rust
use orion::operators::ml::SVMRegressorTrait;
```

### Data types

Orion supports currently only fixed point data types for `SVMRegressorTrait`.

| Data type | dtype |
| -------------------- | ------------------------------------------------------------- |
| Fixed point (signed) | `SVMRegressorTrait<FP8x23 \| FP16x16 \| FP64x64 \| FP32x32>` |


***

| function | description |
| --- | --- |
| [`svm_regressor.predict`](svm_regressor.predict.md) | Returns the regressed values for each input in N. |

Loading

0 comments on commit 3cfa610

Please sign in to comment.