[DRAFT] Support CircleGRU #12319

BalyshevArtem · 2023-12-18T12:39:52Z

This draft supports CircleGRU in compiler.

for issue: #12320
related issue: #12263

ONE-DCO-1.0-Signed-off-by: Artem Balyshev [email protected]

seanshpark · 2024-01-15T21:55:37Z

compiler/circlechef/circle/src/CircleOpRegistry.h

@@ -58,6 +58,7 @@ class CircleOpRegistry
    REG_TFL_OP(BATCH_MATMUL, CircleOpBatchMatMul);
    REG_TFL_OP(BCQ_FULLY_CONNECTED, CircleOpBCQFullyConnected);
    REG_TFL_OP(BCQ_GATHER, CircleOpBCQGather);
+    REG_TFL_OP(CIR_GRU, CircleOpCircleGRU);


Suggested change

REG_TFL_OP(CIR_GRU, CircleOpCircleGRU);

REG_TFL_OP(GRU, CircleOpGRU);

or,
1/ to prepare tflite introduces GRU,
2/ two Circle looks odd

Suggested change

REG_TFL_OP(CIR_GRU, CircleOpCircleGRU);

REG_TFL_OP(CIR_GRU, CircleOpCirGRU);

I think second variant better to avoid future problems with tflite GRU operation

seanshpark · 2024-01-15T22:00:37Z

compiler/circlechef/proto/circlechef.proto

@@ -64,6 +64,7 @@ enum Activation {
  NONE = 0;
  RELU = 1;
  RELU6 = 3;
+  TANH = 4;


Q) is this necessary for our model?

Remove it, thank you

seanshpark · 2024-01-15T22:02:22Z

compiler/luci/export/src/CircleBuiltinTypesExtractor.h

@@ -541,6 +541,13 @@ class BuiltinOptionsExtractor final
                                             to_circle_actfunc(node->fusedActivationFunction()))
      .Union();
  }
+  flatbuffers::Offset<void> visit(luci::CircleGRU *node)


CircleGRU -> need to sync name with chef.
I don't think there is one exact answer so we may need to discuss.

seanshpark · 2024-01-15T22:25:36Z

compiler/luci/pass/src/FuseGRUPass.cpp

+    if (num_fc != 2 or num_mul != 3 or num_logistic != 2 or num_split != 2 or num_add != 6 or
+        num_gather != 1 or num_reshape != 1 or num_sub != 1 or num_tanh != 1 or num_split_out != 6)
+      return false;


I don't think counting elements can provide correct condition.
And tracking the graph for exact Op type looks laborious job.
But we may have to do with some good graph traversal codes.

Okay, I will change it further )

seanshpark · 2024-01-15T22:44:58Z

May need res/TensorFlowLiteRecipes/Net_DecomposedGru_000 for testings

circle2circle-dredd-recipe-test
luci-pass-value-py-test (need luci-interpreter)

You may know that you can create Net_DecomposedGru_000 from tflite file with tflchef-reverse tool.

seanshpark · 2024-01-15T22:45:58Z

Q) do you not have a plan to implement luci-interpreter ?

seanshpark · 2024-01-16T01:04:42Z

compiler/luci/pass/src/FuseGRUPass.cpp

+    cloned->rank(node->rank());
+
+    // values
+    switch (node->dtype())


Q) do we need all these implemented types?

seanshpark · 2024-01-16T01:06:34Z

compiler/luci/pass/src/FuseGRUPass.cpp

+    if (not while_out_node)
+      continue;
+
+    if (fuse_gru(while_out_node))


Q) are other sub graphs like the body graph removed?
I don't recall we had some method to remove graph in the Module.

Q) are other sub graphs like the body graph removed?

Actually - no. We need to support some methods to remove subgraphs in the Module

What do you think about add a new option that will leave only the main sub-graph, and delete others? Smth like --use_only_main_graph and use fuse_gru and use_only_main_graph together. Otherwise, we need to transfer/save the luci::Module entity to CircleOptimizer to remove unnecessary graphs

Otherwise, we need to transfer/save the luci::Module entity to CircleOptimizer to remove unnecessary graphs

Maybe some kind of eliminate dead sub-graph...
1/ As of now, all sub-graphs having ID > 0 are used by some other sub-graph (usually ID == 0).
(I'm not sure this will hold in the future)
2/ We can use void CircleOptimizer::optimize(luci::Module *m) const
3/ Add something like EliminateDeadSubgraphPass (maybe with --elminate-dead-graph option or run this for default without any options)
4/ Traverse main graph and mark sub-graphs that are used
5/ Traverse used sub-graphs and mark other used sub-graphs too
6/ Delete sub-graphs that are not marked

Can this be feasible ?

Can this be feasible ?

I think yes, good idea to add EliminateDeadSubgraphPass and run this for default

BalyshevArtem · 2024-01-16T08:55:52Z

Q) do you not have a plan to implement luci-interpreter ?

I'm not sure if this operation is needed in luci-interpreter, so now I don't have this in plan. But if you think it's better to add it, I'll do it)

seanshpark · 2024-01-16T09:33:25Z

But if you think it's better to add it, I'll do it)

Current purpose I have is to validate fusion implementation through value test (as mathematical equivalence).
If adding the kernel takes heavy work, we may fire a task issue and postpone to later time.
I'll let you decide by your situation.

BalyshevArtem · 2024-01-16T10:27:30Z

You may know that you can create Net_DecomposedGru_000 from tflite file with tflchef-reverse tool.

I got assertion tflchef-reverse: /ONE/compiler/tflchef/tflite/src/RecipeChef.cpp:77: std::unique_ptrtflchef::ModelRecipe tflchef::generate_recipe(const tflite::Model*): Assertion `tflite_import.num_subgraph() == 1' failed.

seanshpark · 2024-01-17T21:32:41Z

`tflite_import.num_subgraph() == 1' failed.

current tflchef-reverse seems to supports only single subgraph model.
you can expand this to support multiple subgraphs or post an issue to request this feature.

This draft supports CircleGRU in compiler. ONE-DCO-1.0-Signed-off-by: Artem Balyshev <[email protected]>

…ypes in FuseCirGruPass.

BalyshevArtem · 2024-02-01T08:07:44Z

May need res/TensorFlowLiteRecipes/Net_DecomposedGru_000 for testings

Done, added it

BalyshevArtem · 2024-02-02T08:35:41Z

@seanshpark, can I split this draft into PRs? For luci/pass/src/FuseGRUPass pass I think better create another draft to solve this problem #12319 (comment) and this #12319 (comment) and add luci-interpreter part to check values.

seanshpark · 2024-02-02T09:34:48Z

can I split this draft into PRs?

Yes, let's go :)

I didn't look deeper into the draft codes... I check while review with PRs.

BalyshevArtem added the DRAFT A draft issue or PR for sharing one's current working status and discussion. label Dec 18, 2023

BalyshevArtem marked this pull request as draft December 18, 2023 12:39

BalyshevArtem force-pushed the draft_compiler_support_circle_gru branch from 29060f0 to 6ee3e3e Compare January 15, 2024 15:25

BalyshevArtem requested a review from seanshpark January 15, 2024 15:25

seanshpark reviewed Jan 15, 2024

View reviewed changes

seanshpark reviewed Jan 16, 2024

View reviewed changes

This was referenced Jan 22, 2024

[tflchef] Support multiple subgraphs #12506

Closed

[DRAFT][tflchef] Support multiple subgraphs #12516

Closed

BalyshevArtem mentioned this pull request Jan 31, 2024

[tflchef] Add filler for Less op #12581

Merged

Artem Balyshev added 5 commits February 1, 2024 10:45

[DRAFT] Support CircleGRU

836c44f

This draft supports CircleGRU in compiler. ONE-DCO-1.0-Signed-off-by: Artem Balyshev <[email protected]>

add circlechef and circledump

0ed0063

fix some errors and add circle2circle part

e518676

fix review comments: rename CircleGRU -> CirGru. Remove unnecessary t…

5dc376a

…ypes in FuseCirGruPass.

add Net_DecomposedGru_000 for testing

aac9083

BalyshevArtem force-pushed the draft_compiler_support_circle_gru branch from 4ed032d to aac9083 Compare February 1, 2024 08:07

BalyshevArtem requested a review from seanshpark February 1, 2024 08:08

BalyshevArtem mentioned this pull request Feb 2, 2024

[res] Add GRU in CircleSchema #12597

Merged

BalyshevArtem mentioned this pull request Feb 2, 2024

[luci/lang] Add CircleGRU node #12598

Merged

BalyshevArtem mentioned this pull request Aug 7, 2024

[DRAFT][compiler] Introduce FuseGRUPass #13602

Open

BalyshevArtem closed this Aug 22, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[DRAFT] Support CircleGRU #12319

[DRAFT] Support CircleGRU #12319

BalyshevArtem commented Dec 18, 2023 •

edited

Loading

seanshpark Jan 15, 2024

seanshpark Jan 15, 2024

BalyshevArtem Jan 16, 2024

BalyshevArtem Jan 16, 2024

seanshpark Jan 15, 2024

BalyshevArtem Jan 16, 2024

seanshpark Jan 15, 2024

seanshpark Jan 15, 2024

BalyshevArtem Jan 16, 2024 •

edited

Loading

seanshpark commented Jan 15, 2024 •

edited

Loading

seanshpark commented Jan 15, 2024

seanshpark Jan 16, 2024

seanshpark Jan 16, 2024

BalyshevArtem Jan 16, 2024

BalyshevArtem Jan 16, 2024

seanshpark Feb 2, 2024 •

edited

Loading

BalyshevArtem Feb 2, 2024

BalyshevArtem commented Jan 16, 2024

seanshpark commented Jan 16, 2024

BalyshevArtem commented Jan 16, 2024

seanshpark commented Jan 17, 2024

BalyshevArtem commented Feb 1, 2024

BalyshevArtem commented Feb 2, 2024 •

edited

Loading

seanshpark commented Feb 2, 2024 •

edited

Loading

	REG_TFL_OP(CIR_GRU, CircleOpCircleGRU);
	REG_TFL_OP(GRU, CircleOpGRU);

	REG_TFL_OP(CIR_GRU, CircleOpCircleGRU);
	REG_TFL_OP(CIR_GRU, CircleOpCirGRU);

[DRAFT] Support CircleGRU #12319

[DRAFT] Support CircleGRU #12319

Conversation

BalyshevArtem commented Dec 18, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

BalyshevArtem Jan 16, 2024 • edited Loading

Choose a reason for hiding this comment

seanshpark commented Jan 15, 2024 • edited Loading

seanshpark commented Jan 15, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

seanshpark Feb 2, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

BalyshevArtem commented Jan 16, 2024

seanshpark commented Jan 16, 2024

BalyshevArtem commented Jan 16, 2024

seanshpark commented Jan 17, 2024

BalyshevArtem commented Feb 1, 2024

BalyshevArtem commented Feb 2, 2024 • edited Loading

seanshpark commented Feb 2, 2024 • edited Loading

BalyshevArtem commented Dec 18, 2023 •

edited

Loading

BalyshevArtem Jan 16, 2024 •

edited

Loading

seanshpark commented Jan 15, 2024 •

edited

Loading

seanshpark Feb 2, 2024 •

edited

Loading

BalyshevArtem commented Feb 2, 2024 •

edited

Loading

seanshpark commented Feb 2, 2024 •

edited

Loading