Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support microscaling (mx) type #14293

Closed
jinevening opened this issue Nov 4, 2024 · 2 comments
Closed

Support microscaling (mx) type #14293

jinevening opened this issue Nov 4, 2024 · 2 comments
Assignees

Comments

@jinevening
Copy link
Contributor

What

Let's introduce mx (microscaling) type to circle.

Since we don't have any backend that uses mx type yet, let's focus on adding new types to circle schema rather than implementing mx type kernels.

This issue will be closed after the following goals are achieved.

  1. Mx type circle can be created from recipe
  2. MX type circle can be imported as luci graph
  3. luci graph with mx type can be exported as circle
  4. MX type circle can be visualized by circledump.

Deadline: 11/30

@jinevening
Copy link
Contributor Author

jinevening commented Nov 4, 2024

#14294 addresses the issues to achieve the goals in the main comment, but there are some remaining issues.

  1. MX type axis
  • Even if tensors have the same MX type, their axes can be different (the axis would be determined by consumer operator). For example, axes of BMM Op's inputs are reduction dimension. So, for A @ B, A's axis would be -1, and B's axis would be -2.
  • We may need a way to describe the axis, e.g., MXQuantizationParameter.
  1. luci-interpreter
  • If we'd like to do value check in luci-interpreter, we may need to extend luci-interpreter::Tensor class (and memory manager that is in charge of alloc/dealloc of Tensor) because it does not have data structure to save shared exponent.
  1. Circle type inference
  • Circle's type inference rule for BMM enforces input dtype == output dtype. This constraint may become a problem when we should support mixed-precision operators.

@jinevening
Copy link
Contributor Author

The goals described in the main comment are all achieved. Let's close this issue.

Further issues listed in #14293 (comment) (and there may be more) will be addressed when necessary.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant