Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[eager_fail_to_run ] mobilenet_v2_quantized_qat, eager run failed on dtype mismatch #2567

Open
IvanKobzarev opened this issue Jan 16, 2025 · 0 comments

Comments

@IvanKobzarev
Copy link

Running from pytorch:

python benchmarks/dynamo/torchbench.py  --only mobilenet_v2_quantized_qat --accuracy --no-translation-validation --training --amp --backend inductor --device cuda --output out.csv

Error:

cuda train mobilenet_v2_quantized_qat
Traceback (most recent call last):
  File "/data/users/ivankobzarev/a/pytorch/benchmarks/dynamo/common.py", line 2744, in validate_model
    self.model_iter_fn(model, example_inputs)
  File "/data/users/ivankobzarev/a/pytorch/benchmarks/dynamo/torchbench.py", line 453, in forward_and_backward_pass
    pred = mod(*cloned_inputs)
  File "/data/users/ivankobzarev/a/pytorch/torch/fx/graph_module.py", line 824, in call_wrapped
    return self._wrapped_call(self, *args, **kwargs)
  File "/data/users/ivankobzarev/a/pytorch/torch/fx/graph_module.py", line 400, in __call__
    raise e
  File "/data/users/ivankobzarev/a/pytorch/torch/fx/graph_module.py", line 387, in __call__
    return super(self.cls, obj).__call__(*args, **kwargs)  # type: ignore[misc]
  File "/data/users/ivankobzarev/a/pytorch/torch/nn/modules/module.py", line 1749, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/data/users/ivankobzarev/a/pytorch/torch/nn/modules/module.py", line 1760, in _call_impl
    return forward_call(*args, **kwargs)
  File "<eval_with_key>.3", line 207, in forward
    activation_post_process_101 = self.activation_post_process_101(classifier_1);  classifier_1 = None
  File "/data/users/ivankobzarev/a/pytorch/torch/nn/modules/module.py", line 1749, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/data/users/ivankobzarev/a/pytorch/torch/nn/modules/module.py", line 1760, in _call_impl
    return forward_call(*args, **kwargs)
  File "/data/users/ivankobzarev/a/pytorch/torch/ao/quantization/fake_quantize.py", line 408, in forward
    return torch.fused_moving_avg_obs_fake_quant(
RuntimeError: expected scalar type Float but found Half

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/data/users/ivankobzarev/a/pytorch/benchmarks/dynamo/common.py", line 4857, in run
    ) = runner.load_model(
  File "/data/users/ivankobzarev/a/pytorch/benchmarks/dynamo/torchbench.py", line 372, in load_model
    self.validate_model(model, example_inputs)
  File "/data/users/ivankobzarev/a/pytorch/benchmarks/dynamo/common.py", line 2746, in validate_model
    raise RuntimeError("Eager run failed") from e
RuntimeError: Eager run failed
@IvanKobzarev IvanKobzarev changed the title [mobilenet_v2_quantized_qat] Eager run failed on dtype mismatch [eager_fail_to_run ] mobilenet_v2_quantized_qat, eager run failed on dtype mismatch Jan 17, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant