Missing out variants: {'torchao::dequantize_affine'}

[vetsasai]

My optimum convertion looks like...

model_id = "meta-llama/Llama-3.2-1B-Instruct"
model = ExecuTorchModelForCausalLM.from_pretrained(
    model_id,
    export=True,
    recipe="xnnpack",
    attn_implementation="custom_sdpa",
    use_custom_kv_cache=True,
    **{"qlinear": "8w", "qlinear_encoder": "4w", "qembedding": "4w"},
)
generated_text = model.text_generation(
    tokenizer=tokenizer,
    prompt=prompt,
    max_seq_len=512,
)

This error comes for Llama3.2-1B-Instruct, Llama3.2-3B-Instruct when trying with --qlinear is set to '4w' or '8w'. But it works fine for '8da4w' irrespective of --qlinear_encoder, --qembedding parameters.

Error:

Exception has occurred: RuntimeError       (note: full exception trace is shown but execution is paused at: _run_module_as_main)
Missing out variants: {'torchao::dequantize_affine'}
  File "/home/user/Desktop/my_code/all_envs/optimum_env/lib/python3.10/site-packages/executorch/exir/passes/__init__.py", line 435, in call
    raise RuntimeError(f"Missing out variants: {missing_out_vars}")
  File "/home/user/Desktop/my_code/all_envs/optimum_env/lib/python3.10/site-packages/torch/fx/passes/infra/pass_base.py", line 46, in __call__
    res = self.call(graph_module)
  File "/home/user/Desktop/my_code/all_envs/optimum_env/lib/python3.10/site-packages/executorch/exir/program/_program.py", line 1716, in to_executorch
    new_gm_res = p(new_gm)
  File "/home/user/Desktop/my_code/all_envs/optimum_env/lib/python3.10/site-packages/executorch/exir/program/_program.py", line 114, in wrapper
    return func(*args, **kwargs)
  File "/home/user/Desktop/my_code/all_envs/optimum_env/lib/python3.10/site-packages/optimum/exporters/executorch/recipes/xnnpack.py", line 93, in _lower_to_executorch
    et_prog = et_prog.to_executorch(
  File "/home/user/Desktop/my_code/all_envs/optimum_env/lib/python3.10/site-packages/optimum/exporters/executorch/recipes/xnnpack.py", line 121, in export_to_executorch_with_xnnpack
    return _lower_to_executorch(exported_progs, model.metadata)
  File "/home/user/Desktop/my_code/all_envs/optimum_env/lib/python3.10/site-packages/optimum/exporters/executorch/convert.py", line 80, in export_to_executorch
    executorch_progs = recipe_func(model, **kwargs)
  File "/home/user/Desktop/my_code/all_envs/optimum_env/lib/python3.10/site-packages/optimum/exporters/executorch/__main__.py", line 145, in main_export
    return export_to_executorch(
  File "/home/user/Desktop/my_code/all_envs/optimum_env/lib/python3.10/site-packages/optimum/executorch/modeling.py", line 287, in _export
    executorch_progs = main_export(
  File "/home/user/Desktop/my_code/all_envs/optimum_env/lib/python3.10/site-packages/optimum/executorch/modeling.py", line 422, in from_pretrained
    models_dict, temp_dir = cls._export(
  File "/home/user/Desktop/my_code/all_envs/optimum_env/optimum_pyscript.py", line 128, in main
    model = ExecuTorchModelForCausalLM.from_pretrained(
  File "/home/user/Desktop/my_code/all_envs/optimum_env/optimum_pyscript.py", line 170, in <module>
    main()
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main (Current frame)
    return _run_code(code, main_globals, None,
RuntimeError: Missing out variants: {'torchao::dequantize_affine'}

Can the respective team support this and help us resolve the error.

[hg0428]

I'm running into the same issue.