You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
using optimum exporter to export qwen2-0.5b to onnx model, I find onnx model attention_mask with shape [batch_size, past_sequence_length + 1]. And in validation, onnx inputs/outputs with shape:
onnx_inputs["input_ids"].shape = (2, 16)
onnx_inputs["attention_mask"].shape = (2, 32)
onnx_inputs["position_ids"].shape = (2, 16)
onnx_inputs["past_key_values.0.key"].shape = (2, 2, 16, 64)
AACengineer
changed the title
qwen2 onnx model attention_mask shape is wrong
qwen2 onnx model attention_mask && output_past_kv shape is wrong
Aug 19, 2024
Who can help?
using optimum exporter to export qwen2-0.5b to onnx model, I find onnx model attention_mask with shape [batch_size, past_sequence_length + 1]. And in validation, onnx inputs/outputs with shape:
onnx_inputs["input_ids"].shape = (2, 16)
onnx_inputs["attention_mask"].shape = (2, 32)
onnx_inputs["position_ids"].shape = (2, 16)
onnx_inputs["past_key_values.0.key"].shape = (2, 2, 16, 64)
onnx_output["logits"].shape=(2, 16, 151936)
onnx_output["present.0.key"].shape=(2, 2, 32, 64)
attention_mask shape should be [batch_size, sequence_length + past_sequence_length]? output_past_kv shape be [batch_size, kv_heads, sequence_length + past_sequence_length, head_dims] ? @michaelbenayoun
Information
Tasks
examples
folder (such as GLUE/SQuAD, ...)Reproduction (minimal, reproducible, runnable)
optimum-cli export onnx --model qwen/qwen2-0___5b-instruct/ ./onnx --task text-generation-with-past
Expected behavior
Confirm whether there are any errors in shape
The text was updated successfully, but these errors were encountered: