Skip to content

Commit

Permalink
fix a typo in multihead-attention.md
Browse files Browse the repository at this point in the history
In `MultiHeadAttention.forward()`, the sizes of the 3rd dim of tensors `queries`, `keys` and `values` should be `query_size`, `key_size` and `value_size` respectively, instead of `num_hiddens` as written in the comments. This commit fixed this typo.
  • Loading branch information
Freeman449s authored Oct 28, 2023
1 parent e6b18cc commit da3d3b7
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions chapter_attention-mechanisms/multihead-attention.md
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,7 @@ class MultiHeadAttention(nn.Block):
def forward(self, queries, keys, values, valid_lens):
# queries,keys,values的形状:
# (batch_size,查询或者“键-值”对的个数,num_hiddens)
# (batch_size,查询或者“键-值”对的个数,query_size或key_size或value_size)
# valid_lens 的形状:
# (batch_size,)或(batch_size,查询的个数)
# 经过变换后,输出的queries,keys,values 的形状:
Expand Down Expand Up @@ -437,4 +437,4 @@ attention(X, Y, Y, valid_lens, training=False).shape

:begin_tab:`paddle`
[Discussions](https://discuss.d2l.ai/t/11843)
:end_tab:
:end_tab:

0 comments on commit da3d3b7

Please sign in to comment.