Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Layer] Add Self-Attention, RMSNorm benchmark #140

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

SamanthaWangdl
Copy link
Contributor

Add

  • Self-Attention
  • RMSNorm


bool areArraysEqual(float array1[], float array2[], int size) {
for (int i = 0; i < size; ++i) {
if (array1[i] != array2[i]) {
Copy link
Contributor

@ShiHaoGao ShiHaoGao Sep 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The result of this benchmark is "Pass," indicating that the calculation result is correct. However, directly using the equality operator (==) to compare floating-point numbers may not be the best practice. This is because floating-point calculations can introduce tiny precision errors.

To more reliably compare floating-point numbers, it's advised to use a small error margin, for instance: MobileNetV3 example


bool areArraysEqual(float array1[], float array2[], int size) {
for (int i = 0; i < size; ++i) {
if (array1[i] != array2[i]) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This place is the same as above.

@SamanthaWangdl
Copy link
Contributor Author

@ShiHaoGao The corresponding issue has been fixed.

@@ -0,0 +1,149 @@
//===- GoogleBenchmarkMain.cpp---------------------------------------------===//
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Recommand to centralize this title and keep the number of characters per line to 80.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

@@ -0,0 +1,147 @@
//===- GoogleBenchmarkMain.cpp---------------------------------------------===//
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Recommand to centralize this title and keep the number of characters per line to 80.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this version of the code standard, the title remains not centered, otherwise it will be too troublesome to check each file and centralize the tiles. Please help me change it back. Thanks~

@SamanthaWangdl
Copy link
Contributor Author

All errors have been fixed. Ready to merge.

@xlinsist
Copy link
Collaborator

Thanks! Just resolve the conflicts and make changes according to the code standards.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants