Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

don't fail if profiler already initialized #2

Closed
michael-kenzel opened this issue Feb 21, 2024 · 2 comments
Closed

don't fail if profiler already initialized #2

michael-kenzel opened this issue Feb 21, 2024 · 2 comments

Comments

@michael-kenzel
Copy link
Contributor

Instrumentation calls cuptiProfilerInitialize to enable higher precision GPU timestamps. However, we currently fail if the profiler was already initialized, which prevents running with nvprof. We should fix this by not failing if cuptiProfilerInitialize fails due to the profiler already having been initialized.

@richardmembarth
Copy link
Member

This fails also on my machine without running nvprof.
Just running any CUDA binary fails with error code 15: CUPTI_ERROR_NOT_INITIALIZED

@richardmembarth
Copy link
Member

Maybe, we should check for this error code and continue without CUPTI.

CUPTI calls will fail with error code CUPTI_ERROR_NOT_INITIALIZED if the CUDA driver version is not compatible with the CUPTI version.

https://docs.nvidia.com/cupti/r_main.html

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants