Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

python3Packages.dlib: Inherit stdenv and build flags from main derivation, fix tests #273665

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

hacker1024
Copy link
Member

@hacker1024 hacker1024 commented Dec 11, 2023

Description of changes

This PR updates dlib's Python derivation to use the same build flags as the C library derivation it's based on. Notably, this fixes CUDA support and removes duplication of optional feature logic.

Tests are also fixed in this PR by switching to pytestCheckHook. At the moment, builds are failing on master.

Closes #243880.

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

Comment on lines 22 to 30
# Pass CMake flags through to the build script
preConfigure = ''
setupPyBuildFlags="$setupPyBuildFlags $(sed 's/-D\(\S*\)/--set \1/g' <<< "$cmakeFlags")"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm ok with this, I guess. But to add more options:

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The sed command also filters for -D prefixes, in case there are other items in cmakeFlags. I'll keep it as is for simplicity.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not okay with this. Please use something simpler and easier.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done - now using a for loop.

);

checkPhase = ''
${python.interpreter} nix_run_setup test
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any advantage using "${python.interpreter}" over "python"?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure; it was like this already.
I'm pretty sure we should be using pytestCheckHook anyway.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When rebasing please be aware that this way of running python tests will be removed in the current staging run and pytestCheckHook should be used

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

];
doCheck = !(
# The tests attempt to use CUDA on the build platform.
dlib.cudaSupport
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a way to disable just a subset of tests?

Copy link
Member Author

@hacker1024 hacker1024 Dec 12, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably, but as each build takes 10-15 minutes on my laptop I don't have the time to determine which fail

@hacker1024 hacker1024 mentioned this pull request Jan 15, 2024
13 tasks
@SomeoneSerge
Copy link
Contributor

SomeoneSerge commented Jan 16, 2024

I fell out of this for a bit... what's the state of this PR? Should look into merging this or into closing this?

EDIT: In retrospective, it's ok to address thecudatoolkit comments later. Addressing the obviously incorrect behaviour is the priority

@hacker1024
Copy link
Member Author

hacker1024 commented Jan 16, 2024

I fell out of this for a bit... what's the state of this PR? Should look into merging this or into closing this?

Sorry for the delay - I've been meaning to make changes based on your feedback, but haven't found the time. If #279927 adds the Python support and non-cudatoolkit-ness implemented here I'm happy to close it. Otherwise I can hopefully finish it sometime this week.

My main concern re cudatookit is Jetson support - I can confirm this PR as it is works on a Jetson Orin NX.

@hacker1024
Copy link
Member Author

I have updated this branch to work with recent changes in #279927. It is now only changing the Python side of things.

@hacker1024 hacker1024 changed the title dlib: Properly enable CUDA dlib: Properly enable CUDA in Python May 2, 2024
checkPhase = ''
${python.interpreter} nix_run_setup test
'';
} // lib.optionalAttrs dlib.cudaSupport { stdenv = dlib.cudaPackages.backendStdenv; })
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please inline this

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@hacker1024 hacker1024 changed the title dlib: Properly enable CUDA in Python python3Packages.dlib: Inherit stdenv and build flags from main derivation, fix tests Aug 29, 2024
@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/prs-ready-for-review/3032/4631

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: 👀 Awaits reviews
Development

Successfully merging this pull request may close these issues.

The dlib Python package does not respect cudaSupport from the given dlib library derivation
6 participants