Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add tril_ layer for lower triangular matrix operations #3017

Closed
wants to merge 0 commits into from

Conversation

Cydral
Copy link
Contributor

@Cydral Cydral commented Sep 23, 2024

This PR introduces a new tril_ layer to dlib, which implements lower triangular matrix operations similar to PyTorch's torch.tril() function. The layer allows for flexible lower triangular matrix masking with customizable diagonal offset and diagonal value.

Key features:

  • Implements lower triangular masking for tensors
  • Supports custom diagonal offset and diagonal value
  • Offers three convenient alias templates: tril, tril_mask, and tril_diag

This addition enhances dlib's neural network capabilities, allowing for more complex architectures that require lower triangular matrix operations.
The new layer can be particularly useful in attention mechanisms, triangular matrix operations, and other scenarios where lower triangular masking is required in neural network architectures.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant