Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

svd - check the size of storage space #881

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

jvdp1
Copy link
Member

@jvdp1 jvdp1 commented Oct 18, 2024

The procedure gesdd might return values for storage space larger than huge(lwork), resulting in a negative value fro lwork. Here is a proposition to solve this issue.

@jvdp1 jvdp1 requested a review from perazz October 18, 2024 07:45
Copy link
Contributor

@perazz perazz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM @jvdp1. Have you experienced cases where this actually happens?

Internally LAPACK is converting the integer(ilp) size request to a real value (can't permalink due to large file):

              work( 1 ) = stdlib_droundup_lwork( maxwrk )
              if( lwork<minwrk .and. .not.lquery ) then
                 info = -12
              end if

So maybe we should give a warning or return an error message when this happens?

@jvdp1
Copy link
Member Author

jvdp1 commented Oct 18, 2024

Here is a small program to repeat the issue:

program test_svd
  use, intrinsic:: iso_fortran_env, only: wp => real64, real64
  use stdlib_stats_distribution_normal, only: rvs_normal
  use stdlib_linalg, only: svd
  implicit none
  integer, parameter :: m = 50008, n = 25153
  integer, parameter :: neig = 10
  real(wp), allocatable :: A(:,:)
  real(wp), allocatable :: tu(:,:), ts(:), tvt(:,:)

  allocate(A(m,n), source = 10._wp)

  A = rvs_normal(A, 15._wp)

  print*,'size ',m, n

 !Compute SVD(A)
 associate(k => min(m, n))
  allocate(ts(k))
  allocate(tu(m,k))
  allocate(tvt(k,n))
 end associate

 call svd(A, ts, tu, tvt, full_matrices=.false.)

 print*,'ts ',ts(1:neig)

end program

@perazz
Copy link
Contributor

perazz commented Oct 18, 2024

OK I see, thank you @jvdp1. It seems like the test program requires ~10G elements, which is more than the max number of elements that an int32 integer can store. So I think that the current fix is definitely OK to avoid overflow, but we should think about providing the ilp64 option with support for int64 integer sizes.

That could be always provided by templating the ilp integer pointer. However, I'm worried that doubling the whole implementation would slow down build times even further. On the other hand, having to decide between ilp or ilp64 at compile time is not very user friendly IMHO.

@jalvesz
Copy link
Contributor

jalvesz commented Oct 18, 2024

@perazz maybe one idea to balance out these cases would be to provide a flag with_linalg_ilp64 and with_linalg_ilp32_64. The former would allow replacing the ilp by ilp64, the latter, enabling both. Have yet to think this through but it might enable the choice of a compact library or a bigger one that covers everything, depending on the needs. Don't know if it should be cpp only, fypp only or a combination of both that could enable it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants