Turn on operator partial assembly by default #168

sebastiangrimberg · 2024-01-09T01:54:53Z

NOTE: Based on #167

All examples use p-refinement in place of mesh refinement now as well.

In order to ensure consistency across runs for eigenvector normalizations, we now normalize with respect to the mass matrix (unit electric field energy). The previous l2-norm on the true dofs is not consistent for high-order Nedelec spaces where the orientations (and thus definitions of the dofs) change based on the partitioning.

Reference issue: #3

sebastiangrimberg · 2024-02-01T03:54:23Z

palace/drivers/eigensolver.cpp

+    if (i < iodata.solver.eigenmode.n)
+    {
+      // Only consider the desired number of modes for the error indicator.
+      estimator.AddErrorIndicator(E, indicator);


Note: I realized this is incorrect, since below the indicator is postprocessed for i == 0 which means the value written isn't actually considering all modes. I should move it back outside the loop.

Fixed in 95dbe4d

… partial assembly for all operators

…ctors for consistency of results across processor counts (where true dof definitions, and thus l2 norm, might change) Also re-baseline cavity results for the new change.

hughcars

A tiny comment fix but otherwise LGTM.

CHANGELOG.md

palace/drivers/eigensolver.cpp

hughcars · 2024-02-02T23:15:57Z

palace/linalg/arpack.cpp

@@ -682,25 +713,20 @@ int ArpackPEPSolver::Solve()
  if (!eig)
  {
    eig = std::make_unique<std::complex<double>[]>(nev + 1);
-    perm = std::make_unique<int[]>(nev + 1);
-    res = std::make_unique<double[]>(nev + 1);
+    perm = std::make_unique<int[]>(nev);


Why the change from nev+1 to nev, was it previously oversized?

Yep, only the eigenvectors need to be eig + 1 for ARPACK. Everything else is sized for nev and used by on the Palace side.

hughcars · 2024-02-02T23:25:54Z

test/examples/ref/cavity/impedance/domain-E.csv

+ 1.000000000e+00,        +9.139656208e-02,        +9.139657101e-02,        +0.000000000e+00,        +0.000000000e+00,        +9.139656208e-02,        +1.000000000e+00,        +9.139657101e-02,        +1.000000000e+00
+ 2.000000000e+00,        +9.139656208e-02,        +9.139657093e-02,        +0.000000000e+00,        +0.000000000e+00,        +9.139656208e-02,        +1.000000000e+00,        +9.139657093e-02,        +1.000000000e+00
+ 3.000000000e+00,        +9.139656208e-02,        +9.139657091e-02,        +0.000000000e+00,        +0.000000000e+00,        +9.139656208e-02,        +1.000000000e+00,        +9.139657091e-02,        +1.000000000e+00
+ 4.000000000e+00,        +9.139656208e-02,        +9.139657118e-02,        +0.000000000e+00,        +0.000000000e+00,        +9.139656208e-02,        +1.000000000e+00,        +9.139657118e-02,        +1.000000000e+00
+ 5.000000000e+00,        +9.139656208e-02,        +9.139657079e-02,        +0.000000000e+00,        +0.000000000e+00,        +9.139656208e-02,        +1.000000000e+00,        +9.139657079e-02,        +1.000000000e+00
+ 6.000000000e+00,        +9.139656208e-02,        +9.139657070e-02,        +0.000000000e+00,        +0.000000000e+00,        +9.139656208e-02,        +1.000000000e+00,        +9.139657070e-02,        +1.000000000e+00
+ 7.000000000e+00,        +9.139656208e-02,        +9.139657058e-02,        +0.000000000e+00,        +0.000000000e+00,        +9.139656208e-02,        +1.000000000e+00,        +9.139657058e-02,        +1.000000000e+00
+ 8.000000000e+00,        +9.139656208e-02,        +9.139657024e-02,        +0.000000000e+00,        +0.000000000e+00,        +9.139656208e-02,        +1.000000000e+00,        +9.139657024e-02,        +1.000000000e+00
+ 9.000000000e+00,        +9.139656208e-02,        +9.139657046e-02,        +0.000000000e+00,        +0.000000000e+00,        +9.139656208e-02,        +1.000000000e+00,        +9.139657046e-02,        +1.000000000e+00
+ 1.000000000e+01,        +9.139656208e-02,        +9.139657036e-02,        +0.000000000e+00,        +0.000000000e+00,        +9.139656208e-02,        +1.000000000e+00,        +9.139657036e-02,        +1.000000000e+00
+ 1.100000000e+01,        +9.139656208e-02,        +9.139657066e-02,        +0.000000000e+00,        +0.000000000e+00,        +9.139656208e-02,        +1.000000000e+00,        +9.139657066e-02,        +1.000000000e+00
+ 1.200000000e+01,        +9.139656208e-02,        +9.139657004e-02,        +0.000000000e+00,        +0.000000000e+00,        +9.139656208e-02,        +1.000000000e+00,        +9.139657004e-02,        +1.000000000e+00
+ 1.300000000e+01,        +9.139656208e-02,        +9.139657079e-02,        +0.000000000e+00,        +0.000000000e+00,        +9.139656208e-02,        +1.000000000e+00,        +9.139657079e-02,        +1.000000000e+00
+ 1.400000000e+01,        +9.139656208e-02,        +9.139657060e-02,        +0.000000000e+00,        +0.000000000e+00,        +9.139656208e-02,        +1.000000000e+00,        +9.139657060e-02,        +1.000000000e+00
+ 1.500000000e+01,        +9.139656208e-02,        +9.139657051e-02,        +0.000000000e+00,        +0.000000000e+00,        +9.139656208e-02,        +1.000000000e+00,        +9.139657051e-02,        +1.000000000e+00


Every number in here being functionally equal now makes this look a little weird, an argument could be made it's not necessary to write.

I'd argue keeping the format consistent with the other simulation types is worth it. Also it's a good check to make sure something did not break! Note also, if you're referring to electric vs. magnetic field energy, they are not equal as a result of the error tolerance associated with the eigenvalue solver. Also for problems with lumped elements, they won't be equal in general.

hughcars · 2024-02-02T23:26:32Z

test/examples/ref/cavity/impedance/error-indicators.csv

@@ -1,2 +1,2 @@
                    Norm,                 Minimum,                 Maximum,                    Mean
-        +1.831462799e-02,        +4.326597058e-04,        +1.397659264e-03,        +7.450386480e-04
+        +1.786937592e-03,        +1.175148951e-04,        +3.263378876e-04,        +2.040337637e-04


Good old spectral convergence.

I liked it too!

hughcars · 2024-02-02T23:33:16Z

test/examples/ref/cpw/lumped_uniform/error-indicators.csv

@@ -1,2 +1,2 @@
                    Norm,                 Minimum,                 Maximum,                    Mean
-        +7.056448953e-01,        +1.149467165e-06,        +1.689680136e-02,        +8.751155027e-04
+        +6.039988870e-01,        +2.399647627e-06,        +2.597476399e-02,        +2.443466119e-03


Nice demonstration of irregularity, the min max and mean rise, but the norm decreases.

sebastiangrimberg added enhancement New feature or request performance Related to performance labels Jan 9, 2024

sebastiangrimberg changed the base branch from sjg/libceed-qdata-dev-bkp to sjg/libceed-solvers-dev January 9, 2024 01:55

sebastiangrimberg force-pushed the sjg/pa-default-dev branch 2 times, most recently from 09672fe to 43804da Compare January 10, 2024 03:18

sebastiangrimberg force-pushed the sjg/libceed-solvers-dev branch from fba6b59 to 6a5574b Compare January 10, 2024 03:19

sebastiangrimberg force-pushed the sjg/pa-default-dev branch 2 times, most recently from deabef5 to 5fc48fe Compare January 10, 2024 03:30

sebastiangrimberg force-pushed the sjg/libceed-solvers-dev branch from 6a5574b to e321258 Compare January 10, 2024 03:30

sebastiangrimberg force-pushed the sjg/libceed-solvers-dev branch 2 times, most recently from b04a669 to 8ec1ad4 Compare January 18, 2024 17:53

sebastiangrimberg force-pushed the sjg/pa-default-dev branch 3 times, most recently from 8a72dc9 to b2aa993 Compare January 22, 2024 20:06

sebastiangrimberg marked this pull request as ready for review January 23, 2024 22:28

Base automatically changed from sjg/libceed-solvers-dev to main January 29, 2024 17:20

sebastiangrimberg force-pushed the sjg/pa-default-dev branch from b2aa993 to aacc041 Compare January 29, 2024 17:22

sebastiangrimberg requested a review from hughcars January 31, 2024 02:08

sebastiangrimberg force-pushed the sjg/pa-default-dev branch 3 times, most recently from f2674b5 to e86da12 Compare January 31, 2024 03:15

sebastiangrimberg commented Feb 1, 2024

View reviewed changes

sebastiangrimberg added 5 commits February 2, 2024 13:32

Change default partial assembly order threshold to 1, which activates…

cf8852f

… partial assembly for all operators

Update examples: Prefer high-order over mesh refinement with PA

b79dd52

Rebaseline regression examples after order/mesh refinement change

fec9a70

Improvements to baseline script

2d4740b

Adjust eigenvalue solver to always use mass matrix-normalized eigenve…

43b7b42

…ctors for consistency of results across processor counts (where true dof definitions, and thus l2 norm, might change) Also re-baseline cavity results for the new change.

sebastiangrimberg force-pushed the sjg/pa-default-dev branch from 95dbe4d to 43b7b42 Compare February 2, 2024 22:37

Revert error estimation regression for eigenvalue solves from c4bbffb

18f7b46

hughcars approved these changes Feb 2, 2024

View reviewed changes

PR feedback: Update changelog

4ec2ca6

hughcars approved these changes Feb 4, 2024

View reviewed changes

sebastiangrimberg merged commit 4a1a873 into main Feb 5, 2024
17 checks passed

sebastiangrimberg deleted the sjg/pa-default-dev branch February 5, 2024 16:40

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Turn on operator partial assembly by default #168

Turn on operator partial assembly by default #168

sebastiangrimberg commented Jan 9, 2024 •

edited

Loading

sebastiangrimberg Feb 1, 2024

sebastiangrimberg Feb 1, 2024

hughcars left a comment

hughcars Feb 2, 2024

sebastiangrimberg Feb 2, 2024

hughcars Feb 2, 2024

sebastiangrimberg Feb 2, 2024 •

edited

Loading

hughcars Feb 2, 2024

sebastiangrimberg Feb 2, 2024

hughcars Feb 2, 2024

Turn on operator partial assembly by default #168

Turn on operator partial assembly by default #168

Conversation

sebastiangrimberg commented Jan 9, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hughcars left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sebastiangrimberg Feb 2, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sebastiangrimberg commented Jan 9, 2024 •

edited

Loading

sebastiangrimberg Feb 2, 2024 •

edited

Loading