Problems in setting data for β in MLABecLaplacian.

[solarmaterial]

Hey amrex team,
It's me again. Now I have a question about adding data to the face type Multifab $\beta$ in the equation $(A\alpha - B\nabla \cdot \beta \nabla)\phi^{n+1} = \phi^n$. In HeatEquation_EX3_C, we know that the data of the face_bcoef is simply set to dt by face_bcoef[idim].setVal(dt); since the $\beta$ is not affected by the $\phi$. Now the equation become non-linearzed, and the face_bcoef do effected by the $\phi$, so I have to transfer data from the cell-centerd $\phi$ to the face type $\beta$, and I use MFIter to do so. Here are two ways I've tried,(phi_k is a cell-centerd Multifab).
(1d calculation)
code 1. (for(MFIter mfi(face_bcoef[0]);......const Box& bx = mfi.validbox();)

     for(MFIter mfi(face_bcoef[0]); mfi.isValid(); ++mfi)
        {
            const Box& bx = mfi.validbox();
            const & array_phi_k = phi_k.array(mfi);
            const & face_bcoef_x = face_bcoef[0].array(mfi);     
            Dim3 lox = lbound(xbx); Dim3 hix = ubound(xbx);
        ParallelFor(bx, [=] AMREX_GPU_DEVICE(int i, int j, int k)
            {
            if(i == hix.x)
            {face_bcoef_x(i,j,k) = face_bcoef_x(i-1,j,k);}
            else if(i == lox.x)
            {face_bcoef_x(i,j,k) = (array_phi_k(i,j,k) + array_phi_k(i+1,j,k)) / 2;}
            else
            {face_bcoef_x(i,j,k) = (array_phi_k(i-1,j,k) + array_phi_k(i,j,k)) / 2;}
            });
        }

code 2.(for ( MFIter mfi(phi_k); mfi.isValid();...... const Box& xbx = mfi.nodaltilebox(0);)

for ( MFIter mfi(phi_k); mfi.isValid(); ++mfi )
    {
        const Box& xbx = mfi.nodaltilebox(0);
        auto const& face_bcoef_x = face_bcoef[0].array(mfi);  
        auto const& array_phi_k = phi_k.array(mfi);
        Dim3 lox = lbound(xbx); Dim3 hix = ubound(xbx);
     ParallelFor(xbx,
        [=] AMREX_GPU_DEVICE (int i, int j, int k)
        {
            if(i == hix.x)
            {face_bcoef_x(i,j,k) = face_bcoef_x(i-1,j,k);}
            else if(i == lox.x)
            {face_bcoef_x(i,j,k) = (array_phi_k(i,j,k) + array_phi_k(i+1,j,k)) / 2;}
            else
            {face_bcoef_x(i,j,k) = (array_phi_k(i-1,j,k) + array_phi_k(i,j,k)) / 2;}
        });
    }

the face_bcoef[0] printed by the above codes are exactly the same, after I load them into the MLMG solver, the solver both broken at the beginning. Why is that? Since the initial rhs has no NaNs, and the $\alpha$,(acoef.setVal(1.0);) $\beta$ have no NaNs either, the MLMG should work at least one iteration. I've set max_grid_size = n_cell =256 so that the face_bcoef[0] has no overlapping point to make sure the parallelfor above is correct.

./main1d.gnu.DEBUG.ex inputs_1d amrex.fpe_trap_invalid=1 amrex.fpe_trap_zero=1 amrex.fpe_trap_overflow=1
AMReX (23.05) initialized
MLMG: # of AMR levels: 1
      # of MG levels on the coarsest AMR level: 8
Erroneous arithmetic operation
See Backtrace.0 file for details

will the code above be able to make sure that the $\beta$ is correctly passed to the MLMG? Or did I miss something? I'm sorry for bothering you again. Still looking forward to your reply.

Yours sincerely,
solar

[WeiqunZhang]

Is this related to #3834? What's in Backtrace.0?

The code above has potential issues. For it to be more robust, you will need to call FillBoundary before that and you should use the lo and hi of Geometry's Domain Box, not the local tilebox or validbox. And face_bcoef_x(i,j,k) = face_bcoef_x(i-1,j,k) will not work if the code is compiled for GPU. Nevertheless, if your have max_grid_size = n_cell = 256 and it's CPU, it seems fine.

If you put your code or a reproducer in a public github repo, I am happy to take a look.

[solarmaterial]

Thank you very much! Yes, this is related to #3834, and the Backtrace.0 is below. I'm also trying to upload my code to the github repository. I really, really appreciate your help!!!

 0: ./main1d.gnu.DEBUG.ex(+0x1e4786) [0x5591af593786]
    amrex::BLBackTrace::print_backtrace_info(_IO_FILE*) at /home/solarmaterial/desktop/testhanshu/non-linear/HeatEquation_EX3_C/Exec/../../../../amrex/Src/Base/AMReX_BLBackTrace.cpp:190

 1: ./main1d.gnu.DEBUG.ex(+0x1e4277) [0x5591af593277]
    amrex::BLBackTrace::handler(int) at /home/solarmaterial/desktop/testhanshu/non-linear/HeatEquation_EX3_C/Exec/../../../../amrex/Src/Base/AMReX_BLBackTrace.cpp:100

 2: /lib/x86_64-linux-gnu/libc.so.6(+0x43090) [0x7f9ecaccb090]

 3: ./main1d.gnu.DEBUG.ex(+0x254dbf) [0x5591af603dbf]
    void amrex::mlabeclap_adotx<double>(int, int, int, int, amrex::Array4<double> const&, amrex::Array4<double const> const&, amrex::Array4<double const> const&, amrex::Array4<double const> const&, amrex::GpuArray<double, 1u> const&, double, double) at /home/solarmaterial/desktop/testhanshu/non-linear/HeatEquation_EX3_C/Exec/../../../../amrex/Src/LinearSolvers/MLMG/AMReX_MLABecLap_1D_K.H:17 (discriminator 11)
 (inlined by) amrex::MLABecLaplacianT<amrex::MultiFab>::Fapply(int, int, amrex::MultiFab&, amrex::MultiFab const&) const at /home/solarmaterial/desktop/testhanshu/non-linear/HeatEquation_EX3_C/Exec/../../../../amrex/Src/LinearSolvers/MLMG/AMReX_MLABecLaplacian.H:815 (discriminator 11)

 4: ./main1d.gnu.DEBUG.ex(+0x238250) [0x5591af5e7250]
    amrex::MLCellLinOpT<amrex::MultiFab>::apply(int, int, amrex::MultiFab&, amrex::MultiFab&, amrex::LinOpEnumType::BCMode, amrex::LinOpEnumType::StateMode, amrex::MLMGBndryT<amrex::MultiFab> const*) const at /home/solarmaterial/desktop/testhanshu/non-linear/HeatEquation_EX3_C/Exec/../../../../amrex/Src/LinearSolvers/MLMG/AMReX_MLCellLinOp.H:1172

 5: ./main1d.gnu.DEBUG.ex(+0x2383de) [0x5591af5e73de]
    amrex::MLCellLinOpT<amrex::MultiFab>::solutionResidual(int, amrex::MultiFab&, amrex::MultiFab&, amrex::MultiFab const&, amrex::MultiFab const*) at /home/solarmaterial/desktop/testhanshu/non-linear/HeatEquation_EX3_C/Exec/../../../../amrex/Src/LinearSolvers/MLMG/AMReX_MLCellLinOp.H:1200

 6: ./main1d.gnu.DEBUG.ex(+0x21bbf8) [0x5591af5cabf8]
    amrex::MLMGT<amrex::MultiFab>::computeMLResidual(int) at /home/solarmaterial/desktop/testhanshu/non-linear/HeatEquation_EX3_C/Exec/../../../../amrex/Src/LinearSolvers/MLMG/AMReX_MLMG.H:1628 (discriminator 4)

 7: ./main1d.gnu.DEBUG.ex(+0x2469c) [0x5591af3d369c]
    double amrex::MLMGT<amrex::MultiFab>::solve<amrex::MultiFab>(amrex::Vector<amrex::MultiFab*, std::allocator<amrex::MultiFab*> > const&, amrex::Vector<amrex::MultiFab const*, std::allocator<amrex::MultiFab const*> > const&, double, double, char const*) at /home/solarmaterial/desktop/testhanshu/non-linear/HeatEquation_EX3_C/Exec/../../../../amrex/Src/LinearSolvers/MLMG/AMReX_MLMG.H:393

 8: ./main1d.gnu.DEBUG.ex(+0x21b4e) [0x5591af3d0b4e]
    double amrex::MLMGT<amrex::MultiFab>::solve<amrex::MultiFab>(std::initializer_list<amrex::MultiFab*>, std::initializer_list<amrex::MultiFab const*>, double, double, char const*) at /home/solarmaterial/desktop/testhanshu/non-linear/HeatEquation_EX3_C/Exec/../../../../amrex/Src/LinearSolvers/MLMG/AMReX_MLMG.H:348 (discriminator 4)

 9: ./main1d.gnu.DEBUG.ex(+0x1e501) [0x5591af3cd501]
    advance(amrex::MultiFab&, amrex::MultiFab&, double, amrex::Geometry const&, amrex::BoxArray const&, amrex::DistributionMapping const&, amrex::Vector<amrex::BCRec, std::allocator<amrex::BCRec> > const&) at /home/solarmaterial/desktop/testhanshu/non-linear/HeatEquation_EX3_C/Exec/../Source/advance.cpp:224 (discriminator 2)

10: ./main1d.gnu.DEBUG.ex(+0x155ff) [0x5591af3c45ff]
    main_main() at /home/solarmaterial/desktop/testhanshu/non-linear/HeatEquation_EX3_C/Exec/../Source/main.cpp:183

11: ./main1d.gnu.DEBUG.ex(+0x145df) [0x5591af3c35df]
    main at /home/solarmaterial/desktop/testhanshu/non-linear/HeatEquation_EX3_C/Exec/../Source/main.cpp:17

12: /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3) [0x7f9ecacac083]

13: ./main1d.gnu.DEBUG.ex(+0x144ae) [0x5591af3c34ae]
    ?? ??:0

[solarmaterial]

I've uploaded my code to my github repository: "amrex". Thank you again for your help!