Skip to content

Commit

Permalink
Namelist for input files (#86)
Browse files Browse the repository at this point in the history 
* added more logical tweaking for the grid clustering

* minor documentation fix

* implemented namelists for the input file

* updated the docs

* deleted `cudecomp.in` file

* ammended spurious mixed-precision code

* namelists need to be treated as a declaration

* Typo in test input file.
  • Loading branch information
@p-costa
p-costa authored Sep 1, 2023
1 parent 859e1d2 commit a54dd3d
Show file tree
Hide file tree
Showing 49 changed files with 852 additions and 701 deletions.
6 changes: 3 additions & 3 deletions Makefile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ SHELL=/bin/bash
NAME := cans

TARGET := $(NAME)
INPUT_FILE := input.nml

#PWD=$(shell pwd)
#ROOT_DIR := $(PWD)
Expand Down Expand Up @@ -77,9 +78,8 @@ $(EXE): $(OBJS)
$(FC) $(FFLAGS) $^ $(LIBS) $(INCS) -o $(EXE)

run: $(EXE)
@cp $(SRCS_DIR)/dns.in $(EXE_DIR)
@cp $(SRCS_DIR)/cudecomp.in $(EXE_DIR)
@printf "\nDefault input files *.in copied to run folder $(EXE_DIR)\n"
@cp $(SRCS_DIR)/$(INPUT_FILE) $(EXE_DIR)
@printf "\nDefault input file $(INPUT_FILE) copied to run folder $(EXE_DIR)\n"

# Create object files from Fortran source
$(OBJS): %.o: %
Expand Down
2 changes: 2 additions & 0 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@ P. Costa. *A FFT-based finite-difference solver for massively-parallel direct nu

## News

**[10/08/2023]:** The input files `dns.in` and `cudecomp.in` have been replaced with the namelist file `input.nml`, which makes parsing of input files and extensions in more complex solvers based on CaNS simpler. See the updated [`docs/INFO_INPUT.md`](docs/INFO_INPUT.md) file for more details. Additionally, we have added a new input parameter, `gtype` to explicitly select the type of grid stretching function.

**[03/02/2023]:** The input file `dns.in` has been simplified to avoid a common source of confusion. Instead of prescribing `uref`, `lref`, and `rey` (reference velocity and length scales, and Reynolds number) to calculate the fluid viscosity as `visc = uref*lref/rey`, we directly prescribe the inverse of the viscosity, `visci` (`visc = visci**(-1)`), so all inputs are dimensional (see the updated [`docs/INFO_INPUT.md`](docs/INFO_INPUT.md) file). Note that `visci` has the same value as the flow Reynolds number for all files under `examples`, as `uref` and `lref` were always equal to `1`. *This change is backwards-incompatible - former input files should be updated from [v2.2.0](https://github.com/CaNS-World/CaNS/tree/v2.2.0) onward!*

**[24/10/2022]:** Option `SINGLE_PRECISION_POISSON` has been removed from the `main` branch. While solving the Poisson in lower precision equation yields excellent results for many benchmarks, several of these cases also perform well when the whole calculation is performed in lower precision (see https://github.com/CaNS-World/CaNS/pull/42). Since this mode introduces significant complexity, it will be removed from the main branch for now in favor of a more readable code, a decision that can be reconsidered in the future. This option can still be explored in [v2.0.1](https://github.com/CaNS-World/CaNS/tree/v2.0.1), and is valuable for, e.g., setups with high Reynolds numbers and/or with extremely fine grids.
Expand Down
173 changes: 95 additions & 78 deletions docs/INFO_INPUT.md
View file
Original file line number Diff line number Diff line change
@@ -1,45 +1,61 @@
# about `dns.in`

Consider the following input file as example (corresponds to a turbulent plane channel flow):

~~~
512 256 144 ! itot, jtot, ktot
6. 3. 1. ! lx, ly, lz
0 1. ! gtype, gr
.95 1.0e5 ! cfl, dtmin
5640. ! visci
poi ! inivel
T ! is_wallturb
100000 100. 0.1 ! nstep, time_max, tw_max
T F F ! stop_type(1:3)
F T 0 ! restart,is_overwrite_save,nsaves_max
10 10 100 500 10000 5000 ! icheck, iout0d, iout1d, iout2d, iout3d, isave
P P P P D D ! cbcvel(0:1,1:3,1) [u BC type]
P P P P D D ! cbcvel(0:1,1:3,2) [v BC type]
P P P P D D ! cbcvel(0:1,1:3,3) [w BC type]
P P P P N N ! cbcpre(0:1,1:3 ) [p BC type]
0. 0. 0. 0. 0. 0. ! bcvel(0:1,1:3,1) [u BC value]
0. 0. 0. 0. 0. 0. ! bcvel(0:1,1:3,2) [v BC value]
0. 0. 0. 0. 0. 0. ! bcvel(0:1,1:3,3) [w BC value]
0. 0. 0. 0. 0. 0. ! bcpre(0:1,1:3 ) [p BC value]
0. 0. 0. ! bforce(1:3)
T F F ! is_forced(1:3)
1. 0. 0. ! velf(1:3)
2 2 ! dims(1:2)
~~~
# about the input file `input.nml`

Consider the following input file as example (corresponds to a turbulent plane channel flow). `&dns` defines a so-called Fortran namelist containing all the necessary physical and computational parameters to set a case.


```fortran
&dns
ng(1:3) = 512, 256, 144
l(1:3) = 6., 3., 1.
gtype = 1, gr = 0.
cfl = 0.95, dtmin = 1.e5
visci = 5640.
inivel = 'poi'
is_wallturb = T
nstep = 100000, time_max = 100., tw_max = 0.1
stop_type(1:3) = T, F, F
restart = F, is_overwrite_save = T, nsaves_max = 0
icheck = 10, iout0d = 10, iout1d = 100, iout2d = 500, iout3d = 10000, isave = 5000
cbcvel(0:1,1:3,1) = 'P','P', 'P','P', 'D','D'
cbcvel(0:1,1:3,2) = 'P','P', 'P','P', 'D','D'
cbcvel(0:1,1:3,3) = 'P','P', 'P','P', 'D','D'
cbcpre(0:1,1:3) = 'P','P', 'P','P', 'N','N'
bcvel(0:1,1:3,1) = 0.,0., 0.,0., 0.,0.
bcvel(0:1,1:3,2) = 0.,0., 0.,0., 0.,0.
bcvel(0:1,1:3,3) = 0.,0., 0.,0., 0.,0.
bcpre(0:1,1:3 ) = 0.,0., 0.,0., 0.,0.
bforce(1:3) = 0., 0., 0.
is_forced(1:3) = T, F, F
velf(1:3) = 1., 0., 0.
dims(1:2) = 2, 2
\
```
<details>

<summary>Tip for vim/nvim users</summary>
Consider adding the following lines in your `.vimrc` file for syntax highlighting of the namelist file:

```vim
if has("autocmd")
au BufNewFile,BufRead *.nml set filetype=fortran
au BufNewFile,BufRead *.namelist set filetype=fortran
endif
```

</details>

---
---

~~~
512 256 144 ! itot, jtot, ktot
6. 3. 1. ! lx, ly, lz
0 0. ! gtype, gr
~~~
```fortran
ng(1:3) = 512, 256, 144
l(1:3) = 6., 3., 1.
gtype = 1, gr = 0.
```

These lines set the computational grid.

`itot, jtot, ktot ` and `lx, ly, lz` are the **number of points** and **domain length** in each direction.
`ng(1:3)` and `l(1:3)` are the **number of points** and **domain length** in each direction.

`gtype` and `gr` are the **grid stretching type** and **grid stretching parameter** that tweak the non-uniform grid in the third direction; zero `gr` implies no stretching. See `initgrid.f90` for more details. The following options are available for `gtype`:

Expand All @@ -50,9 +66,9 @@ These lines set the computational grid.

---

~~~
.95 1.0e5 ! cfl, dtmin
~~~
```fortran
cfl = 0.95, dtmin = 1.e5
```

This line controls the simulation time step.

Expand All @@ -61,18 +77,18 @@ The time step is set to be equal to `min(cfl*dtmax,dtmin)`, i.e. the minimum val

---

~~~
5640. ! visci
~~~
```fortran
visci = 5640.
```

This line defines the inverse of the fluid viscosity, `visci`, meaning that the viscosity is `visc = visci**(-1)`. Note that, for a setup defined with unit reference length and velocity scales, `visci` has the same value as the flow Reynolds number.

---

~~~
poi ! inivel
T ! is_wallturb
~~~
```fortran
inivel = 'poi'
is_wallturb = T
```

These lines set the initial velocity field.

Expand All @@ -95,11 +111,11 @@ See `initflow.f90` for more details.

---

~~~
100000 100. 0.1 ! nstep, time_max, tw_max
T F F ! stop_type(1:3)
F T 0 ! restart,is_overwrite_save,nsaves_max
~~~
```fortran
nstep = 100000, time_max = 100., tw_max = 0.1
stop_type(1:3) = T, F, F
restart = F, is_overwrite_save = T, nsaves_max = 0
```

These lines set the simulation termination criteria and whether the simulation should be restarted from a checkpoint file.

Expand All @@ -125,9 +141,9 @@ a checkpoint file `fld.bin` will be saved before the simulation is terminated.

---

~~~
10 10 20 5000 10000 2000 ! icheck, iout0d, iout1d, iout2d, iout3d, isave
~~~
```fortran
icheck = 10, iout0d = 10, iout1d = 100, iout2d = 500, iout3d = 10000, isave = 5000
```

These lines set the frequency of time step checking and output:

Expand All @@ -142,16 +158,16 @@ These lines set the frequency of time step checking and output:

---

~~~
P P P P D D ! cbcvel(0:1,1:3,1) [u BC type]
P P P P D D ! cbcvel(0:1,1:3,2) [v BC type]
P P P P D D ! cbcvel(0:1,1:3,3) [w BC type]
P P P P N N ! cbcpre(0:1,1:3 ) [p BC type]
0. 0. 0. 0. 0. 0. ! bcvel(0:1,1:3,1) [u BC value]
0. 0. 0. 0. 0. 0. ! bcvel(0:1,1:3,2) [v BC value]
0. 0. 0. 0. 0. 0. ! bcvel(0:1,1:3,3) [w BC value]
0. 0. 0. 0. 0. 0. ! bcpre(0:1,1:3 ) [p BC value]
~~~
```fortran
cbcvel(0:1,1:3,1) = 'P','P', 'P','P', 'D','D'
cbcvel(0:1,1:3,2) = 'P','P', 'P','P', 'D','D'
cbcvel(0:1,1:3,3) = 'P','P', 'P','P', 'D','D'
cbcpre(0:1,1:3) = 'P','P', 'P','P', 'N','N'
bcvel(0:1,1:3,1) = 0.,0., 0.,0., 0.,0.
bcvel(0:1,1:3,2) = 0.,0., 0.,0., 0.,0.
bcvel(0:1,1:3,3) = 0.,0., 0.,0., 0.,0.
bcpre(0:1,1:3 ) = 0.,0., 0.,0., 0.,0.
```

These lines set the boundary conditions (BC).

Expand All @@ -173,11 +189,11 @@ The **last four rows** follow the same logic, but now for the BC **values** (dum

---

~~~
0. 0. 0. ! bforce(1:3)
T F F ! is_forced(1:3)
1. 0. 0. ! velf(1:3)
~~~
```fortran
bforce(1:3) = 0., 0., 0.
is_forced(1:3) = T, F, F
velf(1:3) = 1., 0., 0.
```

These lines set the flow forcing.

Expand All @@ -189,22 +205,23 @@ These lines set the flow forcing.

---

~~~
2 2 ! dims(1:2)
~~~
```fortran
dims(1:2) = 2, 2
```

This line set the grid of computational subdomains.

`dims` is the **processor grid**, the number of domain partitions along the first and second decomposed directions (which depend on the selected default pencil orientation). `dims(1)*dims(2)` corresponds therefore to the total number of computational subdomains. Setting `dims(:) = [0,0]` will trigger a runtime autotuning step to find the processor grid that minimizes transpose times. Note, however, that other components of the algorithm (e.g., collective I/O) may also be affected by the choice of processor grid.

# about `cudecomp.in`
# about the `&cudecomp` namelist under `input.nml`

In addition to the `dns.in` file, there is an **optional** input file to set some runtime configurations for the *cuDecomp* library. Consider the following file, which corresponds to the default options in case the file is not provided:
In addition to the `&dns` namelist in the input file, there is an **optional** namelist to set some runtime configurations for the *cuDecomp* library. Consider the following `&cudecomp` namelist, which corresponds to the default options in case the file is not provided:

~~~
0 T T ! cudecomp_t_comm_backend,cudecomp_is_t_enable_nccl,cudecomp_is_t_enable_nvshmem
0 T T ! cudecomp_t_comm_backend,cudecomp_is_t_enable_nccl,cudecomp_is_t_enable_nvshmem
~~~
```fortran
&cudecomp
cudecomp_t_comm_backend = 0, cudecomp_is_t_enable_nccl = T, cudecomp_is_t_enable_nvshmem = T
cudecomp_h_comm_backend = 0, cudecomp_is_h_enable_nccl = T, cudecomp_is_h_enable_nvshmem = T
```

The first line sets the configuration for the transpose communication backend autotuning. Here `cudecomp_t_comm_backend` can be one of:

Expand All @@ -230,4 +247,4 @@ The second line is analogous to the first one, but for halo communication backen

The other two boolean values, enable/disable the NCCL (`cudecomp_is_h_enable_nccl`) and NVSHMEM (`cudecomp_is_h_enable_nvshmem`) options for *halo* communication backend autotuning.

Finally, it is worth recalling that passing `dims(1:2) = [0,0]` under `dns.in` will trigger the *processor grid* autotuning, so there is no need to provide that option in the `cudecomp.in` input file.
Finally, it is worth recalling that passing `dims(1:2) = [0,0]` under `&dns` will trigger the *processor grid* autotuning, so there is no need to provide that option in the `&cudecomp` namelist.
23 changes: 0 additions & 23 deletions examples/_manuscript_lid_driven_cavity/dns.in
View file

This file was deleted.

30 changes: 30 additions & 0 deletions examples/_manuscript_lid_driven_cavity/input.nml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
&dns
ng(1:3) = 128, 128, 128
l(1:3) = 1., 1., 1.
gtype = 1, gr = 0.
cfl = 0.95, dtmin = 1.e5
visci = 1000.
inivel = 'zer'
is_wallturb = F
nstep = 10000, time_max = 100., tw_max = 0.1
stop_type(1:3) = T, F, F
restart = F, is_overwrite_save = T, nsaves_max = 0
icheck = 10, iout0d = 10, iout1d = 100, iout2d = 500, iout3d = 1000, isave = 2000
cbcvel(0:1,1:3,1) = 'D','D', 'D','D', 'D','D'
cbcvel(0:1,1:3,2) = 'D','D', 'D','D', 'D','D'
cbcvel(0:1,1:3,3) = 'D','D', 'D','D', 'D','D'
cbcpre(0:1,1:3) = 'N','N', 'N','N', 'N','N'
bcvel(0:1,1:3,1) = 0.,0., 0.,0., 0.,1.
bcvel(0:1,1:3,2) = 0.,0., 0.,0., 0.,0.
bcvel(0:1,1:3,3) = 0.,0., 0.,0., 0.,0.
bcpre(0:1,1:3 ) = 0.,0., 0.,0., 0.,0.
bforce(1:3) = 0., 0., 0.
is_forced(1:3) = F, F, F
velf(1:3) = 0., 0., 0.
dims(1:2) = 0, 0
\

&cudecomp
cudecomp_t_comm_backend = 0, cudecomp_is_t_enable_nccl = T, cudecomp_is_t_enable_nvshmem = T
cudecomp_h_comm_backend = 0, cudecomp_is_h_enable_nccl = T, cudecomp_is_h_enable_nvshmem = T
\
23 changes: 0 additions & 23 deletions examples/_manuscript_taylor_green_vortex/dns.in
View file

This file was deleted.

30 changes: 30 additions & 0 deletions examples/_manuscript_taylor_green_vortex/input.nml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
&dns
ng(1:3) = 512, 512, 512
l(1:3) = 6.2831853071795, 6.283185307179586, 6.283185307179586
gtype = 1, gr = 0.
cfl = 0.95, dtmin = 1.e5
visci = 1600.
inivel = 'tgv'
is_wallturb = F
nstep = 100000, time_max = 100., tw_max = 0.1
stop_type(1:3) = T, F, F
restart = F, is_overwrite_save = T, nsaves_max = 0
icheck = 10, iout0d = 10, iout1d = 20, iout2d = 500, iout3d = 1000, isave = 2000
cbcvel(0:1,1:3,1) = 'P','P', 'P','P', 'P','P'
cbcvel(0:1,1:3,2) = 'P','P', 'P','P', 'P','P'
cbcvel(0:1,1:3,3) = 'P','P', 'P','P', 'P','P'
cbcpre(0:1,1:3) = 'P','P', 'P','P', 'P','P'
bcvel(0:1,1:3,1) = 0.,0., 0.,0., 0.,0.
bcvel(0:1,1:3,2) = 0.,0., 0.,0., 0.,0.
bcvel(0:1,1:3,3) = 0.,0., 0.,0., 0.,0.
bcpre(0:1,1:3 ) = 0.,0., 0.,0., 0.,0.
bforce(1:3) = 0., 0., 0.
is_forced(1:3) = F, F, F
velf(1:3) = 0., 0., 0.
dims(1:2) = 0, 0
\

&cudecomp
cudecomp_t_comm_backend = 0, cudecomp_is_t_enable_nccl = T, cudecomp_is_t_enable_nvshmem = T
cudecomp_h_comm_backend = 0, cudecomp_is_h_enable_nccl = T, cudecomp_is_h_enable_nvshmem = T
\
Loading

0 comments on commit a54dd3d

Please sign in to comment.