From 44b1cc84a3a2fe84c7e14faaa1ecdad149663ba9 Mon Sep 17 00:00:00 2001 From: Edward Hartnett Date: Wed, 7 Aug 2024 06:40:45 -0600 Subject: [PATCH 1/7] added file --- docs/building_autotools.md | 87 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 87 insertions(+) create mode 100644 docs/building_autotools.md diff --git a/docs/building_autotools.md b/docs/building_autotools.md new file mode 100644 index 0000000000..f19c0f7c9f --- /dev/null +++ b/docs/building_autotools.md @@ -0,0 +1,87 @@ +Build Instructions for NetCDF-C using autoconf/automake/libtool {#netCDF-autotools} +=========================================== + +[TOC] + +# Overview {#cmake_overview} + +Starting with netCDF-C 4.3.0, we are happy to announce the inclusion of CMake support. CMake will allow for building netCDF on a wider range of platforms, include Microsoft Windows with Visual Studio. CMake support also provides robust unit and regression testing tools. We will also maintain the standard autotools-based build system in parallel. + +In addition to providing new build options for netCDF-C, we will also provide pre-built binary downloads for the shared versions of netCDF for use with Visual Studio. + + +# Requirements {#autotools_requirements} +The following packages are required to build netCDF-C. + +* netCDF-C Source Code +* Optional Requirements: + * HDF5 Libraries for netCDF4/HDF5 support. + * libcurl for DAP support. + +
+ +
+ +# The Autotools Build Process {#autotools_build} + +There are four steps in the Build Process when using autotools: + +1. Configuration: Before compiling, the software is configured based on the desired options. +2. Building: Once configuration is complete, the libraries are compiled. +3. Testing: Post-build, it is possible to run tests to ensure the functionality of the netCDF-C libraries. +4. Installation: If all tests pass, the libraries can be installed in the location specified during configuration. + +For users who prefer pre-built binaries, installation packages are available at \ref winbin + +## Configuration {#autotools_configuration} + +The output of the configuration step are the makefiles which build the library, utilities, and tests. + +### Common Configure Options {#autotools_common_options} + +| **Option** | **Autotools** | **CMake** | +| :------- | :---- | :----- | +Specify Install Location | --prefix=PREFIX | -D"CMAKE\_INSTALL\_PREFIX=PREFIX" +Enable/Disable netCDF-4 | --enable-netcdf-4
--disable-netcdf-4 | -D"ENABLE\_NETCDF\_4=ON"
-D"ENABLE\_NETCDF\_4=OFF" +Enable/Disable DAP | --enable-dap
--disable-dap | -D"ENABLE\_DAP=ON"
-D"ENABLE\_DAP=OFF" +Enable/Disable Utilities | --enable-utilities
--disable-utilities | -D"BUILD\_UTILITIES=ON"
-D"BUILD\_UTILITIES=OFF" +Specify shared/Static Libraries | --enable-shared
--enable-static | -D"BUILD\_SHARED\_LIBS=ON"
-D"BUILD\_SHARED\_LIBS=OFF" +Enable/Disable Tests | --enable-testsets
--disable-testsets | -D"ENABLE\_TESTS=ON"
-D"ENABLE\_TESTS=OFF" +Specify a custom library location | Use *CFLAGS* and *LDFLAGS* | -D"CMAKE\_PREFIX\_PATH=/usr/custom_libs/" + +A full list of options can be found by invoking `configure --help`. + +### Configuring your build from the command line. {#autotools_command_line} + +The easiest configuration case would be one in which all of the +dependent libraries are installed on the system path (in either +Unix/Linux or Windows) and all the default options are desired. In the +source directory: + +> $ ./configure + +If you have libraries installed in a custom directory, you will need to +set the CPPFLAGS and LDFLAGS environment variables to tell the compiler where the +libraries are installed. For example: + +> $ CPPFLAGS=-I/usr/local/hdf5-1.14.3/include LDFLAGS=-L/usr/local/hdf5-1.14.3/lib ./configure --prefix=/usr/local/netcdf-c-4.9.3 + +## Building {#autotools_building} + +The compiler can be executed directly with 'make'. + +> $ make + +## Testing {#autotools_testing} + +Testing can be executed with: + +> $ make check + +## Installation {#autotools_installation} + +Once netCDF has been built and tested, it may be installed using the following command: + +> $ make install + + From 23a41dcee5cc149c2771784db12a2039031c82c5 Mon Sep 17 00:00:00 2001 From: Edward Hartnett Date: Wed, 7 Aug 2024 06:41:08 -0600 Subject: [PATCH 2/7] added file --- docs/{building_autotools.md => building-with-autotools.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename docs/{building_autotools.md => building-with-autotools.md} (100%) diff --git a/docs/building_autotools.md b/docs/building-with-autotools.md similarity index 100% rename from docs/building_autotools.md rename to docs/building-with-autotools.md From 0c89b71120b3cbf47cd214151106b4877127e0d9 Mon Sep 17 00:00:00 2001 From: Edward Hartnett Date: Wed, 7 Aug 2024 06:58:30 -0600 Subject: [PATCH 3/7] added autotools build document --- docs/building-with-autotools.md | 94 +++++++++++++++++++++++++++++++++ 1 file changed, 94 insertions(+) diff --git a/docs/building-with-autotools.md b/docs/building-with-autotools.md index f19c0f7c9f..faf4b36381 100644 --- a/docs/building-with-autotools.md +++ b/docs/building-with-autotools.md @@ -48,6 +48,7 @@ Enable/Disable Utilities | --enable-utilities
--disable-utilities | -D"BUIL Specify shared/Static Libraries | --enable-shared
--enable-static | -D"BUILD\_SHARED\_LIBS=ON"
-D"BUILD\_SHARED\_LIBS=OFF" Enable/Disable Tests | --enable-testsets
--disable-testsets | -D"ENABLE\_TESTS=ON"
-D"ENABLE\_TESTS=OFF" Specify a custom library location | Use *CFLAGS* and *LDFLAGS* | -D"CMAKE\_PREFIX\_PATH=/usr/custom_libs/" +Enable parallel I/O tests | --enable-parallel-tests | -D"NETCDF\_ENABLE_PARALLEL\_TESTS=ON" A full list of options can be found by invoking `configure --help`. @@ -66,6 +67,99 @@ libraries are installed. For example: > $ CPPFLAGS=-I/usr/local/hdf5-1.14.3/include LDFLAGS=-L/usr/local/hdf5-1.14.3/lib ./configure --prefix=/usr/local/netcdf-c-4.9.3 +#### Building with Parallel I/O. {#autotools_parallel_io} + +NetCDF will build with parallel I/O if the C compiler is an MPI +compiler, and HDF5 was built for parallel I/O. To build netcdf-c for +parallel I/O, first build HDF5 for parallel I/O, then build netcdf-c +like this: + +> $ CC=mpicc CPPFLAGS=-I/usr/local/hdf5-1.14.3_mpich/include LDFLAGS=-L/usr/local/hdf5-1.14.3_mpicj/lib ./configure --prefix=/usr/local/netcdf-c-4.9.3_mpich --enable-parallel-tests + +The parallel I/O tests will only run if the additional configure +option is used: --enable-parallel-tests. Those tests run (by default) +with mpiexec, on 4, 16, or 32 processors. If mpiexec cannot be used on +your login-nodes, a different command can be used to launch the +parallel I/O tests. Used the --with-mpiexec configure option to set a +different parallel I/O job launcher: + +> $ CC=mpicc CPPFLAGS=-I/usr/local/hdf5-1.14.3_mpich/include LDFLAGS=-L/usr/local/hdf5-1.14.3_mpicj/lib ./configure --prefix=/usr/local/netcdf-c-4.9.3_mpich --enable-parallel-tests --with-mpiexec='srun -A acct_name -q queue_name' + +## Checking the Configure Summary + +At the end of the configure, a summary of the build will be +output. It's important to check this to ensure that the desired build +features are accurately set. For example: + +
+# NetCDF C Configuration Summary
+==============================
+
+# General
+-------
+NetCDF Version:		4.9.4-development
+Dispatch Version:       5
+Configured On:		Wed Aug  7 06:53:22 MDT 2024
+Host System:		x86_64-pc-linux-gnu
+Build Directory: 	/home/ed/netcdf-c
+Install Prefix:         /usr/local/netcdf-c-4.9.3
+Plugin Install Prefix:  N.A.
+
+# Compiling Options
+-----------------
+C Compiler:		/usr/bin/gcc
+CFLAGS:			 -fno-strict-aliasing
+CPPFLAGS:		-I/usr/local/hdf5-1.14.3/include
+LDFLAGS:		-L/usr/local/hdf5-1.14.3/lib
+AM_CFLAGS:		
+AM_CPPFLAGS:		
+AM_LDFLAGS:		
+Shared Library:		yes
+Static Library:		yes
+Extra libraries:	-lhdf5_hl -lhdf5 -lm -lz -lsz -lzstd -lxml2 -lcurl 
+XML Parser:             libxml2
+
+# Features
+--------
+Benchmarks:		no
+NetCDF-2 API:		yes
+HDF4 Support:		no
+HDF5 Support:		yes
+CDF5 Support:		yes
+NC-4 Parallel Support:	no
+PnetCDF Support:	no
+
+DAP2 Support:		yes
+DAP4 Support:		yes
+Byte-Range Support:	yes
+
+S3 Support:	        no
+S3 SDK:  	        none
+
+NCZarr Support:		yes
+NCZarr Zip Support:     no
+
+Diskless Support:	yes
+MMap Support:		no
+ERANGE Fill Support:	no
+Relaxed Boundary Check:	yes
+
+Quantization:		yes
+Logging:     		no
+SZIP Write Support:     yes
+Standard Filters:       bz2 deflate szip zstd
+ZSTD Support:           yes
+Parallel Filters:       yes
+
+ +Important settings include: +* Install Prefix - this is where netCDF will be installed. +* HDF5 Support - must be 'yes' to use netcdf-4/HDF5 files. +* NC-4 Parallel Support - Must be 'yes' to use HDF5 parallel I/O. +* PnetCDF Support - Must be 'yes' to use pnetcdf library for parallel I/O with classic netCDF files. +* Standard Filters - Must include 'zstd' if zstandard compression is to be used. +* DAP2/DAP4 Support - Must be 'yes' if OPeNDAP is to be used. + ## Building {#autotools_building} The compiler can be executed directly with 'make'. From 996d7e6d4266480dafe6e8bc3e7e078a6e226cd9 Mon Sep 17 00:00:00 2001 From: Edward Hartnett Date: Wed, 7 Aug 2024 07:14:26 -0600 Subject: [PATCH 4/7] added parallel build info to cmake build document --- docs/building-with-cmake.md | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/docs/building-with-cmake.md b/docs/building-with-cmake.md index 3448976b4b..003fed6b49 100644 --- a/docs/building-with-cmake.md +++ b/docs/building-with-cmake.md @@ -67,6 +67,24 @@ If you have libraries installed in a custom directory, you may need to specify t > $ cmake [Source Directory] -DCMAKE\_PREFIX\_PATH=/usr/custom_libraries/ +#### Building with Parallel I/O. {#cmake_parallel_io} + +NetCDF will build with parallel I/O if the C compiler is an MPI +compiler, and HDF5 was built for parallel I/O. To build netcdf-c for +parallel I/O, first build HDF5 for parallel I/O, then build netcdf-c +like this: + +> $ CC=mpicc cmake [Source Directory] -DNETCDF\_ENABLE\_PARALLEL_TESTS=ON + +The parallel I/O tests will only run if the additional configure +option is used: NETCDF_ENABLE_PARALLEL_TESTS. Those tests run (by default) +with mpiexec, on 4, 16, or 32 processors. If mpiexec cannot be used on +your login-nodes, a different command can be used to launch the +parallel I/O tests. Used the NETCDF_MPIEXEC option to set a +different parallel I/O job launcher: + +> $ CC=mpicc cmake [Source Directory] -DNETCDF\_ENABLE\_PARALLEL_TESTS=ON -DNETCDF\_MPIEXEC='srun -A account' + ## Building {#cmake_building} The compiler can be executed directly with 'make' or the appropriate command for the configurator which was used. From 7748352b05a3bdd9c4e96d927b9d0dc1dd70219c Mon Sep 17 00:00:00 2001 From: Edward Hartnett Date: Wed, 7 Aug 2024 07:16:41 -0600 Subject: [PATCH 5/7] cleanup --- docs/building-with-autotools.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/docs/building-with-autotools.md b/docs/building-with-autotools.md index faf4b36381..e094b2b16b 100644 --- a/docs/building-with-autotools.md +++ b/docs/building-with-autotools.md @@ -41,14 +41,14 @@ The output of the configuration step are the makefiles which build the library, | **Option** | **Autotools** | **CMake** | | :------- | :---- | :----- | -Specify Install Location | --prefix=PREFIX | -D"CMAKE\_INSTALL\_PREFIX=PREFIX" -Enable/Disable netCDF-4 | --enable-netcdf-4
--disable-netcdf-4 | -D"ENABLE\_NETCDF\_4=ON"
-D"ENABLE\_NETCDF\_4=OFF" -Enable/Disable DAP | --enable-dap
--disable-dap | -D"ENABLE\_DAP=ON"
-D"ENABLE\_DAP=OFF" -Enable/Disable Utilities | --enable-utilities
--disable-utilities | -D"BUILD\_UTILITIES=ON"
-D"BUILD\_UTILITIES=OFF" -Specify shared/Static Libraries | --enable-shared
--enable-static | -D"BUILD\_SHARED\_LIBS=ON"
-D"BUILD\_SHARED\_LIBS=OFF" -Enable/Disable Tests | --enable-testsets
--disable-testsets | -D"ENABLE\_TESTS=ON"
-D"ENABLE\_TESTS=OFF" -Specify a custom library location | Use *CFLAGS* and *LDFLAGS* | -D"CMAKE\_PREFIX\_PATH=/usr/custom_libs/" -Enable parallel I/O tests | --enable-parallel-tests | -D"NETCDF\_ENABLE_PARALLEL\_TESTS=ON" +Specify Install Location | --prefix=PREFIX | -DCMAKE\_INSTALL\_PREFIX=PREFIX +Enable/Disable netCDF-4 | --enable-netcdf-4
--disable-netcdf-4 | -DENABLE\_NETCDF\_4=ON
-DENABLE\_NETCDF\_4=OFF +Enable/Disable DAP | --enable-dap
--disable-dap | -DENABLE\_DAP=ON
-DENABLE\_DAP=OFF +Enable/Disable Utilities | --enable-utilities
--disable-utilities | -DBUILD\_UTILITIES=ON
-DBUILD\_UTILITIES=OFF +Specify shared/Static Libraries | --enable-shared
--enable-static | -DBUILD\_SHARED\_LIBS=ON
-DBUILD\_SHARED\_LIBS=OFF +Enable/Disable Tests | --enable-testsets
--disable-testsets | -DENABLE\_TESTS=ON
-DENABLE\_TESTS=OFF +Specify a custom library location | Use *CFLAGS* and *LDFLAGS* | -DCMAKE\_PREFIX\_PATH=/usr/custom_libs/ +Enable parallel I/O tests | --enable-parallel-tests | -DNETCDF\_ENABLE_PARALLEL\_TESTS=ON A full list of options can be found by invoking `configure --help`. From 871c5ac24fb69b2e17bc778fb8fdd76badb3beea Mon Sep 17 00:00:00 2001 From: Ed Date: Tue, 13 Aug 2024 08:04:40 -0600 Subject: [PATCH 6/7] adding info about plugins --- docs/building-with-autotools.md | 32 ++++++++++++++++++++++++++------ 1 file changed, 26 insertions(+), 6 deletions(-) diff --git a/docs/building-with-autotools.md b/docs/building-with-autotools.md index e094b2b16b..ffe8ce7669 100644 --- a/docs/building-with-autotools.md +++ b/docs/building-with-autotools.md @@ -3,12 +3,18 @@ Build Instructions for NetCDF-C using autoconf/automake/libtool {#netCDF-autotoo [TOC] -# Overview {#cmake_overview} - -Starting with netCDF-C 4.3.0, we are happy to announce the inclusion of CMake support. CMake will allow for building netCDF on a wider range of platforms, include Microsoft Windows with Visual Studio. CMake support also provides robust unit and regression testing tools. We will also maintain the standard autotools-based build system in parallel. - -In addition to providing new build options for netCDF-C, we will also provide pre-built binary downloads for the shared versions of netCDF for use with Visual Studio. - +# Overview {#autotools_overview} + +Starting with netCDF-C 4.3.0, we are happy to announce the inclusion +of CMake support. CMake will allow for building netCDF on a wider +range of platforms, include Microsoft Windows with Visual Studio. +CMake support also provides robust unit and regression testing tools. +We will also maintain the standard autotools-based build system in +parallel. + +In addition to providing new build options for netCDF-C, we will also +provide pre-built binary downloads for the shared versions of netCDF +for use with Visual Studio. # Requirements {#autotools_requirements} The following packages are required to build netCDF-C. @@ -85,6 +91,20 @@ different parallel I/O job launcher: > $ CC=mpicc CPPFLAGS=-I/usr/local/hdf5-1.14.3_mpich/include LDFLAGS=-L/usr/local/hdf5-1.14.3_mpicj/lib ./configure --prefix=/usr/local/netcdf-c-4.9.3_mpich --enable-parallel-tests --with-mpiexec='srun -A acct_name -q queue_name' +## Setting the HDF5 Plugin Path + +Use the --with-plugin-dir= option to set the HDF5 plugin path. This +path must contain the directory where HDF5 filters have been +installed. --with-plugin-dir accepts three values: + +* yes - use the first directory in environment variable +HDF5_PLUGIN_PATH (if it is set), or /usr/local/hdf5/lib/plugin (Unix) +or ${ALLUSERSPROFILE}\\hdfd5\\lib\\plugin (Windows). + +* a path - path to be used for plugins. + +* no - do not install or use plugins. + ## Checking the Configure Summary At the end of the configure, a summary of the build will be From 2ce551b820b638d81ca5c069324eea20cdf7c137 Mon Sep 17 00:00:00 2001 From: Ed Date: Tue, 13 Aug 2024 08:09:36 -0600 Subject: [PATCH 7/7] adding info about plugins --- docs/building-with-autotools.md | 18 ++++++++---------- docs/building-with-cmake.md | 13 ++++++++++--- 2 files changed, 18 insertions(+), 13 deletions(-) diff --git a/docs/building-with-autotools.md b/docs/building-with-autotools.md index ffe8ce7669..962fd20cbb 100644 --- a/docs/building-with-autotools.md +++ b/docs/building-with-autotools.md @@ -5,16 +5,14 @@ Build Instructions for NetCDF-C using autoconf/automake/libtool {#netCDF-autotoo # Overview {#autotools_overview} -Starting with netCDF-C 4.3.0, we are happy to announce the inclusion -of CMake support. CMake will allow for building netCDF on a wider -range of platforms, include Microsoft Windows with Visual Studio. -CMake support also provides robust unit and regression testing tools. -We will also maintain the standard autotools-based build system in -parallel. - -In addition to providing new build options for netCDF-C, we will also -provide pre-built binary downloads for the shared versions of netCDF -for use with Visual Studio. +A CMake build of netCDF is also supported. Users should consider using +the CMake build, which builds netCDF on a wider range of platforms, +include Microsoft Windows with Visual Studio. The autotools based +build may eventually be retired; users may considering transitioning +to the CMake build. See \ref netCDF-CMake. + +We also provide pre-built binary downloads for the shared versions of +netCDF for use with Visual Studio. # Requirements {#autotools_requirements} The following packages are required to build netCDF-C. diff --git a/docs/building-with-cmake.md b/docs/building-with-cmake.md index 003fed6b49..a47caa301a 100644 --- a/docs/building-with-cmake.md +++ b/docs/building-with-cmake.md @@ -5,9 +5,16 @@ Build Instructions for NetCDF-C using CMake {#netCDF-CMake} # Overview {#cmake_overview} -Starting with netCDF-C 4.3.0, we are happy to announce the inclusion of CMake support. CMake will allow for building netCDF on a wider range of platforms, include Microsoft Windows with Visual Studio. CMake support also provides robust unit and regression testing tools. We will also maintain the standard autotools-based build system in parallel. - -In addition to providing new build options for netCDF-C, we will also provide pre-built binary downloads for the shared versions of netCDF for use with Visual Studio. +Starting with netCDF-C 4.3.0, we are happy to announce the inclusion +of CMake support. CMake will allow for building netCDF on a wider +range of platforms, include Microsoft Windows with Visual Studio. +CMake support also provides robust unit and regression testing tools. +We will also maintain the standard autotools-based build system in +parallel. + +In addition to providing new build options for netCDF-C, we will also +provide pre-built binary downloads for the shared versions of netCDF +for use with Visual Studio. # Requirements {#cmake_requirements}