-
Notifications
You must be signed in to change notification settings - Fork 5
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
4 changed files
with
179 additions
and
20 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,45 @@ | ||
| ## Description of protocol | ||
|
|
||
| This is not a complete description, but describes the very basics. Contributions | ||
| are welcome. | ||
|
|
||
| There are two versions of protocol used. One is older and used for legacy purposes, | ||
| and a newer version currently used. | ||
|
|
||
| Both protocols have a header and include at least two fields: type and length. | ||
| All the types and what kind of data it includes are described in | ||
| SFSCommunication.h. The format in the comments are: | ||
|
|
||
| - name:\[bits\], where the bits part describes how many bits is included. Since | ||
| you are most likely running this on modern hardware, these will always be a | ||
| multiple of 8, so think of them as how many bytes it includes (8 for 1 byte, 16 | ||
| for 2 etc.). As for the exact type, it's not described and you must look at the | ||
| implementation code (mostly it's some type of unsigned integer). | ||
| - name:STDSTRING, where the data type is a string | ||
| - name:(N * \[various data\]), represents a vector for various data types | ||
|
|
||
| All multi-byte values are ordered big-endian. | ||
|
|
||
| ### V1 protocol | ||
|
|
||
| The first version of the protocol uses a data type and length fields as its | ||
| header fields, both of which are 32-bit integers (may be both unsigned and | ||
| signed). The length field indicates the length of the data after the header. | ||
|
|
||
| Strings include a 8, 16 or 32-bit integer before the string to indicate the size | ||
| of the string. The string does not end with a NULL byte. | ||
|
|
||
| ### V2 protocol | ||
|
|
||
| In this new version, the length is the sum of the version and data lengths. | ||
| The version field is a 32-byte unsigned integer. | ||
|
|
||
| These are prefixed by "SAU" in SFSCommunication.h, and includes several changes | ||
| to the first protocol: | ||
|
|
||
| - String lengths are always 32-bit unsigned integers. | ||
| - Strings end with a NULL byte. | ||
| - Vectors and other collections include a 32-bit integer for indicating length. | ||
|
|
||
| In addition, the packet types can have multiple versions, which is indicated | ||
| by `version==N`. The default version is 0. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
120 changes: 120 additions & 0 deletions
120
tests/test_suites/GaneshaTests/test_nfs_ganesha_file_corruption_on_master_failover.sh
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,120 @@ | ||
| # | ||
| # To run this test you need to add the following lines to /etc/sudoers.d/saunafstest: | ||
| # | ||
| # saunafstest ALL = NOPASSWD: /bin/mount, /bin/umount, /bin/pkill, /bin/mkdir, /bin/touch | ||
| # saunafstest ALL = NOPASSWD: /usr/bin/ganesha.nfsd | ||
| # | ||
| # The path for the Ganesha daemon should match the installation folder inside the test. | ||
| # | ||
|
|
||
| timeout_set 2 minutes | ||
|
|
||
| CHUNKSERVERS=3 \ | ||
| MOUNT_EXTRA_CONFIG="sfscachemode=NEVER" \ | ||
| CHUNKSERVER_EXTRA_CONFIG="READ_AHEAD_KB = 1024|MAX_READ_BEHIND_KB = 2048" | ||
| setup_local_empty_saunafs info | ||
|
|
||
| test_error_cleanup() { | ||
| cd "${TEMP_DIR}" | ||
| sudo umount -l "${TEMP_DIR}/mnt/ganesha" | ||
| sudo pkill -9 ganesha.nfsd | ||
| } | ||
|
|
||
| # Function to get the checksum of a given file | ||
| get_checksum() { | ||
| sha256sum $1 | awk '{ print $1 }' | ||
| } | ||
|
|
||
| mkdir -p "${TEMP_DIR}/mnt/ganesha" | ||
|
|
||
| # Create PID file for Ganesha | ||
| PID_FILE=/var/run/ganesha/ganesha.pid | ||
| if [ ! -f ${PID_FILE} ]; then | ||
| echo "ganesha.pid doesn't exists, creating it..."; | ||
| sudo mkdir -p /var/run/ganesha; | ||
| sudo touch "${PID_FILE}"; | ||
| else | ||
| echo "ganesha.pid already exists"; | ||
| fi | ||
|
|
||
| cd "${info[mount0]}" | ||
|
|
||
| cat <<EOF > "${info[mount0]}/ganesha.conf" | ||
| NFS_KRB5 { | ||
| Active_krb5=false; | ||
| } | ||
| NFSV4 { | ||
| Grace_Period = 5; | ||
| Lease_Lifetime = 3; | ||
| } | ||
| EXPORT { | ||
| Attr_Expiration_Time = 0; | ||
| Export_Id = 99; | ||
| Path = /; | ||
| Pseudo = /; | ||
| Access_Type = RW; | ||
| FSAL { | ||
| Name = SaunaFS; | ||
| hostname = localhost; | ||
| port = ${saunafs_info_[matocl]}; | ||
| # How often to retry to connect | ||
| io_retries = 5; | ||
| cache_expiration_time_ms = 2500; | ||
| } | ||
| Protocols = 4; | ||
| CLIENT { | ||
| Clients = localhost; | ||
| } | ||
| } | ||
| EOF | ||
|
|
||
| sudo /usr/bin/ganesha.nfsd -f "${info[mount0]}/ganesha.conf" | ||
| assert_eventually 'showmount -e localhost' | ||
|
|
||
| sudo mount -vvvv localhost:/ "${TEMP_DIR}/mnt/ganesha" | ||
|
|
||
| # Create a file for testing with checksum | ||
| head -c 3G /dev/random | tee "${TEMP_DIR}/test_file" > /dev/null | ||
|
|
||
| # Restart master server after 15 seconds | ||
| ( | ||
| sleep 15 | ||
| assert_success saunafs_master_daemon restart | ||
| ) & | ||
|
|
||
| # Wait for Grace period so NFS Ganesha server will be ready | ||
| sleep 5 | ||
|
|
||
| # Try to copy the file after master restart | ||
| while true; do | ||
| cp "${TEMP_DIR}/test_file" "$TEMP_DIR/mnt/ganesha/test_file" && break | ||
| echo "Unable to copy test_file through NFS, retrying in 5 seconds..." | ||
| sleep 5 | ||
| done | ||
|
|
||
| # Get checksums | ||
| checksum1=$(get_checksum "${TEMP_DIR}/test_file") | ||
|
|
||
| # To get the checksum of the file from NFS could require several retries in case | ||
| # NFS mount will not be available after restarting master server | ||
| while true; do | ||
| checksum2=$(get_checksum "${TEMP_DIR}/mnt/ganesha/test_file") | ||
| if [ "${checksum2}" != "" ]; then | ||
| break | ||
| fi | ||
| echo "Unable to get checksums through NFS, retrying in 5 seconds..." | ||
| sleep 5 | ||
| done | ||
|
|
||
| checksum3=$(get_checksum "${info[mount0]}/test_file") | ||
|
|
||
| # Print checksums | ||
| echo "Checksum of original file: ${checksum1}" | ||
| echo "Checksum of file from NFS mount: ${checksum2}" | ||
| echo "Checksum of file from SaunaFS mount: ${checksum3}" | ||
|
|
||
| # Verify checksums | ||
| assert_equals "${checksum1}" "${checksum2}" | ||
| assert_equals "${checksum1}" "${checksum3}" | ||
|
|
||
| test_error_cleanup || true |