Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: non positive ttl in slot migration #4

Open
wants to merge 22 commits into
base: unstable
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
62daa73
Add the hrange command (#895)
tanruixiang Sep 20, 2022
01c2d87
Export the rocksdb write options (#885)
tufitko Sep 20, 2022
04f25c8
test: harden quit_test and list_test (#899)
tisonkun Sep 21, 2022
87644ab
Add EVAL_RO (#782)
xiaobiaozhao Sep 21, 2022
e8721a2
Move TCL test unit/scripting to Go case (#900)
tisonkun Sep 21, 2022
399a33d
Support quoted string and inline comment in config (#849)
PragmaTwice Sep 22, 2022
d469155
Fix sanitizer report checking in CI (#912)
PragmaTwice Sep 23, 2022
671cab3
Remove unnecessary Tostring (#909)
tanruixiang Sep 23, 2022
114af54
Add the FilterBlobByKey for SubKeyFilter (#902)
tufitko Sep 23, 2022
2c88a86
Fix config set rocksdb.blob_garbage_collection_age_cutoff (#914)
shangxiaoxiong Sep 23, 2022
afd8668
Avoid using Get when iterating sub-keys during migration (#906)
caipengbo Sep 24, 2022
231d48d
Send SIGTERM after finishing the unit test (#898)
ColinChamber Sep 24, 2022
91cebf2
Disable the TLS by default (#911)
git-hulk Sep 24, 2022
6d8e8f5
Replace `std::stol` with ParseInt (#897)
tanruixiang Sep 25, 2022
9b6cc11
Fix many typos in cluster.cc (#919)
git-hulk Sep 26, 2022
f02029a
fix: non positive ttl in slot migration
ellutionist Sep 20, 2022
5dfee33
check meta.expire greater than zero
ellutionist Sep 20, 2022
469fd84
implement "set exat|pxat" && employ in migration
ellutionist Sep 22, 2022
33121ef
test: cases for EXAT|PXAT && slot migrate
ellutionist Sep 23, 2022
8398e79
use ParseInt
ellutionist Sep 26, 2022
cb6d1d8
fix template misuse
ellutionist Sep 26, 2022
73a27c3
make style more concise
ellutionist Sep 27, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 16 additions & 8 deletions .github/workflows/kvrocks.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -72,55 +72,65 @@ jobs:
- name: Darwin Clang
os: macos-11
compiler: auto
with_openssl: -DENABLE_OPENSSL=ON
- name: Darwin Clang without Jemalloc
os: macos-11
compiler: auto
without_jemalloc: -DDISABLE_JEMALLOC=ON
with_openssl: -DENABLE_OPENSSL=ON
- name: Ubuntu GCC
os: ubuntu-18.04
compiler: gcc
with_openssl: -DENABLE_OPENSSL=ON
- name: Ubuntu Clang
os: ubuntu-18.04
compiler: clang
with_openssl: -DENABLE_OPENSSL=ON
- name: Ubuntu GCC ASan
os: ubuntu-18.04
without_jemalloc: -DDISABLE_JEMALLOC=ON
with_sanitizer: -DENABLE_ASAN=ON
with_openssl: -DENABLE_OPENSSL=ON
compiler: gcc
- name: Ubuntu GCC TSan
os: ubuntu-18.04
without_jemalloc: -DDISABLE_JEMALLOC=ON
with_openssl: -DENABLE_OPENSSL=ON
with_sanitizer: -DENABLE_TSAN=ON
compiler: gcc
runtime_env_vars: TSAN_OPTIONS="suppressions=$(realpath ./tests/tsan-suppressions)"
- name: Ubuntu Clang ASan
os: ubuntu-18.04
with_sanitizer: -DENABLE_ASAN=ON
without_jemalloc: -DDISABLE_JEMALLOC=ON
with_openssl: -DENABLE_OPENSSL=ON
compiler: clang
- name: Ubuntu Clang TSan
os: ubuntu-18.04
with_sanitizer: -DENABLE_TSAN=ON
without_jemalloc: -DDISABLE_JEMALLOC=ON
with_openssl: -DENABLE_OPENSSL=ON
compiler: clang
runtime_env_vars: TSAN_OPTIONS="suppressions=$(realpath ./tests/tsan-suppressions)"
- name: Ubuntu GCC Ninja
os: ubuntu-18.04
with_ninja: --ninja
with_openssl: -DENABLE_OPENSSL=ON
compiler: gcc
- name: Ubuntu GCC Ninja without Jemalloc
os: ubuntu-18.04
with_ninja: --ninja
without_jemalloc: -DDISABLE_JEMALLOC=ON
with_openssl: -DENABLE_OPENSSL=ON
compiler: gcc
- name: Ubuntu GCC without luajit
os: ubuntu-18.04
without_luajit: -DUSE_LUAJIT=OFF
with_openssl: -DENABLE_OPENSSL=ON
compiler: gcc
- name: Ubuntu GCC without OpenSSL
os: ubuntu-18.04
compiler: gcc
without_openssl: -DENABLE_OPENSSL=OFF

runs-on: ${{ matrix.os }}
steps:
Expand Down Expand Up @@ -164,10 +174,8 @@ jobs:

- name: Build Kvrocks
run: |
./x.py build -j$NPROC --unittest --compiler \
${{ matrix.compiler }} ${{ matrix.with_ninja }} ${{ matrix.with_sanitizer }} \
${{ matrix.without_jemalloc }} ${{ matrix.without_openssl }} ${{ matrix.without_luajit }} \
${{ env.CMAKE_EXTRA_DEFS }}
./x.py build -j$NPROC --unittest --compiler ${{ matrix.compiler }} ${{ matrix.without_jemalloc }} ${{ matrix.without_luajit }} \
${{ matrix.with_ninja }} ${{ matrix.with_sanitizer }} ${{ matrix.with_openssl }} ${{ env.CMAKE_EXTRA_DEFS }}

- name: Run Unit Test
run: |
Expand Down Expand Up @@ -196,7 +204,7 @@ jobs:
run: |
export ${{ matrix.runtime_env_vars }}
./x.py test tcl build --cli-path $HOME/local/bin/redis-cli --dont-clean
if [[ -z "${{ matrix.without_openssl }}" ]] && [[ "${{ matrix.os }}" == ubuntu* ]]; then
if [[ -n "${{ matrix.with_openssl }}" ]] && [[ "${{ matrix.os }}" == ubuntu* ]]; then
git clone https://github.com/jsha/minica
cd minica && go build && cd ..
./minica/minica --domains localhost
Expand All @@ -208,11 +216,11 @@ jobs:
cp minica.pem tests/tcl/tests/tls/ca.crt
./x.py test tcl build --tls --single unit/tls --dont-clean
fi
SANITIZER_OUTPUT=$(grep "Sanitizer:" tests/tmp -r || true)
SANITIZER_OUTPUT=$(grep "Sanitizer:" tests/tcl/tests/tmp -r || true)
if [[ $SANITIZER_OUTPUT ]]; then
echo "$SANITIZER_OUTPUT"
echo "\ndetail reports:\n"
cat $(find tests/tmp -iname stderr)
cat $(find tests/tcl/tests/tmp -iname stderr)
echo "sanitizer error was reported, exiting..."
exit 1
fi
Expand Down
2 changes: 1 addition & 1 deletion CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ option(ENABLE_TSAN "enable thread santinizer" OFF)
option(ASAN_WITH_LSAN "enable leak santinizer while address santinizer is enabled" ON)
option(ENABLE_STATIC_LIBSTDCXX "link kvrocks with static library of libstd++ instead of shared library" ON)
option(USE_LUAJIT "use luaJIT instead of lua" ON)
option(ENABLE_OPENSSL "enable openssl to support tls connection" ON)
option(ENABLE_OPENSSL "enable openssl to support tls connection" OFF)

if (CMAKE_VERSION VERSION_GREATER_EQUAL "3.24.0")
cmake_policy(SET CMP0135 NEW)
Expand Down
4 changes: 2 additions & 2 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ RUN apt install -y git gcc g++ make cmake autoconf automake libtool python3 libs
WORKDIR /kvrocks

COPY . .
RUN ./x.py build
RUN ./x.py build -DENABLE_OPENSSL=ON

FROM ubuntu:focal

Expand All @@ -38,5 +38,5 @@ COPY ./kvrocks.conf ./conf/
RUN sed -i -e 's%dir /tmp/kvrocks%dir /var/lib/kvrocks%g' ./conf/kvrocks.conf
VOLUME /var/lib/kvrocks

EXPOSE 6666:6666
EXPOSE 6666:6666
ENTRYPOINT ["./bin/kvrocks", "-c", "./conf/kvrocks.conf"]
7 changes: 7 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,13 @@ $ ./x.py build # `./x.py build -h` to check more options;
# especially, `./x.py build --ghproxy` will fetch dependencies via ghproxy.com.
```

To build with TLS support, you'll need OpenSSL development libraries (e.g. libssl-dev on Debian/Ubuntu) and run:

```shell
$ ./x.py build -DENABLE_OPENSSL=ON
```


### Running kvrocks

```shell
Expand Down
39 changes: 39 additions & 0 deletions kvrocks.conf
Original file line number Diff line number Diff line change
Expand Up @@ -711,5 +711,44 @@ rocksdb.max_bytes_for_level_base 268435456
# Default: 10
rocksdb.max_bytes_for_level_multiplier 10

# If yes, the write will be flushed from the operating system
# buffer cache before the write is considered complete.
# If this flag is enabled, writes will be slower.
# If this flag is disabled, and the machine crashes, some recent
# rites may be lost. Note that if it is just the process that
# crashes (i.e., the machine does not reboot), no writes will be
# lost even if sync==false.
#
# Default: no
rocksdb.write_options.sync no

# If yes, writes will not first go to the write ahead log,
# and the write may get lost after a crash.
#
# Deafult: no
rocksdb.write_options.disable_wal no

# If enabled and we need to wait or sleep for the write request, fails
# immediately.
#
# Default: no
rocksdb.write_options.no_slowdown no

# If enabled, write requests are of lower priority if compaction is
# behind. In this case, no_slowdown = true, the request will be canceled
# immediately. Otherwise, it will be slowed down.
# The slowdown value is determined by RocksDB to guarantee
# it introduces minimum impacts to high priority writes.
#
# Default: no
rocksdb.write_options.low_pri no

# If enabled, this writebatch will maintain the last insert positions of each
# memtable as hints in concurrent write. It can improve write performance
# in concurrent writes if keys in one writebatch are sequential.
#
# Default: no
rocksdb.write_options.memtable_insert_hint_per_batch no

################################ NAMESPACE #####################################
# namespace.test change.me
Loading