Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize write metric update for insertTablets and insertRelationalTablet #13793

Merged
merged 3 commits into from
Oct 17, 2024

Conversation

HTHou
Copy link
Contributor

@HTHou HTHou commented Oct 16, 2024

Description

In this PR, we can update write metrics for tablet and tablets in batch.

Benchmark configuration

########### Test Mode ###########
BENCHMARK_WORK_MODE=testWithDefaultPath
IoTDB_DIALECT_MODE=TABLE
########### Database Connection Information ###########
DOUBLE_WRITE=false
DBConfig=
  DB_SWITCH=IoTDB-200-SESSION_BY_TABLET
  HOST=[127.0.0.1]
########### Data Mode ###########
GROUP_NUMBER=1
IoTDB_TABLE_NUMBER=1
DEVICE_NUMBER=10000
REAL_INSERT_RATE=1.0
SENSOR_NUMBER=10
IS_SENSOR_TS_ALIGNMENT=true
IS_OUT_OF_ORDER=true
OUT_OF_ORDER_RATIO=0.5
########### Data Amount ###########
OPERATION_PROPORTION=1:0:0:0:0:0:0:0:0:0:0:0
CLIENT_NUMBER=10
LOOP=10000
BATCH_SIZE_PER_WRITE=1
DEVICE_NUM_PER_WRITE=1000
START_TIME=2024-01-01T00:00:00+08:00
POINT_STEP=1000
OP_MIN_INTERVAL=0
OP_MIN_INTERVAL_RANDOM=false
INSERT_DATATYPE_PROPORTION=1:1:1:1:1:1:1:1:1:1
ENCODINGS=RLE/TS_2DIFF/TS_2DIFF/GORILLA/GORILLA/DICTIONARY/PLAIN/PLAIN/TS_2DIFF/TS_2DIFF
COMPRESSOR=LZ4
########### Other Param ###########
IS_DELETE_DATA=true
CREATE_SCHEMA=true
BENCHMARK_CLUSTER=false
TEMPLATE=true
VECTOR=true

Before this PR.

----------------------------------------------------------Result Matrix----------------------------------------------------------
Operation                okOperation              okPoint                  failOperation            failPoint                throughput(point/s)      
INGESTION                100000                   1100000000               0                        0                        4128115.64    

After then

----------------------------------------------------------Result Matrix----------------------------------------------------------
Operation                okOperation              okPoint                  failOperation            failPoint                throughput(point/s)      
INGESTION                100000                   1100000000               0                        0                        5116482.89 

Copy link

sonarcloud bot commented Oct 16, 2024

Copy link

codecov bot commented Oct 16, 2024

Codecov Report

Attention: Patch coverage is 46.47887% with 38 lines in your changes missing coverage. Please review.

Project coverage is 40.83%. Comparing base (6510fb8) to head (fc51446).
Report is 8 commits behind head on master.

Files with missing lines Patch % Lines
.../iotdb/db/storageengine/dataregion/DataRegion.java 41.66% 35 Missing ⚠️
.../statemachine/dataregion/DataExecutionVisitor.java 0.00% 3 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master   #13793      +/-   ##
============================================
- Coverage     40.83%   40.83%   -0.01%     
  Complexity       71       71              
============================================
  Files          4057     4061       +4     
  Lines        254110   254675     +565     
  Branches      31030    31087      +57     
============================================
+ Hits         103771   103988     +217     
- Misses       150339   150687     +348     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@jt2594838 jt2594838 merged commit 47e928d into master Oct 17, 2024
39 of 40 checks passed
@HTHou HTHou deleted the optimize_tablets_metric branch October 17, 2024 02:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants