Skip to content
This repository has been archived by the owner on Mar 31, 2023. It is now read-only.

Fix the comparison results of SQL query vs scan query document #717

Open
wants to merge 40 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
9645772
Documentation update
pkommoju May 12, 2021
183c36a
Apply changes suggested in review
pkommoju May 22, 2021
4896515
Merge branch 'master' of https://github.com/futurewei-cloud/alcor
pkommoju May 26, 2021
f170e1e
Merge branch 'master' of https://github.com/futurewei-cloud/alcor
pkommoju May 28, 2021
bcbf7b1
Merge branch 'master' of https://github.com/futurewei-cloud/alcor
pkommoju May 29, 2021
efc5b6e
Merge branch 'master' of https://github.com/futurewei-cloud/alcor
pkommoju Jun 1, 2021
c785697
Merge branch 'master' of https://github.com/futurewei-cloud/alcor
pkommoju Jun 8, 2021
d213c03
Merge branch 'master' of https://github.com/pkommoju/alcor
Jun 8, 2021
0f56454
Merge branch 'master' of https://github.com/futurewei-cloud/alcor
pkommoju Jun 8, 2021
4a2a382
Merge branch 'master' of https://github.com/pkommoju/alcor
pkommoju Jun 9, 2021
7f93fe3
Merge branch 'master' of https://github.com/futurewei-cloud/alcor
pkommoju Jun 10, 2021
8d6dd0e
Merge branch 'master' of https://github.com/futurewei-cloud/alcor
pkommoju Jun 15, 2021
b19d428
Merge branch 'master' of https://github.com/futurewei-cloud/alcor
kevin-zhonghao Jun 16, 2021
539d0c0
Merge branch 'master' of https://github.com/futurewei-cloud/alcor
pkommoju Jun 22, 2021
6a9a1c6
Merge branch 'master' of https://github.com/pkommoju/alcor
pkommoju Jun 22, 2021
19430b2
Merge branch 'master' of https://github.com/futurewei-cloud/alcor
pkommoju Jul 1, 2021
1d7ef7e
Merge branch 'master' of https://github.com/futurewei-cloud/alcor
pkommoju Jul 2, 2021
cd1fec6
Merge branch 'master' of https://github.com/futurewei-cloud/alcor
pkommoju Jul 6, 2021
e57abe4
Merge branch 'master' of https://github.com/futurewei-cloud/alcor
pkommoju Jul 20, 2021
1b63054
Add missing file
pkommoju Jul 21, 2021
fe81008
Merge branch 'master' of https://github.com/futurewei-cloud/alcor
pkommoju Jul 21, 2021
118dd53
Merge branch 'master' of https://github.com/futurewei-cloud/alcor
pkommoju Jul 23, 2021
7d9a2e5
Merge branch 'master' of https://github.com/futurewei-cloud/alcor
pkommoju Aug 2, 2021
3884a44
Merge branch 'master' of https://github.com/futurewei-cloud/alcor
pkommoju Aug 4, 2021
0b5cd11
Merge branch 'master' of https://github.com/futurewei-cloud/alcor
pkommoju Aug 6, 2021
3f094be
Merge branch 'master' of https://github.com/futurewei-cloud/alcor
pkommoju Aug 23, 2021
bb777ec
Merge branch 'master' of https://github.com/futurewei-cloud/alcor
pkommoju Sep 10, 2021
c0e8c2d
Merge branch 'master' of https://github.com/futurewei-cloud/alcor
pkommoju Sep 24, 2021
206abc7
Merge branch 'master' of https://github.com/futurewei-cloud/alcor
pkommoju Oct 8, 2021
fab8a00
Merge branch 'master' of https://github.com/futurewei-cloud/alcor
pkommoju Oct 22, 2021
69268a3
Merge branch 'master' of https://github.com/futurewei-cloud/alcor
pkommoju Oct 29, 2021
335b9b8
Merge branch 'master' of https://github.com/futurewei-cloud/alcor
pkommoju Nov 5, 2021
6a95d98
Merge branch 'master' of https://github.com/futurewei-cloud/alcor
pkommoju Nov 29, 2021
a8fce0e
Merge branch 'master' of https://github.com/futurewei-cloud/alcor
pkommoju Nov 29, 2021
884412b
Merge branch 'master' of https://github.com/futurewei-cloud/alcor
pkommoju Dec 2, 2021
ea547f7
Merge branch 'master' of https://github.com/futurewei-cloud/alcor
pkommoju Dec 3, 2021
3e5b025
Merge branch 'master' of https://github.com/futurewei-cloud/alcor
pkommoju Dec 6, 2021
c7a29f1
Merge branch 'master' of https://github.com/futurewei-cloud/alcor
pkommoju Dec 13, 2021
f97e2d0
Fix the performance measurments
pkommoju Dec 14, 2021
fa90f5e
Fix more descripancies in SQL and Scan performance results
pkommoju Dec 14, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file modified docs/modules/ROOT/images/sql_and_scan_query_perf_comp.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
Expand Up @@ -13,13 +13,15 @@ Ignite supports secondary keys, called QuerySqlFields which will use a tradional
This report is about comparing the performance of the queries on non-key fields using Ignite's SCAN query and through SQL queries.

== Test setup
All times are in micro seconds. Time axis (y) in the histogram is in logscale.

Operations are on NodeInfo object. Benchmark tool, DPM, NCM and NMM run on the same physical machine in the lab. Ignite is run on two different physical machines in the lab (partitioned).
Number of entries in both SQL and Scan are 1M, 1000 entries are queried on.

The first query always takes way too much time compared to the subsequent
ones and hence ignored when computing the statistics.
Number of entries in both SQL and Scan are 1M.

All times are in micro seconds. Time axis (y) in the histogram is in logscale.
1000 queries are run but the performance statistics are only for 999
queries, ignoring the first one which always takes orders of
magnitude longer.

SQL_QUERY_EXEC time represents time required to execute the SQLFieldsQuery
(cache.query() API).
Expand All @@ -33,19 +35,23 @@ instantiation, cache.withKeepBinary().query(...)
SCAN_CURSOR_FETCH time represents time required to extract the result from
result set (cursor, cursor.getAll() API).

For Insert operation, MAX represents total time to insert 1M entires into the cache and AVG represents the average time to insert one single entry. All other statistcs for Insert are not relevant and are shown as zero.
Total time is applicable only to insert operations therefore query execution
and fetch times show "N/A" for these operations.

For Insert operation, only total time for million insertions and average for
signle insert is relevant therefore other times are set to "N/A" (Not
Applicable).

== Results
|===
|OPERATION TYPE| MIN| MAX| AVG| MEDIAN| MODE| P99
|SQL_QUERY_EXEC| 1.0000| 16.0000| 1.8739| 1.0000| 1.0000| 1.0000
|SCAN_QUERY_EXEC| 530182.0000| 785735.0000| 561550.7678| 552705.0000| 543704.0000| 531308.3565
|SQL_CURSOR_FETCH| 0.0000| 1.0000| 0.6717| 1.0000| 1.0000| 0.0000
|SCAN_CURSOR_FETCH| 0.0000 | 8.0000 | 0.1411| 0.0000| 0.0000| 0.0000
|SQL_INSERT| 0.0000 | 929676278.00| 929.6762| 0.0| 0.0| 0.0
|SCAN_INSERT| 0.0000| 812451365.00| 812.4514| 0.0| 0.0| 0.0
|OPERATION_TYPE|MIN |MAX |AVG |TOTAL |MEDIAN |MODE |P99
|SQL_EXEC | 1.0000| 16.0000| 1.8739|N/A | 1.0000| 1.0000| 7.0000
|SCAN_EXEC |530,182.0000|785,735.0000|561,550.0000|N/A |552,705.0000|543,704.0000|667,240.0000
|SQL_FECTH | 0.0000| 3.0000| 0.7537|N/A | 1.0000| 1.0000| 1.0000
|SCAN_FETCH | 0.0000| 8.0000| 0.1411|N/A | 0.0000| 0.0000| 3.0000
|SQL_INSERT | 0.0000|N/A | 929.6800|929,676,278.0000|N/A |N/A |N/A
|SCAN_INSERT | 0.0000|N/A | 812.4500|812,451,365.0000|N/A |N/A |N/A
|===

== Plot of the results
image::sql_and_scan_query_perf_comp.png[]

Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
OPERATION TYPE MIN MAX AVG MEDIAN MODE P99
SQL_QUERY_EXEC 1.0000 16.0000 1.8739 1.0000 1.0000 1.0000
SCAN_QUERY_EXEC 530182.0000 785735.0000 561550.7678 552705.0000 543704.0000 531308.3565
SQL_CURSOR_FETCH 0.0000 1.0000 0.6717 1.0000 1.0000 0.0000
SCAN_CURSOR_FETCH 0.0000 8.0000 0.1411 0.0000 0.0000 0.0000
SQL_INSERT 0.0000 929676278.00 929.676278 0.0 0.0 0.0
SCAN_INSERT 0.0000 812451365.00 812.451365 0.0 0.0 0.0
OPERATION_TYPE MIN MAX AVG TOTAL MEDIAN MODE P99
SQL_EXEC 1.0000 16.0000 1.8739 N/A 1.0000 1.0000 7.0000
SCAN_EXEC 530182.0000 785735.0000 561550.0000 N/A 552705.0000 543704.0000 667240.0000
SQL_FECTH 0.0000 3.0000 0.7537 N/A 1.0000 1.0000 1.0000
SCAN_FETCH 0.0000 8.0000 0.1411 N/A 0.0000 0.0000 3.0000
SQL_INSERT 0.0000 N/A 929.6800 929676278.0000 N/A N/A N/A
SCAN_INSERT 0.0000 N/A 812.4500 812451365.0000 N/A N/A N/A
pkommoju marked this conversation as resolved.
Show resolved Hide resolved
7 changes: 4 additions & 3 deletions scripts/sql_and_scan_query_perf_comp.gps
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,9 @@ set output 'sql_and_scan_query_perf_comp.png'
set style fill solid 1.00 border lt -1
set key fixed right top vertical Right noreverse noenhanced autotitle nobox
set style histogram clustered gap 1 title textcolor lt -1
set datafile missing '-'
# set datafile separator "|"
set datafile missing 'N/A'
set decimalsign locale
set decimalsign "."
set grid
show grid
set style data histograms
Expand All @@ -18,4 +19,4 @@ set ylabel "Time in Micro seconds"
set logscale y
NO_ANIMATION=1

plot newhistogram "", 'sql_and_scan_query_perf_comp.csv' using 2:xtic(1) ti "min", '' u 3:xtic(1) ti "max", '' u 4:xtic(1) ti "avg", '' u 5:xtic(1) ti "median", '' u 6:xtic(1) ti "P95", '' u 7:xtic(1) ti "P99"
plot newhistogram "", 'sql_and_scan_query_perf_comp.csv' using 2:xtic(1) ti "min", '' u 3:xtic(1) ti "max", '' u 4:xtic(1) ti "avg", '' u 5:xtic(1) ti "total", '' u 6:xtic(1) ti "median", '' u 7:xtic(1) ti "P99"