Performance Discrepancy: Velox Accelerates Spark on x86 but Not on ARM

[sachin220594]

Backend

VL (Velox)

Bug description

I conducted a performance comparison of Spark with and without Velox on both x86 and ARM architectures using the TPC-H Query 5. The results indicate that while Velox enhances performance on x86, it does not yield the same improvement on ARM.

Observed Performance Metrics:
x86 Environment:
Without Velox: 109.37 seconds
With Velox: 44.04 seconds
Improvement: ~59.7% reduction in execution time
ARM Environment:
Without Velox: 102.82 seconds
With Velox: 102.96 seconds
Improvement: No significant improvement observed

Spark version

Spark-3.5.x

Spark configurations

--conf spark.memory.offHeap.enabled=true
--conf spark.memory.offHeap.size=12g
--conf spark.gluten.sql.columnar.forceShuffledHashJoin=true
--conf spark.driver.memory=4G
--conf spark.executor.instances=1
--conf spark.executor.memory=30G
--conf spark.executor.cores=16
--conf spark.sql.shuffle.partitions=32
--conf spark.executor.memoryOverhead=2g
--conf spark.driver.maxResultSize=2g

System information

Configuration Details:
Hardware Details:
x86 Environment:
Instance Type: m6i.4xlarge
CPU: Intel Xeon (Ice Lake)
Cores: 16 vCPUs
RAM: 64 GB
ARM Environment:
Instance Type: m7g.4xlarge
CPU: AWS Graviton3
Cores: 16 vCPUs
RAM: 64 GB

Relevant logs

No response