ETL Validator is the leading Data Testing solution with capability to compare millions of rows across heterogeneous systems. This post goes in the details of performance benchmark test for ETL Validator with different data volumes.
Test Machine Configuration
Operating System : Windows 10
Processor : Intel Core i7 CPU 3.40 GHZ
RAM : 32 GB
System type : 64-bit
Software in the test machine :
- ETL Validator Client v3.4.7
- ETL Validator Server v3.4.7 (running on tomcat 8 and JDK 1.8 with maximum JDK memory setting of 1 GB)
- PostgreSQL 10.3 database hosting ETL Validator Repository and Workschema
Source and Target Database :
SQL Server database located in close network proximity of the test machine was used both as the Source and Target data source. The SQL Server database was seeded with Adventure Works data.
Test Description:
A Query Compare Test Case was created with the SQL Server as the source and target database. Below is a screenshot of the columns used for the test:
Test 1: For ~1 million Records (156 MB):
Source Data Load : 22 secs
Target Data Load : 23 secs
Data Comparison : 17 secs
Total Run Time : 1 min 2 secs
Test 2: For ~20 million Records (3 GB):
Source Data Load : 4 mins 47 secs
Target Data Load : 4 mins 49 secs
Data Comparison : 5 mins 18 secs
Total Run Time : 14 mins 54 secs
Attached are the screenshots of Test 2 along with the recommended RAM and Cores.