ETL Validator is the leading Data Testing solution with capability to compare millions of rows across heterogeneous systems. This post goes in the details of performance benchmark test for ETL Validator with different data volumes. 

Test Machine Configuration 

Operating System     :   Windows 10

Processor                  :   Intel Core i7 CPU  3.40 GHZ 

RAM                          :   32 GB

System type              :   64-bit

Software in the test machine : 

  • ETL Validator Client v3.4.7
  • ETL Validator Server v3.4.7 (running on tomcat 8 and JDK 1.8 with maximum JDK memory setting of 1 GB)
  • PostgreSQL 10.3 database hosting ETL Validator Repository and Workschema

Source and Target Database : 

SQL Server database located in close network proximity of the test machine was used both as the Source and Target data source. The SQL Server database was seeded with Adventure Works data. 

Test Description: 

A Query Compare Test Case was created with the SQL Server as the source and target database.  Below is a screenshot of the columns used for the test:

Test 1: For ~1 million Records  (156 MB):

            Source Data Load : 22 secs

            Target Data Load  : 23 secs

            Data Comparison  : 17 secs

            Total Run Time    : 1 min 2 secs

Test 2: For ~20 million Records (3 GB):

            Source Data Load : 4 mins 47 secs

            Target Data Load  : 4 mins 49 secs

            Data Comparison  : 5 mins 18 secs

            Total Run Time    : 14 mins 54 secs

Attached are the screenshots of Test 2 along with the recommended RAM and Cores.