It's 8 am in the morning and there is an escalation from one of the business users that data is missing in their Tableau report because of a data refresh issue. IT teams supporting Tableau frequently run into these issues. They want to proactively check if their reports are working properly on a periodic basis so that they can identify the issue even before someone from the business team notifies them. BI Validator's Report test plan with "data" option can be used to check that the Tableau views/worksheets are working properly and returning data. When the test plan is executed in 'Benchmark' mode, the test is automatically marked as "Error" if there is no data for a worksheet. Additionally, BI Validator automatically captures a screenshot of the error. 


Although the primary intent of the BI Validator Report test plan is to regression test the reports for any changes in report layout, data, or performance, we will be using only the 'Reports' step of the test plan to 'Benchmark' reports and automatically identify any reports that are missing data. Below are the steps for achieving this. 


Step 1: Create a new Report test plan by clicking on the '+' icon in the top menu and select your Tableau BI connection.  Navigate to the 'Options' step and change the 'Row Count' value from the default of 10000 to 5. This property is used by BI Validator to determine the number of rows to save when downloading data for each worksheet. For the purpose of this use case, the number of rows is not important. We just want to make sure that data is returned by each worksheet. Reducing the number of rows will improve the performance of the test plan. Click on the 'Save' button to save the selections and to navigate to the Reports step. 




Step 2: Select 'Data' option as the 'Default Format' since we are primarily interested in verifying that data is returned by the worksheets. Add the workbook or views to the test plan, by either clicking the search icon or by dragging and dropping them from the connection tree in the 'Connection Explorer' on the right side.

  • BI Validator automatically takes a snapshot of the data for views added. If there will be any tableau dashboards having worksheets, BI Validator will get data for each of the worksheets.  with empty data, then the status will be showing marked as "Error".
  • Expand the view, by clicking on "+" on the left side and click on the csv icon button of the worksheet. It will show a popup message like "No Data available" in case of "Error".
  • BI Validator captures a screenshot of the dashboard which can be very helpful in diagnosing the issue whenever there is an error. Please refer to this article on how to check the error :
    Capturing screenshot of the error in the Report Test Plan



Step 3: The views added to the Report test plan can be re-benchmarked by clicking on the 'Benchmark' icon as shown below. All the selected views will automatically be refreshed with the latest snapshot of the report data (and errors where applicable). 




Scheduling: To schedule the 'Benchmark' of the Report test plan, select the 'Is Benchmark Schedule' option so that the test plan will refresh the benchmark and capture any missing data errors. 


Optional: 


Sometimes having empty data in a worksheet output may not be an "Error". If you prefer to show the status as "Warning", in case of an empty worksheet, change the property "EmptyDataAsWarning". 


  • Go to  Admin Options >> Config Settings and change the property "EmptyDataAsWarning" value to true and save settings.
  • Go back to the test plan and benchmark again, it will be showing the status as "Warning" for the views having worksheet with.


Roadmap: 


We are excited to say that we are working on a Machine Learning algorithm to check for the correctness of the data and identify any outliers dynamically based on the previous snapshot of the data.