Data Validation

vali::tool automatically detects, marks and (optionally) corrects untrustworthy data, not by using mean average - it detects outliers, noise and checks for discontinuous data. It ensures only high quality data are fed into the event detection module. It also provides the user with indications on sensor maintenance requirements, as well as automatic detection of malfunctions.

 

Why is Data Validation before Event Detection important?

Automatic data validation makes sure that only unmarked, “clean” data are used for further analysis, training and alarms. Any non-event-related deviating data must be identified and marked before feeding them into the following event detection module.

How does vali::tool work?

The basic steps in the data validation are: outlier detection, noise detection and check for discontinuous data*. The results of the data validation are presented as status information with the respective parameter and sensor. A station status symbol as well as a change in background color in the parameter display indicate that data quality is sub-optimal. Detailed notifications, including suggestions to remedy the issue or for maintenance, can be called up.

 

*Step detection is not yet implemented, but will be in future versions.

 vali::tool - Highlights

    • Provides self-adaptive, self-controlled data validation in real time
    • Ensures both sensitive and reliable alarm limits respectively setpoints for process control
    • Analyzes noise, outliers and other combinations in real time to reliably detect any malfunction at an early stage
    • Considers user interventions in real-time
    • Application-specific training period considers normal fluctuations of individual water matrix and typical process dynamics
    • Helps to dramatically reduce false alarm rates
    • ­Configurable auto-correction of data based on threshold, outlier and noise analysis