Compressed vs Raw Data Analysis
1. Raw Data Sets
This document shows how to apply proper exception and compression settings to PI tags based on the instrument precision. This will reduce the amount of data that needs to be stored and processed for things like trending, analytics, etc. Using the compressed data instead of the raw data for operational analysis will lead to faster performance for applications and end users.
The analysis below was conducted for a period of 29 days using 3 datasets from a customer’s process:
- Tank level
- Tank pressure
- Chill water temperature
Raw Tank Level data for 29 days.

Raw Tank Pressure data for 29 days.
IMG 2

Raw Chiller Temperature data for 29 days.
IMG 3

2. Summary of the Analysis
This summary table shows the results of the analysis setting the proper exception and compression on each measurement’s raw data.
IMG 4

The precision of the measurements was taken from typical industrial instrument specifications. The maximum error, (Raw Value – Linear Interpolation of Compressed Values), is below the precision of the instruments. This means the eliminated data from the raw value signals was noise and added no benefit for storing it. This means the compressed data represents raw data accurately to the precision of the instruments and does not miss any of the true signals in the raw data.
3. Exception and Compression Setting Analysis
The evaluation of the exception and compression settings used the tag tuning spreadsheet that I developed, which mimics the PI System compression and exception behavior. The settings for each tag were tested so that the maximum difference between the original raw value and the interpolated compressed value is less than the precision/repeatability of the instrument. Here is an example of the tag tuning spreadsheet.
IMG 5

4. Tank Pressure
Using the tag tuning spreadsheet to evaluate the correct exception and compression settings for the Tank Pressure tag and then comparing the raw data to interpolated values from the compressed data at the same time stamps as the raw data was conducted. The partial screen capture of this comparison is shown below. The maximum deviation from the any of the raw data values was 0.008 psi, which is well below the instrument precision of 0.01 psi. The resulting compressed data number of values is only 24.4% of the raw data, which will take less storage, less bandwidth for moving between systems, and higher performance on trending and AF Analytics.
IMG 6

Below is a trend comparing the raw and compressed data sets for the 29-day period. The difference is imperceptible.
IMG 7

Below is a trend comparing the raw and compressed data sets for the 1-day period. The difference is imperceptible.
IMG 8

Below is a trend comparing the raw and compressed data sets for the 1-hour period. You can see some differences between the raw and compressed data, but note that each horizontal band is 0.01 psi, which is the precision of the instrument.
IMG 9

5. Tank Level
Using the tag tuning spreadsheet to evaluate the correct exception and compression settings for the Tank Level tag and then comparing the raw data to interpolated values from the compressed data at the same time stamps as the raw data was conducted. The partial screen capture of this comparison is shown below. The maximum deviation from the any of the raw data values was 0.02 ft, which is well below the instrument precision of 0.0208 ft (or ¼ in). The resulting compressed data number of values is only 38.5% of the raw data, which will take less storage, less bandwidth for moving between systems, and higher performance on trending and AF Analytics.
IMG 10

Below is a trend comparing the raw and compressed data sets for the 29-day period.
IMG 11

Below is a trend comparing the raw and compressed data sets for the 1-day period. The difference is imperceptible.
IMG 12

Below is a trend comparing the raw and compressed data sets for the 1-hour period. You can see some differences between the raw and compressed data, but they are negligible.
IMG 13

6. Chiller Temperature
Using the tag tuning spreadsheet to evaluate the correct exception and compression settings for the Tank Pressure tag and then comparing the raw data to interpolated values from the compressed data at the same time stamps as the raw data was conducted. The partial screen capture of this comparison is shown below. The maximum deviation from the any of the raw data values was 0.07 F, which is well below the instrument precision of 0.1 F. The resulting compressed data number of values is only 27% of the raw data, which will take less storage, less bandwidth for moving between systems, and higher performance on trending and AF Analytics.
IMG 14

Below is a trend comparing the raw and compressed data sets for the 29-day period.
IMG 15

Below is a trend comparing the raw and compressed data sets for the 1-day period. The difference is imperceptible.
IMG 16

Below is a trend comparing the raw and compressed data sets for the 1-hour period. Note each horizontal band represents half of the precision of the instrument.
IMG 17

If you would like more information about our services, please fill out the form below and a representative will be in touch with you shortly.