Adaptive Single Parameter Total Variation Regularization for Derivative Estimation

Brief: An adaptive single parameter total variation regularization (ASP-TVR) was developed for accurate and faster estimation of derivatives from noisy data to support the development of physics-informed machine learning methods.

Accomplishment: An ASP-TVR was developed for accurate and faster estimation of temporal derivatives of noisy time series data with varying frequency content.  The accurate computation of these derivatives is necessary for learning the physics of dynamic systems from time series data with 1% mean noise.  The performance of the ASP-TVR shows significant improvement in accuracy when compared against spline fitting, total variation regularization (TVR) [1], and single parameter TVR (SP-TVR) [2] by 92%, 57% and 32% respectively.  The proposed ASP-TVR overcomes the limitation of SP-TVR for these systems while increasing accuracy and speed of estimation.  The reduction of computation time by 5% saves significant resources when this technique is used in iterative machine learning algorithms.  

Specific accomplishments are: 

• The ASP-TVR was developed to accurately estimate derivatives of time series data containing varying frequency content.
• A time-frequency analysis approach was integrated with the SP-TVR method to allow determination of the adaptive parameter, by using windowed frequency content and amplitude threshold at the appropriate scale (higher scales).
• Tests with benchmark dynamic system data indicate the ASP TVR gives a ~30% improvement in accuracy with a modest (5%) improvement in computation speed when compared to SP-TVR method.

Description:  The real advantage of the ASP-TVR is with data containing time-varying frequency content, which is common in many real-world dynamic systems.  In these systems, the use of conventional SP-TVR method results in lower estimation accuracy and higher computation time.  The ASP-TVR determines the adaptive parameter using time-frequency analysis and computing windowed frequency information. The time-frequency analysis allows the selection of the proper scale for applying the amplitude threshold necessary in the ASP-TVR.  TVR uses scaling parameter (alpha)=1x10-9 and optimized derivative order=1 (trial and error-based determination by minimizing spikes in derivative).  SP-TVR uses Max frequency = 25 Hz, determined based on maximum frequency content in signal determined using Fourier transforms of entire data.  Sample rate (dt) considered for data generation 0.02.  Sparse regression library polynomial order 2.  The use of the ASP-TVR for computing time derivatives, in combination with the sparse regression method [3], increases the accuracy of the learned attractor dynamics for a benchmark dynamic system (Lorenz system).