🔥 Physics-Informed Unsupervised Learning with Synthetic Data via Recurrent Neural Networks ≘ Data-Driven Model Reduction for Input-Output Systems via Empirical System Gramians

emgr — Empirical Gramian Framework (5.99)

Model Reduction Software

Name

- EMpirical GRamian framework for (nonlinear) input-output systems.

Description

(image) emgr logo

System gramians are matrices associated to linear input-output systems and quantify their controllability, observability and minimality. Empirical system gramian matrices (empirical gramians) are a generalization computable for linear but also for parametric and nonlinear (state-space) systems and have, among others (see below), applications in model order reduction. Model reduction via empirical gramians is applicable to the state-space, to the parameter-space or both, through combined state and parameter reduction. For state reduction, the empirical reachability gramian / empirical controllability gramian and the empirical observability gramian, or the empirical cross gramian (the empirical linear cross gramian for large-scale linear systems) are available. For sensitivity analysis, parameter identification, and parameter reduction, the empirical sensitivity gramian (controllability of parameters), or the empirical identifiability gramian and empirical cross-identifiability gramian (observability of parameters) are provided. Combined reduction is enabled by the empirical joint gramian, which computes minimality of states (via the cross gramian) and observability of parameters (via the cross-identifiability gramian) simultaneously. Furthermore, the empirical output reachability gramian / empirical output controllability gramian is computable. Overall, empirical system Gramians are an advanced data-driven system-theoretic method and a numerical tool for GRAMIAN-based (non)linear input-output system analysis. The empirical gramian framework — emgr — is a compact open-source toolbox, compatible with OCTAVE and MATLAB, that provides a unified interface for unsupervised learning of system-theoretic operators from small data.

Scope

(image) emgr box

Download

Get emgr here: emgr.m (Version: 5.99, CRC32: f76f4638)    [emgr-5.99.zip]    [mirror]    [source]    [meta]    [repo]

(emgr is written in the Matlab programming language and requires Octave >= 5.2 or MATLAB >= 2017b, yet emgr has no dependencies on toolboxes or packages.)

License

All source code is licensed under the open source BSD-2-Clause license:

Copyright (c) 2013–2022, Christian Himpe

All rights reserved.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.

2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Disclaimer

This is research software!

Model

The target mathematical model is a general input-output system (aka control system): ẋ(t) = f(x(t),u(t),p,t) y(t) = g(x(t),u(t),p,t) consisting of a first-order ordinary differential equation (ODE) with vector field f and an algebraic output function g, of which either (may) depend on input u(t), state x(t), parameter p and time t, yielding the output y(t). This model maps an input (control) to an output (quantities of interest) via the state evolving under a dynamical system.

Algorithm

The computation of empirical system Gramians is described in the open-access article:

   C. Himpe; "emgr -- the Empirical Gramian Framework"; Algorithms, 11(7): 91, 2018;

and references therein.

Usage

General Usage: W = emgr(f,g,s,t,w,pr,nf,ut,us,xs,um,xm,dp);

Minimal Usage: W = emgr(f,g,s,t,w);

About Info Usage: v = emgr('version');

Mandatory Arguments

Optional Arguments

(flowchart) gramian interrelations

Gramian Types

(chart) #emgr application matrix

Option Flags

Return Values

Solver Interface

SSP32 Runge-Kutta schematic A custom solver for quantities of interest of an ordinary differential equations is set by a function handle in the global variable: ODE.

Empirical System Theory

The platform est.m prototypically enables testing and evaluating methods based on empirical gramians. est has the following capabilities: For a detailed documentation of capabilities see est.scm.

Minimal Example

A = -0.5*eye(4) % System matrix B = [0;1;0;1] % Input matrix C = [0,0,1,1] % Output matrix WX = emgr(@(x,u,p,t) A*x + B*u, @(x,u,p,t) C*x,[1,4,1],[0.1,10.0],'x') % ≈ B*C

Tests

Tests are defined in emgrTest.m and are evaluated using est. The tests are performed on a time-invariant, linear, state-space symmetric MIMO system and optional linear parametrization: ẋ = A*x + B*u + F*p y = C*x

Demos

The demo codes can be found in estDemo.m and are evaluated using est.

Combined Reduction: Nonlinear System (WJ)

Here a parametrized nonlinear control system is reduced in states and parameters using the empirical joint gramian. The nonlinearity is given by a hyperbolic tangent with parametrized argument. This is a parametrized hyperbolic network model from "Modeling and control based on a new neural network model" by Y. Quan et al (2001). ẋ = A*tanh(p.*x) + B*u y = C*x

(plot) combined_wj reduction #emgr
Relative L2 output error for varying state and parameter dimension.

Benchmark: Inverse Sylvester Procedure (WC+WO)

The inverse Sylvester procedure can be used to generate random linear systems based on "On generating random systems: a gramian approach" by S.C. Smith and J. Fisher (2003). Here, the required solution of the Sylvester equation is accomplished by the empirical gramians after factoring (-1). The generated random system is then reduced by using balanced truncation. 0 = A*WC + WC*A' - B'*B = (-1*a)*WC + WC*(-1*a') - B'*B = a*(-1*WC) + (-1*WC)*a' - B'*B

(plot) benchmark_ilp reduction #emgr
Relative L1, L2, L∞, L0 output error for varying state dimension.

Benchmark: Flexible Space Structures (WX)

A procedural benchmark modelling modal structural dynamics for applications such as truss masts for space environments (i.e.: COFS-1) based on "Model Reduction for Flexible Space Structures" by W. Gawronski and T. Williams (1991). This is a linear second order model, which is reduced by approximate balancing via the empirical cross gramian.

(plot) benchmark_fss #emgr
Relative L1, L2, L∞, L0 output error for varying state dimension.

Benchmark: Nonlinear Model Reduction (WX)

An implementation of a model reduction benchmark from "Model Reduction for Nonlinear Systems" by Y. Chen (1999). Here the nonlinear RC-ladder benchmark, a resistor-capacitor cascade with nonlinear resistors (diodes), is tested. The empirical cross gramian is utilized to reduce this nonlinear system.

(plot) benchmark_non #emgr
Relative L1, L2, L∞, L0 output error for varying state dimension.

Quadratic Output: Kernel Matrix (WX)

The randomly generated example from "Balanced Model Reduction via the Proper Orthogonal Decomposition" by K. Willcox and J. Peraire (2002), is used with a quadratic output functional, and reduced by a empirical cross kernel matrix utilizing a Gaussian kernel. ẋ = A*x + B*u y = x'*x

(plot) reduced quadratic output #emgr
Relative L1, L2, L∞, L0 output error for varying state dimension.

DMD-Galerkin: All-Pass System (WC)

An all pass system based on "Asymptotically Stable All-Pass Transfer Functions: Canonical Form, Parametrization and Realization" by R.J. Ober (1987), which has a single Hankel singular value of multiplicity of the system's order, is reduced in state dimension by the DMD-Galerkin method via the empirical controllability Gramian! ẋ = A*x + B*u y = C*x

(plot) reduced all pass #emgr
Relative L1, L2, L∞, L0 output error for varying state dimension.

PDE Reduction: Advection Equation (WY)

Discretized with a first order upwind finite difference scheme, the 1D linear transport equation is a hyperbolic partial differential equation, whereas the left boundary of the domain is set as input and the right boundary is set as output; as presented in "Efficient Calculation of Thermoacoustic Modes Utilizing State-Space Models" by M. Meindl, T. Emmert and W. Polifke (2016) as flame state-space model. This system is reduced by the empirical linear cross gramian and empirical dominant subspaces. dz/dt = -a dz/dx z(0,t) = u(t) y(t) = z(1,t)

(plot) reduced advection #emgr
Relative L1, L2, L∞, L0 output error for varying state dimension.

Nonlinear Second Order Reduction: 5-body Choreography (WO)

A 5-body choreography, based on the gravitational N-body problem, resembling the figure eight is reduced in states; based on "New Families of Solutions in N-Body Problems" by C. Simo (2001). This is a particularly difficult model to reduce.

(plot) reduced nbody #emgr
Reduced order 5-body figure-8 orbits.

Sensitivity Analysis: Stable Orbits Inside Black Holes (WS)

This is an implementation of: "Is there life inside black holes?" by V. Dokuchaev (2011), describing stable orbits of photons and particles inside the event horizon of a black hole. The underlying system is nonlinear and the output function is a coordinate transformation from Boyer-Lindquist to Cartesian coordinates. In the computation of particle and photon orbits the associated parameters are identified by the empirical sensitivity gramian.

(plot) blackhole orbit #emgr
Quasi-stable orbits inside the event horizon of a black hole.

About

A gram matrix or gramian matrix W is the result of all inner products of a set of vectors V = [v1 ... vn], in other words: W = VT V, and closely related to covariance matrix as well as cross-covariance matrix, correlation matrix or cross-correlation matrix respectively. Properties of (linear) control systems can be assessed by the system gramians, which are based on the controllability and observability operators. Classically, the controllability gramian and observability gramian are utilized in balancing model reduction methods. The cross gramian is not gramian matrix, but encodes controllability and observability information, and thus minimality, into a single matrix. Overall, system gramians quantifiy the system-theoretic properties controllability (reachability), observability and (structural) identifiability.

Empirical gramians extend this approach to nonlinear control systems and thus enable nonlinear model reduction. For linear systems, these empirical system gramians are equal to the classic system gramians, yet, empirical gramians are also computable for parametric and nonlinear systems. Furthermore, empirical gramians contain more information about the underlying system; and particularly the empirical cross gramian conveys even additional information. This makes empirical system gramians a versatile tool for mathematical system theory, control theory or computational science and engineering (CSE).

The (discrete) empirical cross gramian encloses information on the input-output behavior of the associated control system as well as approximate Hankel Singular Values and can be computed very efficiently. For large-scale linear systems, the linear empirical cross gramian, related to balanced POD, can be utilized. And for parametric systems, the (empirical) joint gramian, derived from the cross gramian, is available for combined reduction. In case of custom training input, instead of an empirical gramian matrix, an empirical covariance matrix can also be computed.

Model order reduction or short model reduction is an active research field in scientific computing and CSE, which investigates the algorithmic construction of low-order surrogate models for high-dimensional differential equation models. The resulting reduced order models allow a substantially more economical evaluation in terms of computational or memory resources compared to the original large-scale full order model. This numerical model reduction is essential, as developments in computing technology are constantly outpaced by the requirements for model simulation in dimensionality or complexity.

A primary application of empirical Gramians is data-driven model reduction of nonlinear input-output systems. In the language of data science, a nonlinear input-output system corresponds to a recurrent neural network, i.e. from (data-driven) system identification, which can also be (empirical) Gramian-based, or mathematical modeling. Empirical-Gramian-based model reduction then translates to unsupervised learning, where the empirical Gramians encode the network's reducibility based on dynamic behavior induced by systematic perturbations, correlating to synthetic data, and based on a model making it physics-informed.

References

Contact

Send feedback to: ch@gramian.de

Cite

gramian.de quick response code

Notes

Troubleshooting

Frequently Asked Questions

See Also

Category

(plot) cross gramian surface #emgr

tl;dr

emgr: Unsupervised learning of I/O system properties for data-driven control.

Λ