[1] Measuring Statistical Dependencies in a Time Series (169kb)
[2] On the Concept of the Generalized Mutual Information Function and Efficient Algorithms for Calculating It (206kb)
[3] A Tool to Measure Dependencies in Data Sequences (1.1Mb)
[4] Ranking and Entropy Estimation in Nonlinear Time Series Analysis (650kb)
[5] Detecting Bifurcations in Voice Signals (934kb)
[6] Nonlinear Analysis of the Cardiorespiratory Coordination of a Newborn Piglet (2.1Mb)
[7] Nonlinear Analysis of Perceptual Motor-Coupling in the Development of Postual Control (284kb)
[8] Human Postual Control - Force Plate Experiments and Modelling (1.0Mb)
[9] Die Messung von Informationsflüssen mit einer verallgemeinerten Transinformation (1.3Mb)
[10] Die Spracherzeugung beim Menschen. Begleittext zum Kolloquium zur Habilitation 21. Oktober 1997 (4.7Mb)
[11] Using Mutual Information to Measure Coupling in the Cardiorespiratory System
[12] Detecting Spatio-Temporal Information Flow in the Cortex by Mutual Information Analysis of MEG Data
[13] Transinformationsanalyse in der digitalen Signalverarbeitung
[14] Ein Verfahren zur mathematischen Abstandsbestimmung von Diskurspartikeln: Form-Funktions-Korrelation
[15] Mutual Information And Relevant Variables for Predictions
[16] Permutation entropy - a natural complexity measure for time series
[17] Entropy of interval maps via permutations
[18] Mutual Information Function Assesses Autonomic Information Flow of Heart Rate Dynamics at Different Time Scales
[19] Permutation entropy improves fetal behavioural state classification based on heart rate analysis from biomagnetic recordings in near term fetuses
[20] Analysis of complex physiological systems by information flow: a time scale--specific complexity assessment
[21] Partial Mutual Information for Coupling Analysis of Multivariate Time Series

[1]

A Tool to Measure Dependencies in Data Sequences
( jourstat.ps 0.5Mb ) ( jourstat.ps.Z 0.2Mb )
Abstract

We propose two methods to measure all (linear and nonlinear) statistical dependences in a stationary time series. Presuming ergodicity, the measures can be obtained from efficient numerical algorithms.

Reference:
Pompe B. (1993): J. Stat. Phys. 73, 587-610

[2]

On the Concept of the Generalized Mutual Information Function and Efficient Algorithms for Calculating It
( mutual.ps 0.7Mb ) ( mutual.ps.Z 0.3Mb )
Abstract

We discuss the concept of the generalized mutual information (GMI) and precisely describe efficient algorithms for the estimation of the GMI function (GMIF) for a time series. The GMIF can be considered as an alternative to the autocorrelation function. However, there are significant differences, because the GMIF measures not only linear but also nonlinear statistical dependences. Moreover, it is invariant with respect to any monotonic distortions of the series.

Reference:
Pompe B. and Heilfort M. (1995): (unpublished)

[3]

A Tool to Measure Dependencies in Data Sequences
( oxford.ps 11.8Mb ) ( oxford.ps.Z 1.4Mb )
Abstract

We present a general mathematical tool to measure statistical dependencies in data sequences. The method is based on a quantity called generalized mutual information (GMI). There are two essentials characterizing the proposed method: 1. The GMI considers all (linear and nonlinear) dependencies on a given quantization level; 2. There are efficient algorithms to estimate the GMI from finite data sequences.
Our method can be applied, for instance, to an ergodic time series leading to the so-called auto GMI function which can be considered as a nonlinear generalization of the well-known autocorrelation function. Cross dependencies between two or more data sequences can be investigated as well. In any case, there must be at least 1000 data in the sequence, and it has to be almost continuous in amplitude (say not less than 256 quantization steps).
In this paper we define the GMI and then summarize some of its properties and interpretations. Moreover, we sketch a fast estimation algorithm. For illustration we apply our method to some time series originating from different sources.

Reference:
Pompe B. (1996): A Tool to Measure Dependencies in Data Sequences. in: P. Ciarlini, M. G. Cox, F. Pavese, and D. Richter (eds.) (1996): Advanced Mathematical Tools in Metrology II (World Scientific Publishing Company, Singapore) 1996, pp. 81-99

[4]

Ranking and Entropy Estimation in Nonlinear Time Series Analysis
( ntptsa.pdf 0.44Mb ) ( ntptsa.zip 0.31Mb )
Abstract

This chapter is concerned with two subjects. The first one is a method of signal preprocessing called ranking. It is of special relevance in nonlinear time series analysis and may cause several computational advantages. The second subject is on the definition and estimation of a generalized mutual information which is useful to analyse statistical dependences in scalar or multivariate time series. A fast algorithm for its estimation is described in detail which essentially profits from ranking of the scalar components of the time series.

Reference:
Pompe B. (1998): (in: H. Kantz, J. Kurths, and G. Mayer-Kress (eds.): Nonlinear Analysis of Physiological Data, (Springer, Berlin, 1998) )

[5]

Detecting Bifurcations in Voice Signals
( hans.ps 3.4Mb ) ( hans.ps.Z 1.2Mb )
Abstract

This chapter is concerned with the detection of bifurcations in voice signals applying several techniques of sliding signal analysis - conventional ones as well as novel methods originating from nonlinear dynamics. The signals come from several models (two-mass and continuum) as well as from an excised larynx experiment and vocalizations of patients with voice disorders. The results of the different techniques were found to be consistent and complementary to each other.

Reference:
Herzel H., Holzfuss J., Kowalik Z. J., Pompe B., Reuter R. (1996): NCVS Status and Progress Report 9, 99-107
(also in: H. Kantz, J. Kurths, and G. Mayer-Kress (eds.): Nonlinear Analysis of Physiological Data, (Springer, Berlin, 1998) )

[6]

Nonlinear Analysis of the Cardiorespiratory Coordination of a Newborn Piglet
( dirk.ps 9.1Mb ) ( dirk.ps.Z 2.3Mb )
Abstract

We investigate the cardiorespiratory system of a newborn piglet during REM and NON-REM sleep as well as general anesthesia, hypoxia, and cholinergic blockade. The coordinated behavior of heart rate fluctuation and respiratory movement reflects essential capabilities of the autonomic coordination. A corresponding multivariate data analysis was done by means of several nonlinear methods: generalized mutual information, redundancy and surrogate data, window pattern entropy, and computation of phase relations. Some of them are applied for the first time in this context.

Reference:
Hoyer D., Bauer R., Pompe B., Palus M., Zebrowski J. J., Rosenblum M., Zwiener U. (1998): (in: H. Kantz, J. Kurths, and G. Mayer-Kress (eds.): Nonlinear Analysis of Physiological Data, (Springer, Berlin, 1998) )

[7]

Nonlinear Analysis of Perceptual Motor-Coupling in the Development of Postual Control
( steven.ps 2.8Mb ) ( steven.ps.Z 0.4Mb )
Abstract

Human bipedal stance requires a control mechanism which can maintain upright posture as well as adopt quickly and flexibly to chnages in the environment. Some sort of dynamical control must link visual, auditory, and proprioceptive perceptual input to the motoric responces required to activate appropriate muscle groups in order to maintain balance. This dynamical control mechanism needs the use of perceptual input to predict the future state of posture with respect to the environment if adaptive balance is to be maintained under changing conditions. These constraints suggest that the purely stochastic random-walk postural control system is unlikely, although others have been unable to reject a linear stochastic model for postural control of quiet standing. The data presented in this chapter are drawn from an experiment that measures center of pressure in a sample of sitting infants who are exposed to a "moving room" stimulus paradigm. Three categories of analyses are applied to these data: mutual information, false nearest neighbors and surrogate data tests. These techniques will be applied to ask whether the center of pressure in sitting infants' posture control can be modeled as a linear system, whether there is a developmental change in the strength and direction of the coupling of that postural control to visual stimuli and whether any developmental change observed in this coupling carries an accompanying reduction in noise in center of pressure.

Reference:
Boker S. M., Schreiber T., Pompe B., Bertenthal B. I. (1998): (in: H. Kantz, J. Kurths, and G. Mayer-Kress (eds.): Nonlinear Analysis of Physiological Data, (Springer, Berlin, 1998) )

[8]

Human Postual Control - Force Plate Experiments and Modelling
( misha.ps 7.5Mb ) ( misha.ps.Z 1.6Mb )
Abstract

We report the results of time series analysis of human body sway while quiet upright stance. The bivariate records (stabilograms) are measured by means of a force plate. To investigate interrelations between oscillations in anterior-posterior and lateral directions we use several techniques: cross-spectrum analysis, generalized mutual information, and calculation of instantaneous relative phase. We find that the stabilograms can be qualitatively rated into two groups: noisy and oscillatory patterns. Further, we show that oscillatory patterns may demonstrate phase locking. We argue that these patterns are due to stochastic and chaotic dynamics, respectively. We discuss the plausible strategy of postural control and present the model that qualitatively describes transitions from noisy to oscillatory patterns and phase synchronization. The relevance of the results of the time series analysis for the diagnostics of neurological pathologies is discussed.

Reference:
Rosenblum M., Firsov G., Kuuz R., and Pompe B. (1996): (in: H. Kantz, J. Kurths, and G. Mayer-Kress (eds.): Nonlinear Analysis of Physiological Data, (Springer, Berlin, 1998) )

[9]

Die Messung von Informationsflüssen mit einer verallgemeinerten Transinformation
( hab.ps 18.1Mb ) ( hab.ps.Z 2.0Mb )
Abstract

This is the main part of the Habilitationsschrift (qualification of a lectureship) of B. Pompe. Here the theory of the generalized mutual information is developed. It is written in german, and it was not published elsewhere.

[10]

Die Spracherzeugung beim Menschen
( hab-lect.ps 60.5Mb ) ( hab-lect.ps.Z 5.7Mb )
Abstract

Dieser Beitrag gibt einen Überblick zur Funktion des menschlichen Stimmapparates. Insbesondere wird auf physikalische Modelle zur Dynamik der Stimmlippen, auf die Akustik in Rachen und Mundhöhle sowie auf einige neuronale Grundlagen der Spracherzeugung eingegangen. Schließlich werden Bezüge zur technischen Verarbeitung von Sprachsignalen hergestellt, zur Codierung und Erkennung.

[11]

Using Mutual Information to Measure Coupling in the Cardiorespiratory System
( pbhe.ps 5.8Mb ) ( pbhe.ps.Z 1.2Mb )
Abstract

Mutual information (MI) analysis represents a general method to detect linear and nonlinear statistical dependencies between time series, and it can be considered as an alternative to the well-known correlation analysis. This article shows how the concept of MI can be used to quantify the coupling between two systems, X and Y. We consider systems as coupled if there are two signals, x(t) and y(t), representing successive measurements of the systems, X and Y, respectively, such that x(t) and y(t) are statistically dependent. Roughly speaking, this means that we can learn anything on x from observations of y, and vice versa. MI represents a measure for the strength of statistical dependencies, hence it could also be used as a measure of coupling.
We apply our method to the cardiorespiratory system of a newborn.

Reference:
Pompe B., Blidh P., Hoyer D., and Eiselt M. (1998):
IEEE Engineering in Medicine and Biology, Magazine 17(1998)32-39

[12]

Detecting Spatio-Temporal Information Flow in the Cortex by Mutual Information Analysis of MEG Data
( cib..pdf 0.23Mb ) ( cib.zip 0.12Mb )
Abstract

We propose a new method to analyse the spatio-temporal statistical dependencies in multivariate time series. When applying this universal method to magnetoencephalogram (MEG) data, we can visualize the spatio-temporal information flow in the human cortex on a certain level of coarse graining.

Reference:
L.-H. Hiss and B. Pompe (1999):
to be published in: Proc. Interdisciplinary Workshop CHAOS IN BRAIN?, 10 - 12 March, 1999, Bonn, Germany, World Scientific

[13]

Transinformationsanalyse in der digitalen Signalverarbeitung
( iuk2.ps 7.8Mb ) ( iuk2.zip 0.78Mb )
Abstract

Mit diesem Beitrag machen wir auf ein neuartiges universelles Verfahren der digitalen Signalverarbeitung aufmerksam, das wir in den letzten Jahren entwickelt und exemplarisch in recht verschiedenen Bereichen angewandt haben. Dabei handelt es sich in einem recht allgemeinen Sinn um die Messung nichtlinearer Korrelationen, was wir Transinformationsanalyse nennen. Mit dem Verfahren könnte beispielsweise die Vorhersage von Zeitreihen verbessert werden. Die Methode ist auch geeignet, um raum-zeitliche Informationsflüsse in komplexen Systemen wie dem menschlichen Cortex quantitativ zu beschreiben. Entsprechend dem konkreten Anwendungsfall kann somit unter Umständen ein hoher praktischer Nutzen erzielt werden.

Reference:
B. Pompe (1999):
to be published in: Tagungsband, 2. IuK-Tage Mecklenburg-Vorpommern, 17. - 19. Juni 1999, Rostock

[14]

Ein Verfahren zur mathematischen Abstandsbestimmung von Diskurspartikeln: Form-Funktions-Korrelation
( ppcppe.zip , 0.74Mb, ziped ps-file, mit zusätzlichen Tabellen und einigen weiteren geringfügige Abweichungen zur veröffentlichten Fassung)
Abstract

Wir entwickeln ein Verfahren, um zu prüfen, inwiefern eine durch Hörtests gefundene sprachlich--funktionale Klassifikation von Diskurspartikeln durch eine kontextfreie mathematische Untersuchung der Struktur der Signale widergespiegelt wird. Angewandt wird das Verfahren auf einen Satz von 239 Diskurspartikeln, die in neun sprachlich--funktionale Funktionsklassen eingeteilt vorlagen. Als Grundlage der mathematisch--formalen Klassifikation dienten der Lautstärkeverlauf, der Grundfrequenzverlauf sowie eine Kombination dieser beiden Merkmale. Als mathematisches Abstandsma&suml; wurde ein geeigneter Transportabstand verwendet. Die beste Übereinstimmung zwischen empirischer und formaler Klassifikation ergab sich mit den kombinierten Merkmalen. Lautstärke und Sprachmelodie reichen demnach für sich allein genommen nicht aus, die empirische Klassifikation adäquat widerzuspiegeln. Allerdings ist im Vergleich beider Merkmale die Sprachmelodie bedeutend aussagekräftiger als der Lautstärkeverlauf.

Reference:
Ch. Bandt, B. Pompe, P. Streufert und P. Zorn (2000):
in: Germanistische Linguistik 157-158, 2001
Neue Wege der Intonationsforschung, Jürgen Erich Schmidt (Hrsg.), p. 51--72

[15]

Mutual Information and Relevant Variables for Predictions
( eco.zip , 0.38Mb, ziped pdf-file )
Abstract

In this chapter we propose a method to select from a possibly large set of observable quantities a minimal subset yielding (nearly) all relevant information on the quantity we are going to predict. We derive the theoretical background and give numerical hints and examples, including results for some daily dollar exchange rates. Our approach essentially profits from the availability of a fast algorithm for mutual information analysis.

Reference:
B. Pompe (2002):
to be published in: Modelling and Forecasting Financial Data: Techniques of Nonlinear Dynamics. Abdol S. Soofi and Liangyue Cao (eds.) (Kluwer Academic Publishers, Boston et al. 2002)

[16]

Permutation entropy - a natural complexity measure for time series
( bandt_pompe_02a.pdf , 0.27Mb )
Abstract

We introduce complexity parameters for time series based on comparison of neighboring values. The definition directly applies to arbitrary real-world data. For some well-known chaotic dynamical systems it is shown that our complexity behaves similar as Lyapunov exponents, and is particularly useful in the presence of dynamical or observational noise. The advantages of our method are its simplicity, extremely fast calculation, its robustness and invariance with respect to non-linear monotonous transformations.

Reference:
C. Bandt and B. Pompe (2002):
a minor modified version is published in: Physical Review Letters, vol. 88, no. 17, p. 174102

[17]

Entropy of interval maps via permutations
( pdf, <0.9MByte )
Abstract

For piecewise monotone interval maps we show that Kolmogorov-Sinai entropy can be obtained from order statistics of the values in a generic orbit. A similar statement holds for topological entropy.

Reference:
C. Bandt, G. Keller, and B. Pompe (2002):
NONLINEARITY 15(2002)1595-1602

[18]

Mutual Information Function Assesses
Autonomic Information Flow of Heart Rate Dynamics
at Different Time Scales
( ieee2005.pdf , <1.8MB)
Abstract

The Autonomic Information Flow (AIF) represents the complex communication within the Autonomic Nervous System (ANS). It can be assessed by the Mutual Information Function (MIF) of heart rate fluctuations (HRF). The complexity of HRF is based on several interacting physiological mechanisms operating at different time scales. Therefore one prominent time scale for HRF complexity analysis is not given a priori. The MIF reflects the information flow at different time scales. This approach is defined and evaluated in the present paper. In order to aggregate relevant physiological time scales, the MIF of HRF obtained from eight adult Lewis rats during the awake state, under general anesthesia, with additional vagotomy, and additional beta1-adrenergic blockade are investigated. Physiologically-relevant measures of the MIF were assessed with regard to the discrimination of these states. A simulation study of a periodically excited pendulum is performed to clarify the influence of the time scale of MIF in comparison to the Kolmogorov Sinai entropy (KSE) of that well defined system. The general relevance of the presented AIF approach was confirmed by comparing mutual information, approximate entropy, and sample entropy at their respective time scales.

Reference:
D. Hoyer, B. Pompe, K. H. Chon, H. Hardraht, Carola Wicher, and U. Zwiener (2005):
IEEE Biomedical Engineering 52,4(2005)584-592

[19]

Permutation entropy improves fetal behavioural state classification based
on heart rate analysis from biomagnetic recordings in near term fetuses

( frank-et-al-2006.pdf , <0.9MB)
Abstract

The relevance of the complexity of fetal heart rate fluctuations with regard to the classification of fetal behavioural states has not been satisfyingly clarified so far. Because of the short behavioural states, the permutation entropy provides an advantageous complexity estimation leading to the Kullback--Leibler entropy (KLE). We test the hypothesis that parameters derived from KLE can improve the classification of fetal behaviour states based on classical heart rate fluctuation parameters (SDNN, RMSSD, ln(LF), ln(HF)). From measured heartbeat sequences (35 healthy fetuses at a gestational age between 35 and 40 completed weeks) representative intervals of 256 heartbeats were visually preclassified into fetal behavioural states. Employing discriminant analysis to separate the states 1F, 2F and 4F, the best classification result by classical parameters was 80.0% (SDNN). After additionally considering KLE parameters it was improved significantly (p<0.0005) to 94.3\% (ln(LF), KLE_Mean). It could be confirmed that KLE can improve the state classification. This might reflect the consideration of different physiological aspects by classical and complexity measures.

Reference:
B. Frank, B. Pompe, U. Schneider, and D. Hoyer:
Med Biol Eng Comput (2006) 44: 179-187

[20]

Analysis of complex physiological systems by information flow: a time scale--specific complexity assessment

( hoyer-et-al-bmte-2006.pdf , <0.3MB)
Abstract

In the last two decades conventional linear methods for biosignal analysis have been substantially extended by non-stationary, non-linear, and complexity approaches. So far, complexity is usually assessed with regard to one single time scale, disregarding complex physiology organised on different time scales. This shortcoming was overcome and medically evaluated by information flow functions developed in our research group in collaboration with several theoretical, experimental, and clinical partners. In the present work, the information flow is introduced and typical information flow characteristics are demonstrated. The prognostic value of autonomic information flow (AIF), which reflects communication in the cardiovascular system, was shown in patients with multiple organ dysfunction syndrome and in patients with heart failure. Gait information flow (GIF), which reflects communication in the motor control system during walking, was introduced to discriminate between controls and elderly patients suffering from low back pain. The applications presented for the theoretically based approach of information flow confirm its value for the identification of complex physiological systems. The medical relevance has to be confirmed by comprehensive clinical studies. These information flow measures substantially extend the established linear and complexity measures in biosignal analysis.

Reference:
Dirk Hoyer, Birgit Frank, Bernd Pompe, Hendrik Schmidt, Karl Werdan, Ursula Müller-Werdan, Rafal Baranowski, Jan J. Zebrowski, Winfried Meissner, Ulf Kletzin, Daniela Adler, Steffen Adler, and Reinhard Blickhan :
Biomed Tech (2006) 51: 41-48

[21]

Partial Mutual Information for Coupling Analysis of Multivariate Time Series

( *.pdf , <0.6MB)
Abstract

We propose a method to discover couplings in multivariate time series, based on partial mutual information, an information-theoretic generalization of partial correlation. It represents the part of mutual information of two random quantities that is not contained in a third one. By suitable choice of the latter, we can differentiate between direct and indirect interactions and derive an appropriate graphical model. An efficient estimator for partial mutual information is presented as well.

Reference:
Stefan Frenzel and Bernd Pompe :
Phys. Rev. Lett. 99(2007)204101

goto Pompe: Homepage
© November 15, 2007, Pompe