Entropy estimators with almost sure convergence and an O(n-1) variance

Alexei Kaltchenko; Nina Timofeeva

doi:10.3934/amc.2008.2.1

Article Contents

2008, Volume 2, Issue 1: 1-13. Doi: 10.3934/amc.2008.2.1

This issue Previous Article Next Article Duality for some families of correction capability optimized evaluation codes

Entropy estimators with almost sure convergence and an O(n^-1) variance

Alexei Kaltchenko^1, and
Nina Timofeeva^2,

1.
Department of Physics and Computer Science, Wilfrid Laurier University, Waterloo, Ontario N2L3C5
2.
Department of Computer Science, Yaroslavl State University, Yaroslavl, 150000

Received: March 31, 2007

Revised: September 30, 2007

Published: January 2008

Abstract Related Papers Cited by

Abstract

The problem of the estimation of the entropy rate of a stationary ergodic process $\mu$ is considered. A new nonparametric entropy rate estimator is constructed for a sample of n sequences $(X_1$⁽¹⁾$,\ldots, (X_m$⁽¹⁾$),\ldots, (X_1$⁽ⁿ⁾ $,\ldots, (X_m$⁽ⁿ⁾$)$ independently generated by $\mu$. It is shown that, for $m=O(\log n)$, the estimator converges almost surely and its variance is upper-bounded by $O(n$⁻¹$)$ for a large class of stationary ergodic processes with a finite state space. As the order $O(n$⁻¹$)$ of the variance growth on $n$ is the same as that of the optimal Cramer-Rao lower bound, presented is the first near-optimal estimator in the sense of the variance convergence.

Keywords:

Entropy rate,
information source,
stationary ergodic stochastic processes,
estimation,
statistics.

Mathematics Subject Classification: Primary: 94A15, 94A17; Secondary: 37M25.

Citation:

Article Metrics

HTML views() PDF downloads(166) Cited by(0)

Entropy estimators with almost sure convergence and an O(n^-1) variance

Abstract

Access History

Article Metrics

Access History

Other Articles By Authors

Catalog

Entropy estimators with almost sure convergence and an O(n-1) variance

Abstract

Access History

SHARE

Article Metrics

Access History

Other Articles By Authors

Catalog

Export File

Citation

Format

Content

Entropy estimators with almost sure convergence and an O(n^-1) variance