Advances in Mathematics of Communications (AMC)

Nearest-neighbor entropy estimators with weak metrics

Pages: 119 - 127, Volume 8, Issue 2, May 2014      doi:10.3934/amc.2014.8.119

       Abstract        References        Full Text (365.5K)       Related Articles       

Evgeniy Timofeev - Department of Computer Science, Yaroslavl State University, Yaroslavl, Russian Federation (email)
Alexei Kaltchenko - Department of Physics and Computer Science, Wilfrid Laurier University, Waterloo, Ontario N2L3C5, Canada (email)

Abstract: A problem of improving the accuracy of nonparametric entropy estimation for a stationary ergodic process is considered. New weak metrics are introduced and relations between metrics, measures, and entropy are discussed. A new nonparametric entropy estimator is constructed based on weak metrics and has a parameter with which the estimator is optimized to reduce its bias. It is shown that estimator's variance is upper-bounded by a nearly optimal Cramér-Rao lower bound.

Keywords:  Entropy, stochastic process, stationary, estimation, nonparametric, nearest neighbor, bias, metric.
Mathematics Subject Classification:  Primary: 58F15, 58F17; Secondary: 53C35.

Received: June 2012;      Available Online: May 2014.