Towards Optimal Moment Estimation in Streaming and Distributed Models

One of the oldest problems in data stream model is to approximate p th moment \(\Vert \mathbf {X}\Vert _p^p = \sum _{i=1}^n {X}_i^p\) an underlying non-negative vector \(\mathbf {X}\in \mathbb {R}^n\) , which presented as a sequence \(\mathrm{poly}(n)\) updates its coordinates. Of particular interest when \(p \in (0,2]\) . Although tight space bound \(\Theta (\epsilon ^{-2} \log n)\) bits known for this problem both positive and negative are allowed, surprisingly, there still gap complexity all positive. Specifically, upper \(O(\epsilon bits, while lower only \(\Omega + bits. Recently, \(\tilde{O}(\epsilon was obtained under assumption that arrive random order We show (0, 1]\) not needed. Namely, we give worst-case streams estimating _p^p\) Our techniques also new bounds empirical entropy stream. However, (1,2]\) natural coordinator blackboard distributed communication topologies, ^{-2})\) bit max-communication based on randomized rounding scheme. protocols rise heavy hitters matrix product. generalize our results arbitrary topologies G obtaining ^{2} d)\) bound, where d diameter Interestingly, rules out complexity-based approaches proving streaming algorithms. In particular, any such must come from topology with large diameter.