It is commonly accepted that machine translation is a more complex task than part of speech tagging. But how much more complex? In this paper we make an attempt to develop a general framework and methodology for computing the informational and/or processing complexity of NLP applications and tasks. We define a universal framework akin to a Turning Machine that attempts to fit (most) NLP tasks i...

Accurately determining dependency structure is critical to discovering a system’s causal organization. We recently showed that the transfer entropy fails in a key aspect of this—measuring information flow—due to its conflation of dyadic and polyadic relationships. We extend this observation to demonstrate that this is true of all such Shannon information measures when used to analyze multivaria...

The paper attempts to provide a conceptual basis for allusions to information processing in discussions about cognition. It de nes the notions of informational medium as a subsystem within a larger dynamical system, the notions of information processing and information management operations, and the notion of informational network a network of informational media. It suggest that situated cogni...

the estimate of entropy (sample entropy), has been introduced by vasicek (1976), for the first time. in this paper, we provide an estimate of entropy of order statistics, that is the extention of the entropy estimate. then we present an application of the entropy estimate of order statistics as a test statistic for symmetry of distribution versus skewness. the proposed test has been compared wi...

This article explores the assumption that a deeper (quantitative) understanding of the information-theoretic implications of sensory-motor coordination can help endow robots not only with better sensory morphologies, but also with better exploration strategies. Specifically, we investigate by means of statistical and informationtheoretic measures, to what extent sensory-motor coordinated activi...

Let p and q be probability vectors with the same entropy h. Denote by B(p) the Bernoulli shift indexed by Z with marginal distribution p. Suppose that φ is a measure preserving homomorphism from B(p) to B(q). We prove that if the coding length of φ has a finite 1/2 moment, then σ2 p = σ 2 q , where σ 2 p = ∑ i pi(− log pi − h)2 is the informational variance of p. In this result, the 1/2 moment ...

A measure is derived to quantify directed information transfer between pairs of vertices in a weighted network, over paths of a specified maximal length. Our approach employs a general, probabilistic model of network traffic, from which the informational distance between dynamics on two weighted networks can be naturally expressed as a Jensen Shannon divergence. Our network transfer entropy mea...

In this paper, we investigate by means of statistical and information-theoretic measures, to what extent sensory-motor coordinated activity can generate and structure information in the sensory channels of a simulated agent interacting with its surrounding environment. We show how the usage of correlation, entropy, and mutual information can be employed (a) to segment an observed behavior into ...

R ecent literature models leadership as a process of communication in which leaders' rhetorical signals facilitate followers' coordination. While some studies have explored the effects of leadership in experimental settings, there remains a lack of empirical research on the effectiveness of informational tools in real political environments. Using quantitative text analysis of federal and sub-n...

Recent literature models leadership as the process of communication where the rhetorical signals of leaders facilitate followers’ coordination. While studies exist that explored the effects of leadership in experimental settings, there remains a lack of empirical research on the effectiveness of informational tools in real political environments. Using quantitative text analysis of federal and ...