DETAiLS FUSE System Architecture

The DETAiLS architecture design was motivated by two concerns: efficiency and modularity. This project involves processing very large corpora of very long full text documents, and investigating the use of potentially computationally intensive natural language, network and time series features over that corpora. As such, it was necessary to develop a robust architecture capable of handling both the large scale of the input corpora as well as the complexity and diversity of techniques to be applied to that corpora. Equally important to these efficiency goals was designing a system flexible enough to support the ongoing research of the team. Since the FUSE problem invites techniques from a variety of domains, it was necessary to make a system that allowed researchers to independently develop and test components to compute new features as well as allowing them to enable or disable other components on the fly.