Fault Tolerant Distributed Stream Processing based on Backtracking
نویسندگان
چکیده
منابع مشابه
Fault-tolerant stream processing using a distributed, replicated file system
We present SGuard, a new fault-tolerance technique for distributed stream processing engines (SPEs) running in clusters of commodity servers. SGuard is less disruptive to normal stream processing and leaves more resources available for normal stream processing than previous proposals. Like several previous schemes, SGuard is based on rollback recovery [18]: it checkpoints the state of stream pr...
متن کاملScalable and Fault-tolerant Stateful Stream Processing
As users of “big data” applications expect fresh results, we witness a new breed of stream processing systems (SPS) that are designed to scale to large numbers of cloud-hosted machines. Such systems face new challenges: (i) to benefit from the “pay-as-you-go” model of cloud computing, they must scale out on demand, acquiring additional virtual machines (VMs) and parallelising operators when the...
متن کاملDiscretized Streams: A Fault-Tolerant Model for Scalable Stream Processing
Many “big data” applications need to act on data arriving in real time. However, current programming models for distributed stream processing are relatively low-level, often leaving the user to worry about consistency of state across the system and fault recovery. Furthermore, the models that provide fault recovery do so in an expensive manner, requiring either hot replication or long recovery ...
متن کاملMillWheel: Fault-Tolerant Stream Processing at Internet Scale
MillWheel is a framework for building low-latency data-processing applications that is widely used at Google. Users specify a directed computation graph and application code for individual nodes, and the system manages persistent state and the continuous flow of records, all within the envelope of the framework’s fault-tolerance guarantees. This paper describes MillWheel’s programming model as ...
متن کاملDistributed Recovery Block Based Fault-tolerant Multicasting
This paper presents a fault-tolerant and adaptive multicasting method that employs a modified distributed recovery block (DRB) approach. The section of a distributed system spanning between the source and destination nodes is partitioned into series of overlapping DRB groups on the multicasting paths. Each DRB group consists of three nodes: a current node and two successor nodes. The source nod...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: International Journal of Networked and Distributed Computing
سال: 2013
ISSN: 2211-7946
DOI: 10.2991/ijndc.2013.1.4.4