The aim of this project is to introduce Spark Clouds, which integrate spark lines into a tag cloud to convey trends between multiple tag clouds. Spark Clouds ability is to show trends compares favorably to the alternative visualizations. In the Existing System, Tag clouds are used to display the relative tag frequency, popularity, or importance by font size. They serve as a visual summary of do...

Decision tree is one of the most widely used classification methods. For massive data processing, MapReduce is a good choice. Whereas, MapReduce is not suitable for iterative algorithms. The programming model of Spark is proposed as a memory-based framework that is fit for iterative algorithms and interactive data mining. In this paper, C4.5 is implemented on both MapReduce and Spark. The resul...

Apache Spark is a popular framework for writing large scale data processing applications. Our long term goal is to develop automatic tools for reasoning about Spark programs. This is challenging because Spark programs combine database-like relational algebraic operations and aggregate operations, corresponding to (nested) loops, with User Defined Functions (UDFs). In this paper, we present a no...

A theory of the Hindenburg fire that has gained popular acceptance proposes that an incendiary paint covered the outer surface of the airship. Further, according to this theory, the paint was both the point of ignition and the sole cause of the rapid spread of the fire. The hydrogen within the airship is held blameless for both the start of the fire and its subsequent advance. We have examined ...

Spark is a popular framework for writing large scale data processing applications. Our goal is to develop tools for reasoning about Spark programs. This is challenging because Spark programs combine database-like relational algebraic operations and aggregate operations with User Defined Functions (UDF s). We present the first technique for verifying the equivalence of Spark programs. We model S...

We introduce SparkCL, an open source unified programming framework based on Java, OpenCL and the Apache Spark framework. The motivation behind this work is to bring unconventional compute cores such as FPGAs/GPUs/APUs/DSPs and future core types into mainstream programming use. The framework allows equal treatment of different computing devices under the Spark framework and introduces the abilit...

Apache Spark is a popular framework for data analytics with attractive features such as fault tolerance and interoperability with the Hadoop ecosystem. Unfortunately, many analytics operations in Spark are an order of magnitude or more slower compared to native implementations written with high performance computing tools such as MPI. There is a need to bridge the performance gap while retainin...

Spark is gaining wide industry adoption due to its superior performance, simple interfaces, and a rich library for analysis and calculation. Like many projects in the big data ecosystem, Spark runs on the Java Virtual Machine (JVM). Because Spark can store large amounts of data in memory, it has a major reliance on Java’s memory management and garbage collection (GC). New initiatives like Proje...

Efficiency and emissions of spark-ignited engines are significantly affected by combustion phase which can usually be indicated by crank angle of 50% mass burnt (CA50). Managing combustion phase at the optimal value at which the maximal efficiency can be achieved is a challenging issue due to the cyclic variations of combustion process. This paper addresses this issue in two loops: CA50 set-poi...

MapReduce and Spark are two very popular open source cluster computing frameworks for large scale data analytics. These frameworks hide the complexity of task parallelism and fault-tolerance, by exposing a simple programming API to users. In this paper, we evaluate the major architectural components in MapReduce and Spark frameworks including: shuffle, execution model, and caching, by using a s...