A MapReduce-supported network structure for data centers
نویسندگان
چکیده
Several novel data center network structures have been proposed to improve the topological properties of data centers. A common characteristic of these structures is that they are designed for supporting general applications and services. Consequently, these structures do not match well with the specific requirements of some dedicated applications. In this paper, we propose a hyper-fat-tree network (HFN): a novel data center structure for MapReduce, a well-known distributed data processing application. HFN possesses the advanced characteristics of BCube as well as fat-tree structures and naturally supports MapReduce. We then address several challenging issues that face HFN in supporting MapReduce. Mathematical analysis and comprehensive evaluation show that HFN possesses excellent properties and is indeed a viable structure for MapReduce in practice. Copyright © 2011 John Wiley & Sons, Ltd.
منابع مشابه
A MapReduce-supported Data Center Networking Topology
Several novel data center networking (DCN) topologies have been proposed to improve the topological properties of data centers. Unfortunately, it is ignored that whether these topologies are suited for the online applications and infrastructure services running on the corresponding data centers. In this paper, we propose a novel DCN topology, named HyperFat-tree Network (HFN). HFN incarnates th...
متن کاملGreenMap: Green mapping of MapReduce-based virtual networks onto a data center network and managing incast queueing delay
Energy consumption is a first-order concern for today’s data centers. MapReduce is a cloud computing approach that is widely deployed in many data centers. Toward virtualizing data centers, we consider MapReduce-based virtual networks that need to be embedded onto a data center network. In this paper, we propose GreenMap, a novel energy-efficient embedding method that maps heterogeneous MapRedu...
متن کاملCloud Computing Technology Algorithms Capabilities in Managing and Processing Big Data in Business Organizations: MapReduce, Hadoop, Parallel Programming
The objective of this study is to verify the importance of the capabilities of cloud computing services in managing and analyzing big data in business organizations because the rapid development in the use of information technology in general and network technology in particular, has led to the trend of many organizations to make their applications available for use via electronic platforms hos...
متن کاملOptimizing MapReduce for Multicore Architectures
MapReduce is a programming model for data-parallel programs originally intended for data centers. MapReduce simplifies parallel programming, hiding synchronization and task management. These properties make it a promising programming model for future processors with many cores, and existing MapReduce libraries such as Phoenix have demonstrated that applications written with MapReduce perform co...
متن کاملAdaptive Dynamic Data Placement Algorithm for Hadoop in Heterogeneous Environments
Hadoop MapReduce framework is an important distributed processing model for large-scale data intensive applications. The current Hadoop and the existing Hadoop distributed file system’s rack-aware data placement strategy in MapReduce in the homogeneous Hadoop cluster assume that each node in a cluster has the same computing capacity and a same workload is assigned to each node. Default Hadoop d...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
- Concurrency and Computation: Practice and Experience
دوره 24 شماره
صفحات -
تاریخ انتشار 2012