Building Data Integration Systems: A Mass Collaboration Approach

Building data integration systems today is largely done by hand, in a very labor intensive and error prone process. In this paper, we describe a conceptually new solution to this problem: that of mass collaboration. The basic idea is to think about a data integration system as having a finite set of parameters whose values must be set. To build such a system, the system administrators can construct and deploy a system “shell”, then ask the users to help the system “automatically converge” to the correct parameter values. This way, the enourmous burden of system developments is lifted from the administrators and spread “thinly” over a multitude of users. We discuss the challenges to this approach and propose solutions. We then describe our current effort in applying this approach to the problem of schema matching in the context of data integration.