Data mining and operational research: techniques and applications

Data mining (DM) involves the use of a suite of techniques that aim to induce from data, models that meet particular objectives. DM algorithms are built on a range of techniques, including information theory, statistics, linear and non-linear models, AI, meta-heuristics. Within the context of data analysis methods, data mining can be considered to be an exploratory knowledge discovery approach to be contrasted with a confirmatory approach in which a hypothesis is specified and the validity of the hypothesis is tested against the data. DM models can be designed to find new useful patterns within the data that can be exploited. For example, an insurance company will already be aware that young men in fast cars pose a high risk (as do older women). Such a pattern in the data will be of no interest to them. However, they may be interested if they were to find that young men in fast, classic cars are much less of a risk. They could then develop pricing and marketing strategies to exploit this newly discovered pattern. In the context of contemporary organizations, the major motivating factors for the interest in, and application of, data mining techniques can be categorized as: changed business environment (eg pressure on traditional marketing techniques , shorter time to market, shorter product life cycles, and increased competition and business risks), drivers (eg customer, competition, and data asset), and enablers (eg data flood in organizations, new information technology solutions in data mining, data warehousing, and hardware). Many organizations routinely collect large amounts of data on their clients, employees, and suppliers. Factories collect data on their machinery and productivity. Hospitals collect data on patients, doctors, medications, and procedures. These data are often untidy, incomplete, and sometimes erroneous and yet, if used properly, it can be a valuable asset for management and effective organizations can find ways to exploit it. Predicting fraudulent behaviour in a customer database, for example, will generally require highly accurate models even if they can only be applied to a small proportion of the database. On the other hand, if the organization is planning a targeted marketing campaign and wishes to predict which customers might buy a certain product, the accuracy of the model can be much lower. Both operational research (OR) and DM are aimed at the development and application of models that can improve the quality of organizational decision-making process. While OR can be considered …