Data mining in today’s business world
In today’s business world, where computers are a standard tool to maintain productivity, improve processes, gather and collect information, many companies face the problem of collecting, generating and storing too much data. This data is not always high quality and therefore is not really contributing to the overall business success. The increasing costs associated with storage solutions and the lack of proper data analysis is not leading to improved work efficiencies and business strategies. Many businesses are trying to maintain their competitive advantage, by trying to implement business intelligence and knowledge manage systems and using various kinds of collected and generated data to analyze trends, generate sales forecasts or implement new marketing strategies. Data mining is what business use for these tasks. But just as any tool it can have advantages as well as disadvantages. While the advantages will contribute to the businesses overall success if the data is mined and analyzed properly, it can also have negative impacts on the business as well as the customers. This paper will show the importance of data mining in today’s business world and how businesses can take advantage of proper data mining techniques to gain competitive advantage and increase their overall business success. At the same time this paper will show what the proper data mining techniques are and what steps businesses should take to keep data miners from overwhelming their organization, because that will not only have negative impacts on the overall success but can lead to lost business opportunities, inefficient marketing strategies and the complete misanalysis of data. Data mining in various forms is becoming a major component of how businesses operate. Almost every business process today involves some form of data mining. These can be in the areas of Customer Relationship Management, Supply Chain Optimization, Demand Forecasting, Assortment Optimization, and Business Intelligence. These are just some examples of business functions that haven been impacted by data mining techniques. Data mining means exploring and analyzing detailed business transactions. It implies "digging through tons of data" to uncover patterns and relationships contained within the business activity and history. It can also be described as "the nontrivial extraction of implicit, previously unknown, and potentially useful information from data" (Frawley, Piatetsky-Shapiro and Matheus, 1992) and "the science of extracting useful information from large data sets or databases" (Hand, Mannila and Smyth, 2001). The process of data mining can be done manually by slicing and dicing the data until a pattern becomes obvious which is a very lengthy process and not efficient at all in today’s fast paced business world and constant changes. Or, it can be done with programs that analyze the data automatically. The increasing volume of data in modern business and science calls for computer-based approaches and automated data analysis using more complex and sophisticated tools. The modern technologies of computers, networks, and sensors have made data collection and organization an almost effortless task. However, the captured data needs to be converted into information and knowledge to become useful (Mehmed, 2003). Data mining is being used everywhere, from e-commerce to retail, telecommunications, banking, insurance and health care. Insurance companies can browse claims and predict the likelihood of fraud, depending on specific attributes and historical experience. Banks might use data mining to determine whether certain loans should be approved based on applicant characteristics. In fact, the applicability of the technology is so wide that it can benefit practically any business with a large database by allowing the business to gain insight into data that would otherwise be too vast or complex. Data mining is especially compelling for e-commerce applications because Web site visitors interact with computers, e-commerce companies can very easily collect a huge body of information and a rich set of data that provide a "360-degree view" of their customers. Mining this collected data will likely cause intriguing patterns and rules to emerge, which in turn can drive business decisions such as product promotions or bundling strategies. The next step that many companies already starting to implement is to apply this set of rules to customers in real time (Newquist, 1997). Companies can gain very valuable understanding of the hidden patterns and relationships in their data by using proper data mining techniques with the right tools. With computers, proper technology and software as well as the a proper data mining strategy hundreds of thousands of variables can be examined, a staggering combination of attributes can be evaluate, various rules and risks can be analyzed and statistical methods can be applied. Generating this type of knowledge can help companies make better decisions, automate their decision process, identify and avoid problems, discern business opportunities and optimize their systems. The overall advantage is that companies can increase their top line by selling more, and decreasing their expense line by identifying unnecessary overhead and costs. Since there are so many ways on how data mining can be beneficial, businesses are starting to implement this technology more and more in their daily processes. But there is one main mistake that many companies make and that is not to have a proper data mining strategy or process which leads to data mining too much data without having an opportunity to analyze it properly and being able to implement it in an efficient way. Data mining usually involves extracting "hidden" information from a database and the understanding process can get a bit complicated and using the results of data mining is even harder because the user has to actually understand what is going on so that direct action can be take. For example, if the user is responsible for ordering a print advertising campaign, then understanding customer demographics is critical. Therefore, data mining has to be done properly without overwhelming the company. And that is not a simple task. In order to avoid data miners from overwhelming the organization and businesses understating the data a data mining strategy has to be developed. Many businesses using data mining tools are not looking in the right places to find what they are looking for. Data mining works best when there is a well-defined profile the business is searching for, for example a reasonable number of online sales per year, and a low cost of false alarms. Credit card fraud is one of data mining's success stories: all credit card companies’ data mine their transaction databases, looking for spending patterns that indicate a stolen card. Many credit card thieves share a pattern -- purchase expensive luxury goods, purchase things that can be easily fenced, etc. -- and data mining systems can minimize the losses in many cases by shutting down the card. But in order to maximize the benefits of data mining without overwhelming the business with data mining activities and too much data, a proper data mining strategy needs to be in place. The strategy consists of the following main steps which are understanding the problem and determining the objective, defining success criteria, assessing the situation, determining data mining goals and producing a project plan (Noyes, 2004). The probably most important step in the development of a data mining strategy is to understand the need to do data mining, i.e. understanding the problem that needs to be solved. To be capable to solve the problem efficiently the business has to understand the problem perspective, competing objectives and constraints as well as all important factors influencing the outcome. Defining the success criteria, which is the next step in defining a strategy is what makes data mining successful. Criteria can be quantitative such as setting a specific number of detected deviations, improved response rate of customers to some marketing campaigns or even percentage of correct patient diagnoses but the criteria can also be subjective or qualitative. In such as case the results of the data mining effort with respect to existing background knowledge about the problem will be assessed and the results must contain some new and useful insight into the relationships of domain variables (Newquist, 1997). The next step is to assess the situation which requires to gather information about what the existing expertise or background knowledge about the problem is, what data is available, if specific terminology for the problem needs to be defined as well as if potential costs need to be estimated. Once this has been accomplished the next task is to determine data mining goals which can range from increasing sales and determining customer properties with respect to their purchasing power or prevent credit card fraud by finding critical patterns for fraudulent card usage. Producing a valid and detailed project plan is part of the process to make sure that data mining does not become overwhelming and generate too much data which can be to broad and will not meet the objective or is not qualitative enough and therefore can lead to a business strategy that is doomed to fail (Noyes, 2004). Once data mining has been performed and data is available, the business has to make sure that the results are understood properly and can be used for the purpose identified previously.  The data has to be explored and surveyed to find out from the general structure of the data, whether or not there is useful amount of information enfolded in the extracted data sets. The exploration can be done in a simple or basic way which includes the examination of nominal attributes for multi-way frequency tables and examining the distributions of values for individual attributes. The verification of data quality, which is part of a proper data mining process, improves the final modeling results by checking the consistency of individual attribute values and types including quantity and distribution of missing values. The next steps are data preparation, evaluation of results and eventually the deployment of the results obtained. The data preparation process can be divided into data selection, data cleaning, formation of new data and data formatting. Data selection is based on criteria stressed in previous stages which can includes data quality properties such as completeness and correctness or technical constraints which can include limits on data volume or data type. Data cleaning which is the next step compliments the data selection by optimizing data quality for future modeling stage. This can be done by using data normalization techniques, treatment of missing values, data smoothing or data reduction. New data construction or data formatting are other steps that might be required based on what particular modeling tool has been chosen for the data modeling task(Mehmed, 2003). Once the data has been prepared the modeling part of data mining is the next step. The most important stages in the modeling phase include the selection of modeling technique, generating a test design, building a model assessment if the model itself. Which modeling techniques from the different kinds there are available will be used is something that needs to be chosen based on that the type of knowledge discovery task one wants to achieve: i.e. prediction or description. Once the technique has been selected, the models built and assessed, the evaluation of the results will reveal additional information, hints or suggestions for future modeling. The deployment of the results is the conclusion of the data mining project. It summarizes important points in the project, experiences gained and explains most important deliverables and results produced. This is the data that businesses can then implement into their business strategy (Noyes, 2004). Although data mining is not an easy task and if not properly executed can become very overwhelming for many businesses and organizations, a proper data mining strategy can help businesses avoiding this. The advantages of data mining are existent and should be used to generate not only more targeted business approaches, but customer satisfaction and increased sales. With the proper approach, technology, tools and the strategy asking the correct questions to analyze the problem and target the identified goals, data mining can be guaranteed success.
Scroll to Top