Home / Big Data / Let your data do the talking

Let your data do the talking

Via LinkedIn : Although the principles of scientific management from Frederick Taylor have long become obsolete, many parts of the theory are still important for organisations today. When was the last time you were involved in a project concerned with efficiency improvement, the elimination of waste or the identification of best practices? These are just a few topics from scientific management that are still part of industrial engineering and everyday management decision making. Key for the success of these kinds of projects is to have (or obtain) an in depth understanding of the work processes that require improvement. You can imagine that without this, changing the process might cause you to end up with a worse performance than you started with. The default way to gather information for analysing a process is by studying business process maps, interviewing people and fact finding on the shop floor. This can however be very time consuming, where the quality and accuracy of the gathered data could be questionable. Business process maps are known for their outdatedness (written to pass some ISO certification step years ago), people have different views on how processes are performed, while fact finding many times can only cover part of the processes under investigation. Not a very good start for a successful improvement project wouldn’t you say. There is however a solution and it’s called data!

Many organisations today have implemented workflow management, CRM and/or ERP systems. These systems are what you could call “process aware”. Key is their capability to log events, like a new order coming in, a request to process an invoice, the rejection of an insurance claim, the admittance of a patient, etc. These systems register very detailed information on the activities that are being performed, information that could be used to mine the data to uncover the actual work processes. Using the logged events related to the same case (e.g a new customer order) in process mining, the sequence in which they were performed is used to identify all the activities required to process the complete case. If the event log also contains information on the performer (person/resource, etc) of the activity and timestamps on when the activity took place; resource usage, duration and productivity can be measured as well. So, the data does all the talking instead of the interviewees.

Traditionally process mining is focussed on deriving information about the actual work process, the organizational context (who performs what), and execution properties (resource usage, duration, performance, etc) from event logs. With the resource information from the event logs social networks can be extracted; this allows organizations to monitor how people, groups, or software/system components are working together. Next to the discovery of actual work processes, process mining can be used to test conformance with the to-be (or designed) work processes, enabling the work processes to be audited in a fast and objective manner. This can especially be of value in highly regulated businesses like banking or insurance, checking conformance with regulations like Basel III. A third area which process mining can be of value is by extending an existing process model with new information, for example using the information from the event log to detect the data dependencies or decision rules for a specific activity.


To illustrate how process mining works I used an example data set containing the event logs of gynaecological oncology patients of a Dutch hospital. The data set contains the event logs of 627 individual patients. Using the open source data mining platform RapidMiner and the Process Mining package ProM from the Process Mining expertise centre at Eindhoven University I created the following process flow from the data using the ILP miner (an integer linear programming based model to extract a process flow). Note that I did not use any a-priori knowledge about the care process of this group of patients. This all comes from the event log data. Using the same tools and data the social network can be constructed providing insight on who works with who, who delegates work to who and the intensity of these work relations. Also using visualisations like the Dotted Chart specific patterns can be detected in the way the patients are treated.


Using process mining to discover work processes from event logs can be very powerful and less time consuming than the “old” way of interviewing, studying outdated process flow descriptions and fact finding expeditions. When you let the data do the talking, first results can be delivered quickly. Crucial is of course to have access to data. It requires skill to extract the right data from an ERP-like system, probably a lot of data cleansing needs to be done, including the check on completeness and validity of the data. Also it’s quite easy to get swamped in data, especially when the number of log events is big and a lot of process steps are involved. In environments like hospitals a lot of unstructured processes exist which will make it more difficult to use techniques like process mining as is, however using techniques from data mining like clustering first, satisfactory results can be achieved. Compared to traditional business Intelligence tools that only provide an aggregate view of the process inputs and outcomes, process mining dives inside the processes and provides the insights to give your next improvement project a head start.

About Editorial Team

Check Also


How Women Are Shaping The Big Data Revolution

Via Forbes : It has become abundantly clear by now. Successful business adoption of Big Data …