Data mining seems to be the holy grail of enterprise software today. Yet while many have heard of it, few have seen it in action. Even fewer seem to know how to actually use it. Mainstream adoption seems to perpetually hover just below the horizon, but creating a solid business case has been challenging, especially for small or mid-size businesses without the budget for tailor made solutions.
The truth is in there
While most implementations crunch large quantities of data for analytical insights and forecasting purposes, there is another emerging perspective that aims to map and visualize business workflows. The new kid in town is called process mining, and we’ll show you how it can help you locate bottlenecks, cut lead times and improve business efficiency. With a relatively small footprint and plenty of low-hanging fruit, process mining can be your introduction to the world of Big Data. And no, it does not require a new million dollar infrastructure.
The scenario
You are responsible for an area, a process or a facility within your business. You are aware that processes fail from time to time, but feel that your ability to improve them is hampered by a lack of insight regarding where, when and why this happens. Under pressure from the C-suit, you are looking for a solution to improve things but are struggling to gather useful inputs to decide where to start.
Process mapping is the first step to process improvement, but often a daunting task. While well-defined processes and workflows are paramount to successful strategy execution, the traditional top-down approach of interviewing managers and senior staff suffers from a couple of fundamental flaws.
First of all, it takes a lot of time. Know-how is usually dispersed among multiple people in different geographies and multiple more or less independently working streams. Creating a realistic picture out of this mosaic can require a lot of travel and calls. While it is an excellent way to improve your frequent flyer membership tier, it may not be the best use of your precious time.
Second, it is imprecise. Executives often have a rose tinted view of how their departments are run, either because it is reported that way from subordinates or because they “tactically forget” flaws out of pride or for political reasons. In addition, some scenarios and exceptions may seem irrelevant or just slip the mind and are therefore never brought up during the interviews.
The Solution
Unbeknownst to most, ERP systems log plenty of data that when gathered and formatted the right way will provide account of how processes are actually run. When presented and visualized with a process mining tool, this data will provide a map of the real process flows from your production systems.
To show an example, this is what the basic customer order flow looks like in the iStone test environment when analyzed and visualized with a process mining tool (QPR Process Analyzer):
Some immediate insights that can be gleaned from this image is:
- Median process time is just 1 day and 10 hours, but the average is over 27 days, indicating that we have a “long tail” of orders with really long lead times
- It seems that we in 48% of the cases register the order after our confirmed delivery date (the green arrow indicates which the first step is in the process)
- We are only invoicing 19% of our order lines – an insanely low hit rate
- Only 37% of all customer order lines are actually delivered
Fortunately this is a test environment, and the horrible performance can be explained by our consultants unit testing different steps and never completing the full process. If this was production data we would be looking at a company with serious problems.
Process mining also allows us to benchmark the processes by different factors, such as the division they are performed in, or in the case of customer orders, the type of customer that made the order. The flowchart can be drawn per any such factor, allowing us to benchmark different units or sets within the organization:
Looking at just two of the divisions in this system, we get some other valuable insights. For example:
- The problem with orders registered after their confirmed delivery date is prevalent in both divisions, with 46% in division AAA and 53% in division 900
- Although the process flow is much more complex in division 900, both the average and median lead time is actually lower there
- The poor rate of orders that are actually invoiced is a problem in both divisions, and division 900 has an even lower rate at just 6%
By comparing all divisions we can deepen this analysis further, and we could also choose to look at the charts by other factors such as the type of item sold or the responsible salesperson. This kind of analysis is only one piece of the puzzle though. It is one thing to understand how the processes run, but the real power in process mining comes from understanding why they run the way they do – especially in the cases that aren’t performing as well as expected.
In Process Analyzer, we can use the Duration Analysis to get an overview of our lead times:
Here we can see that the majority of orders are processed within 9 days, but there are outliers that take severely longer, with a handful of cases taking 99 days or more.
From this view, we can filter to investigate only the orders that has a lead time of over nine days. Based on this selection, we could redraw the flowchart to see the process map only for these cases, but this time we will jump straight into the Influence analysis to see the factors that cause these delays:
This view lists the factors having the greatest impact on lead times. The algorithm gets a bit complex, but what the results say is that Warehouse ‘ERD’ is the worst sinner when it comes to order lead times, with 10 more cases then expected, causing 13% of all delays. Moving down the list, we can see that the item type with the same name is also a bad performer as it ties as the second worst factor together with Division 900, item type 11 and warehouse 40.
Scrolling down the list, we can also find the factors that contribute positively:
These results could be used to find internal role models for benchmarking. Division AAA for example seems to be a good place to start for identifying best practice, or perhaps a promotion for the responsible people.
Both the benchmarking and influence analyses can be performed with any relevant factor from the source system, so the report can be tailored to most any demand. For finance, the cost center might be the most important factor, while for production it could be the work center or production line.
How to act on this info
Having this info is great, but of little use unless we turn it to positive change. Here are some examples of how to proceed with the example insights above:
1. Clarify policy
Judging by the number of orders entered after their expected delivery date, we might need to communicate the importance of prompt order registry and realistic delivery promises. The influence analysis could help us by showing where this problem is most severe.
2. Define best practice
The analyzed divisions differ quite a lot both in regards to process flow and performance. Division AAA would have much to gain by learning from division 900 about efficiency, but could in turn perhaps teach them a thing or two about invoicing (although at 16%, we might be better off with an external consultant...)
3. Shift responsibilities
When comparing units or suppliers that produce or distribute the same types of items, the results can be used to improve our supply chain by changing our sourcing.
Gathering the info above manually, from interviews and workshops, would be time-consuming, patchy and unreliable. With a process mining tool it is a matter of days to gather and produce these results. So no, the truth is not “out there” – it’s already in our databases, just waiting to be unlocked.
