There is no room for inefficiencies in the current business environment as it can lead the business towards losses in various aspects like loss of customer trust, losing from the competition and cost over-runs. Therefore, organizations are now focusing more on monitoring and fine-tuning their business operations to ensure that their performance is at an optimum level.
Real-time process data capture is essential to do so. Process mining is an enabler in this case as it solves the key problem of discovering process inefficiencies and restructures. It visualizes the business process flows and its variations. It bridges the gap between traditional model based approach like simulations and bpm techniques and data centric approach like machine learning and data mining. The process of combining event logs with process mining allows the management to scrutinize problems based on facts.
This document is dedicated to providing a detailed report on process mining and its aspects like industry statistics, trends, importance for a business, implementation and role in optimizing business process management. Some use cases of process mining are also mentioned which can help in grasping the concept better.
Understanding Process Mining
It All Started With Process Thinking & Optimization
Back in the 19th century, a lot of researchers were working on improving and standardizing the manufacturing process with the aim of saving time and efforts, which were being wasted in many ways. Around 1850s, many of them were looking for ways to interpret all the events in a process in simpler terms. With process thinking, they sought to know what the process is and how it works. A century later, IBM launched the database management system which lead to the creation of data warehousing followed by data mining. This helped in providing data from all types of technology systems which could be used to improve current operations.
Back To The Present
In today's world, process mining has become an emerging discipline which provides a comprehensive set of tools for generating fact based insights and leads for process betterment. It is a family of techniques related to the field of data science and Business Process Management (BPM) that helps in the analysis of business operations based on event logs. The goal of process mining is to generate insights and execute accordingly. It can be used to gain insights on what machines, organizations and its people are really doing and identify and address performance and compliance problems.
Market Size And Forecasts
- As mentioned in the survey conducted by Gartner in 2018, process mining is used the mostly in these processes: business process improvement (41%), process automation (18%), compliance and auditing (21%), and digitalization (15%).
- Another fact mentioned in the Gartner report of 2018 states that "Adoption of basic process mining types as follows: process discovery (48%), enhancement (26%) and conformance (25%). However, there is a significant trend towards an increased focus on using process mining for process conformance and enhancement. Adoption of process mining in these areas was expected to reach the level of process discovery by 2020."
- Business decision makers are also planning to increase their adoption rate of process mining, current rate being 83%, for optimizing their customer journey mapping and majority of them will be increasing it significantly, according to a Forrester report.
- Another fact mentioned in the report was that 93% of the questionnaire respondents stated that they have either implemented or plan to implement process mining in their organization.
Key Capabilities Of Process Mining
There are three key capabilities of process mining:
Automated Business Process Delivery:
Learning about the compliance issues, bottlenecks and inefficiencies and conducting automated process model detection.
Business Process Conformance Checking:
Helping with the analysis of root-cause and identifying highest priority issues by comparing actual processes with designated processes.
Enhancing the processes by analyzing, increasing process automation and detecting necessary areas for optimization.
Need For Process Mining
1. Improved Customer Experience
Shorter Lead Time
Companies can find ways to optimize their processes and reduce lead time by using process mining tools. For example, Veco, a Dutch engineering company, implements process mining and analytics in its quality management department. They have been able to reduce lead time by 11 weeks by reconstructing their processes and hence, improving customer satisfaction.
Shorter Reaction Time
Companies can react faster by constantly monitoring processes using process mining tools. Say, a problem occurs, then the software can quickly identify the root cause and hence, the company can react faster too. Shorter reaction time has certain benefits like increased customer satisfaction and improved company performance.
Identifying And Resolving Process Bottlenecks
Sometimes, there can be invisible bottlenecks in the process which can slow down the operation. In such cases, the company has to spend a substantial share of their resources to locate the bottleneck and then also in solving the problems. Process mining can be a lifesaver in such cases.
2. General Benefits
Data-Driven Decision Making
Process mining tools increase the availability of high quality and strong data for visualization/analytics capabilities and data driven decision making. Once the data is collected, strong data visualization softwares are used to allow employees to analyze and easily visualize the data, hence enabling rapid and improved decision making driven by data.
Improved Performance Management
It is difficult and monotonous to accurately measure employee performance. Process mining tools are helpful in this situation as they automate process performance data collection. This allows process owners to continuously make changes to improve processes by enabling KPIs like First time right, SLA and time to resolution.
3. Reduced Cost By
Event logs play an important role in helping process mining tools in deriving how distinct cases are needed to be handled and how operational decisions are made. Companies are helped in this way in knowing how exactly the current processes are running. Programmers can build RPA bots that help to automate processes once they know how the process runs.
Elimination Of Unnecessary Steps
Companies can cut their wasteful expenditure drastically by removing unnecessary steps in processes, This can understood by the statement by a process vendor called Eneco, which claimed that they saved around 15 million euros ever since they started utilizing process mining in operations.
Cheaper And Faster Auditing
Auditing is a boring and time and efforts consuming activity. Again, process mining helps to solve the problem by analyzing the data faster. For example, using a process mining solution, firms can complete their end-customer process analysis in less than a week.
Market Sentiment On Process Mining
- According to a Forrester report, Digitalization of processes is delayed due to misunderstood processes. The picture becomes even more complicated because of process gaps and manual routing. 37% of organizations and tech-related decision-makers stated that they face such problems.
- According to Jan Claes, 61% respondents believe that one of the most profound benefits of process mining is that factual process data can be used for further diagnosis too.
- According to QPR, businesses can reduce RPA implementation time by 50% and project risk by 60% while at the same time, increase the business value by more than 40%.
- According to UiPath, of the people who use RPA in their business, 78% believe that process mining is key to enabling their RPA efforts.
- The most common types of answers when executives are asked where they see most opportunities are audit and control activities (19%) and optimization of procurement process (22%).
- According to QPR, Process automation rates are increased by 50% by using process mining.
- Average level of process conformance is about 40-65%, according to QPR2.
Benefits Of Using Process Mining Tools
There are several process mining tools in the market, some are only there for commercial appeal while others are used for actual application and improvement. In order to allow process mining to help in achieving the necessary objectives, there are two minimum functionalities that act as threshold. These functionalities are:
- Track event logs
- Monitor process performance
Tools also offer additional value–adding features such as:
- Faster and more accurate integration of other technologies.
- Being collaborative and more secure
- Offering flexible process reporting
- Performing key performance indicator (KPI) calculation
Based on EY findings, mentioned in its process mining white paper, the six most used process mining tools are: Celonis, ARIS Process Mining, Minit, QPR, Signavio Process Intelligence and ProcessGold. All of them are also recognized by Gartner as market–leading tools.
Based on the findings by EY in its process mining white paper, the top six market-leading process mining softwares are:
All these tools present the following capabilities:
- Process discovery
- Conformance checking
- Process visualization and performance reporting
- Insights for improvement (enhancement)
Perspectives Covered In Process Mining
There are four key perspectives covered in process mining during process discovery:
- Control flow perspective
This perspective is focused on controlling the flow or order of the activities. Finding, exploring and modelling a comprehensive depiction of all possible paths of an activity flow is the main goal here.
- Organizational perspective
This perspective is focused on the interconnectivity of different resources (system, people and departments) in an activity, their roles involved, and how they are related. The goal here is to achieve an organized structure of system and people in terms of roles, accountabilities and answerability's.
- Case perspective
This perspective focuses on the properties of cases. Usually, a case is defined as per its path in the process or the individuals working on it. However, the values of the corresponding data elements can also be used to characterize the cases.
- Time perspective
This perspective is concerned with the frequency of events and their timings. We can find bottlenecks, monitor utilization of resources, measure service levels and predict the remaining time of a currently running process if the events have timestamps.
Roadblocks in Application of Process Mining
Process mining is a very useful tool but when it comes to its application, a number of difficulties are faced by the management since data is needed to be perfectly aligned to ensure risk management, compliance, conserving financial resources, exploring ways to foster business agility, conserving business agility and adapting to change. Some of these roadblocks are:
Dealing With Complex Event Logs
- Event logs may vary in their complexity from simple to intensely complex. Several event logs may be too complex to handle which call for additional efforts for their analysis.
- Sometimes low level events might not give insights, hence, we might have to aggregate low level events into high level events.
Dealing With Process Change And Data Flow
- A very common problem is that certain processes might be changing while they are being analyzed.
- In such cases, additional efforts are required to assess and analyze the additional data flow.
Finding, Merging And Cleaning Event Data
- Relevant data is distributed across a number of different sources. It makes it difficult to collect data.
- Event data might also be incomplete and in many cases, incorrect. This can lead to inaccuracy in analysis process.
- Some external situations which are not in control of the organization can make it difficult for data retrieval. For example, rainfall causing network and server problems.
Cross-Organizational Process Mining
- Event logs of a number of organizations are needed to be analyzed and merged. Such a cross organizational process mining requires new and on-the-spot analysis techniques to be developed which consider certain factors like privacy and security issues.
Trustworthiness And Understandability Of Data
- The user may face problems in understanding the result if it is not presented in an easy to understand manner. The user can also infer incorrect conclusions.
- The trustworthiness of the results should always be clearly stated as many-a-times, the data might not be adequate to make solid conclusions.
Process mining is quite adaptable in any type of business in any industry. Even within any business, it can be applied to any point and any level in an operation.
Process mining shows equally well results in any business or industry and can be applied at any point or any level, be it a process, an operation or an end to end through an organization. A typical process mining project has to go through four stages:
This stage is dedicated to planning, scoping and reviewing. During this phase, the initial analysis stages and goals such as period of the analysis, business questions needed to be answered, processes in scope, analysis timeline and team composition are stated.
This stage is concerned with data extraction and data connection. During this phase, things like the type of data required and scope of data extraction is stated and the relevant data is retrieved. Various systems and tables that are needed to be retrieved, the logic behind data, the flow and granularity of data, all of these aspects are needed to be considered, should be carefully collected and connected at every stage.
Certain situations can arise wherein the collected data is not directly usable for process mining so the it is refined and transformed. Data processing may be executed several times to enable a specific analysis, depending on different analysis questions.
a. This is the main phase wherein the actual process mining and analysis is done. First, various process mining methods are applied in this phase to answer the analysis questions. Process enhancement, process analytics and analysis, process discovery and conformance checking are some of the process mining techniques.
b. Secondly, analysis work is done to provide improvement ideas which can help in achievement of project goals. This includes the correct interpretation of the results. It is important that the interpretation is validated by verified domain experts and professionals.
Sustenance And Monitoring Stage
This is the final stage and is concerned with process monitoring, supervision and improvement. The actual insights obtained from all the previous stages are used in the modification and improvement of the process. And finally, the process mining tool is used to constantly monitor the process in order to identify any unplanned occurrence..
Process Mining And Process Analytics
The combination of process mining and process analytics can be used to efficiently discover and monitor business processes spanning up-to multiple systems and organization silos in the following way:
Use Of Process Mining In Discovering And Optimizing
Process mining automatically constructs a model of the analyzed event information from all systems contributing to a single business process by synchronizing the events of each operation moment so that each one of its activities become traceable between all systems that impact the moment. Each separate moment will create an operations model that can trace its activities that will combine the information of these instances and as a result, there will be an overall end-to-end operations model for that business process.
According to UBM Techweb white paper, "Process mining can be used for more than discovery of the process model, or flow diagram: it can pinpoint process bottlenecks, rework and inefficiencies, prompting changes to both manual procedures and the information systems that make up the business process. It can identify differences in business processes between business units or other data dimensions in order to understand anomalies."
Use Of Process Analytics For Visualizing And Taking Action
Process analytics has the ability to analyze and visualize information about processes while they are being executed in order to provide actionable information and hence, it is a critical functionality of BPMS. According to UBM Techweb, " The techniques and tools of business intelligence (BI) are well-understood by most organizations, and process analytics puts a process-centric spin on that: since processes are both time-and task-oriented, different views are required beyond standard BI dashboards to properly visualize the flow of work over time as well as the process data."
Real-time monitoring of process performance may include simple dashboards showing aggregate information, such as a chart of how many process instances are waiting at each step in the process at this time; more complex interfaces that allow the operator to drill into a specific instance; or generate real-time alerts at certain thresholds to allow people to take corrective action.
The process performance being monitored on real-time may include a complex interface that allows the operator to dive into a specific instance, simple dashboards representing aggregate of all information, charts showing the number of process instances and their chronology, or create real-time alert systems at certain levels to allow the concerned staff to take corrective action.
Process Mining: Examples And Use Cases
To understand the value added by implementation of process mining, one needs to look not only at the theoretical part but also at how numerous companies have been benefitting from it by overhauling their processes and features. The following use cases will help you for this purpose:
A Finance Use Case: Process Mining Helps APG Solve Their Pension Problem
The perspective and exchange of money are rapidly changing, from fintech startups valuing billions of dollars to contactless payments and cryptocurrencies, also in terms of usage. Even pension funds, considered to be the least exciting side of the financial sector, are finding new markets through process mining technology.
APG's website has all the information that a client needs. The website was genuine, whether the client is a 60 y-o who is going to become a pensioner or a 24 y-o college student looking to secure his future with a plan, they all could easily understand when and how they will receive the benefits. Content category, digital aspects and user profiles were mapped in a way such that there was no need to provide external support. However, the as-is process was quite different. APG identified and fixed this problem using process mining.
Even with the convenient features, APG noticed that some particular groups were taking a lot of time surfing the same pages, maybe they were finding some kind of difficulty in navigating. For example, a divorcee trying to understand her rights and financial obligations after investment. It took 4 to 5 calls with the customer support to satisfy her doubts. This is where APG brought process mining into the picture. APG used it to identify such groups, traced their digital footprints and provide the solution in such a way as to ensure that these people do not require customer support service to understand their needs.
A Retail Use Case: Walmart Launches A New Feature To Reduce Time And Efforts For Purchasing Garden Products
Walmart, being the world's largest retailer, has over 20000 brick and mortar stores in more than 25 countries. It also spent the last five years making the world's largest cloud network. This cloud will compete with Amazon's and process more than 2.5 petabytes of data every hour.
But Walmart may not achieve such a scale without using process mining. “If you can’t get insights until you’ve analyzed your sales for a week or a month, then you’ve lost sales within that time,” says Senior Statistical Analyst Naveen Peddamal. Walmart says that big data, when supported by process mining, can be used as an essential component of their strategy to understand their customers in a better way and help themselves make better decisions.
This improvement is being adopted in more than 350 U.S Lawn and Garden Centers where the company identified this need of process mining in operations for even more seamless and faster payment and checkout experience. Earlier, the customers who came to buy heavy products, for example, 60 pounds of lemon tree, needed to bring the load in the store to be checked at a fixed account. But now, the system is reversed. Now, the account can come to them with the feature "Check Out With Me" launched in April 2018. This is Walmart's attempt to cut the monotonous and time consuming task of purchasing products in the way mentioned above. Customers simply put the necessary items in their cart and in the parking area, a checkout attendant scans and finalizes the purchase after payment.
By using process mining, specific time delays in processes are also pinpointed. Process mining helped in process improvement by checking the time fluctuation in checkout and over-all transaction and decide if the customers are being satisfied by this feature to further decide if the feature needs to be launched globally or should it stay domestic.
How Process Mining helps companies in optimization of their BPM life cycle?
Traditionally, tools of business process management rely on inputs provided by employees and other experts which is used to improve and model while business intelligence tools are used to evaluate and monitor KPIs. Here, process mining works as the link between BPM and BI.
Process mining covers all stages of the Business Process Management life cycle from the initial analysis and modeling to the implementation and monitoring of refined processes. Process mining's role in optimization of BPM life cycle can be understood in the following way:
One of the biggest problems in any BPM operation is finding out how the actual process looks like. Optimization processes cannot unleash their full potential unless the as-is process is modelled correctly. If process discovery is done by traditional approach, i.e. through workshops and interviews, there is a great chance of inaccuracy, misconception and gap while taking the time of employees.
This problem can be overcome by using process mining as it uses data to trace the process through the system and visualizes the process. The results are not just mere theories but objective and fact based and includes all variants and deviations and arrive within a reasonable time.
Process mining tool helps in mapping out the process and makes it much easier to determine the areas where changes are needed. Aspects like waste, bottlenecks, rework, lead times and other variants become quite apparent after that. Process mining goes even further and determines the root cause of the problems. Some process mining tools even walk the extra mile to allow companies to model their processes in they way they desire.
Process mining tools provide insights are derived from Business Intelligence. They allow people to truly understand the findings and the changes needed as they are presented in an understandable manner by minimizing resistance to change, using easily understandable visuals and showing involved people, directly and indirectly, in the optimization process.
It needs to be ensured that the processes that have been changed and improved are having the desired effect. Process mining again comes handy in this situation. It allows tracking of the change and the achieved success is measured and compared with a benchmark. It also allows live tracking of the process. If some changes are needed to be done during that, then data can be automatically or manually updated during the execution.
Continuous data driven process improvement is enabled by process mining. As several inefficiencies and bottlenecks are discovered, they can be constantly measured, compared and improved. Business decision makers gain deep insights and hence, form a particular optimization strategy according to the problems and are able to spot other minor inefficiencies before they into major ones.
- Sandy Kemsley, Enabling process intelligence through process mining and analytics, UBM techweb.
- Win Van Der Aalst, Process Mining: Overview and opportunities, ACM transactions on MIS.
- Ken P Hammond, Process mining and its impact on BPM, EY.
- Jason Bloomberg, Process and performance mining in the digital era, Intellyx.
- Win Van Der Aalst, Process mining: data science in action.