What is process mining?

Process mining is a software-supported method for analysing and optimising business processes. It utilises data from IT systems to make process flows transparent, uncover bottlenecks and increase efficiency. Visualisation and data analyses enable processes to be understood and improved.

Commercial enterprises want to be efficient, i.e., to achieve a specific economic goal that is as economical (or waste-free) as possible. To achieve this, they must consistently monitor the input and output of one or more combined activities.

Things get complicated when several people, departments, and IT systems are involved in a single process. Process mining is a technique used to address this challenge and is suitable for many use cases.

Process mining vs. data mining – what’s the difference?

Process mining and data mining are powerful data analysis tools, but they differ in focus and area of application.

While data mining is dedicated to discovering patterns, trends and correlations within large volumes of data for forecasting and decision-making, process mining focuses on analysing and optimising business processes. It visualises and analyses workflows in companies using event logs, reveals weaknesses and highlights potential for improvement.

This distinction makes process mining particularly valuable for process optimisation and making companies more efficient, while data mining is used more broadly in data analysis and interpretation.

What are the advantages of process mining?

The benefits of process mining are obvious: systematically logged and regularly analysed insights can provide excellent insights for improving business process management.

1. Objective evidence of adverse developments that have already been subjectively identified
Often, teams are already aware of problems with a business process. With process mining, these challenges are documented quantitatively using concrete and quantifiable performance figures. For example: “Internal order processing takes 30% of the entire throughput time. This is too long.”

2. Identifying cost savings potential
Overlooked pain points can uncover additional cost-cutting potential. For example, process mining software is used as a customer support tool to identify whether a customer has already exceeded their contractually agreed support allowance and, therefore, needs to be charged an additional fee.

The benefits vary depending on the process being analysed. However, the typical benefits of automated process analysis are clear: faster throughput times, streamlined and standardised communication, greater transparency, improved customer service and – not to be overlooked – happier teams due to less frustrating processes.

HOW DOES PROCESS MINING WORK?

In order to be mined, processes must be digital. In addition to the typical benefits such as increased transparency, simplified communication, streamlined processes, and faster processing, digitalisation also lays the foundation for process mining. A ticket system, for example, digitalises the process of “asking the IT team for help”. Every ticket holds fascinating (meta/transactional) insights in its log file. Was the ticket set as high or low priority? What category was it filed under? When was it created (beginning or end of the week, time of day, etc.)? Which team member was assigned the ticket, and how long did it take to process? Were they able to handle the ticket independently, or did a colleague have to step in? Of course, some of this data will have to be entered by humans (e.g. priority or category). However, some is created automatically by the system (e.g. time of submission or handler). The additional automated information, in particular, is a benefit of a digitalised process landscape.

Next, the data is downloaded and analysed, so the mining or “digging” for new insights can begin. One thing is certain: the more thought that has gone into structuring the (meta) data in advance, the easier mining will be. Ultimately, one can only measure what has been collected ahead of time. If most tickets in the ticket system are falsely marked as urgent by the creator, it can make sense to introduce an urgency scale. After all, if all tickets are marked as urgent, then none of them are handled urgently. One benchmark for objectively determining urgency is to consider the ticket creator’s response time once the matter is resolved. If the ticket creator takes a long time to respond despite indicating the matter is urgent, then it can’t have been that urgent. It can, therefore, be helpful to make assumptions before the analysis as to where possible efficiency blockers could be hiding in the digital workflow. These theses are then progressively addressed during data analysis.

Data analyses can also be more or less detailed, which is where dedicated process mining technology comes in. These solutions automatically recognise structures and examine the data based on set questions.

Challenges and solutions in process mining

Dealing with data quality and integrity

A fundamental challenge in process mining is the quality and accessibility of the data. Only complete or correct data can lead to accurate analyses. Solutions include stricter data quality standards and advanced data cleansing algorithms. In addition, integrating different IT systems is crucial to ensuring a seamless data flow.

Resilience to change

Process mining can be met with resistance from employees who are sceptical about the resulting changes. To overcome this resistance, it is essential to communicate the benefits and involve employees in the process. Training and workshops can help ease fears and promote understanding of the advantages of process mining.

Process mining scalability

As a company grows, its process mining tools must grow with it. One challenge is finding solutions that can be scaled to accommodate increasing data volumes and complexities. Cloud-based solutions and the flexibility of mining tools are crucial here to create a flexible and scalable analysis environment.

Streamlining complex processes

Business processes can be complex and challenging to understand. Process mining must be able to map this complexity and make it understandable. One approach is to use a modular strategy that breaks down complex processes into smaller, manageable units. Visualisation tools also play an essential role in presenting process structures clearly.

Data protection and compliance

Handling personal data is a major challenge from a GDPR perspective. Companies must ensure their process mining strategy complies with data protection regulations. This often involves anonymising data and implementing security measures to prevent data misuse.

Implementing process mining in your company

It is essential to structure your process mining strategy in stages. Each phase is a crucial step towards success.

1. Scope and target definition

Before you embark on your process mining journey, you must define your project’s clear goals. Ask yourself: What do you want to achieve? Possible goals include optimising processes, reducing throughput times or improving compliance. You should also determine the project’s scope and decide which processes will be analysed.

2. Data collection and preparation

Next, you will need to collect the necessary data. This data can come from various sources, such as ERP systems, CRM software or other IT systems. The data must be complete, accurate and accessible. Once collected, the data will need to be processed for analysis, which may include cleansing, normalising, and consolidating.

3. Choosing the right process mining tool

There are many process mining tools out there. Choose the software that best suits your requirements. Consider factors such as user-friendliness, features, integration capabilities and costs.

4. Integration into existing IT landscapes

The first step is to ensure compatibility with your existing systems, such as ERP, CRM or SCM, as you will be mining data from these systems. Remember to consider interfaces and data formats. Once you have selected a suitable process mining tool that fits seamlessly into the existing infrastructure, you will need to connect the data sources. This phase can involve configuring APIs or using standard connectors to ensure your data flows smoothly.

5. Analysis and visualisation of the processes

Once the software has been implemented, the actual analysis begins. The software visualises your processes, highlighting deviations, bottlenecks and inefficiencies. These visualisations help you to understand process performance and identify potential for improvement.

6. Identifying improvement potential

You can then use these findings to develop measures for process optimisation. This could include automating certain steps, reorganising processes or training employees.

7. Implementation and monitoring

Once the measures have been defined, they will need to be implemented. It is crucial to continuously monitor and measure the changes by tracking them in real time.

8. Continuous improvement

Process mining is not a one-off project. It is an ongoing process. Regular reviews and process adjustments are necessary to stay efficient and competitive in the long term.

Resources required for implementation

Introducing a process mining tool requires careful planning and the availability of specific resources. High-performance hardware is essential, although this can be accommodated via the cloud. There also needs to be sufficient storage capacity and computing power to process large volumes of data efficiently.

From a software perspective, you need a tool that is compatible with your existing systems. Look for solutions that can be seamlessly integrated within your ERP, CRM, and other business systems. Choosing the right software is crucial, as it significantly impacts user-friendliness and the efficiency of your data analysis.

Another important consideration is access to high-quality data. The mining process thrives on data, so your systems must provide reliable and comprehensive information. Ensure your data sources are well-maintained and can provide the required information.

The value of specialist expertise should also not be underestimated. You need a team that knows your IT landscape and your organisation’s business processes inside out. Only then can you implement and customise your process mining software and use the insights gained effectively.

Finally, ongoing maintenance and data management are essential. Remember to allocate resources to regular updates, bug fixes, and possible software customisations. Continuous support ensures that your process mining system functions optimally and adapts to changing business requirements.

WHAT TO CONSIDER WHEN PROCESS MINING?

Whenever people work together within a typical digital workflow, there is potential for transfer errors, ineffective communication or – most importantly – unnecessary processing delays. Process mining can, therefore, be beneficial in all settings, whether it is the IT ticket system, the ordering process between manufacturer and supplier, a logistics process for shipping a consignment, an admin process such as allocating company cars or a complex food retail complaints process. The possibilities for using process mining are endless.

The variety of applications gives rise to various process mining types and associated challenges. The following questions highlight these considerations:

· Who is responsible for process mining and maintaining the related analysis structures? The IT department? Controlling? The business divisions?
· Is the process already digital? I.e., has the order already been placed via an online portal, or has it been placed over the phone? Can the partners involved provide any data (e.g., on the progress of a shipment)?

What are the suspected weak points? Is the logistics service provider reacting too slowly? Is the supplier able to ‘go digital’? Are tickets consistently and accurately logged?

What needs to be done to collect high-quality metadata digitally? Are the time stamps correct (e.g., the shipment left the warehouse in the morning but was not set as “shipped” until later due to a long lunch break)?

How can employees be motivated to stick to the agreed process? It happens all the time: a new workflow has been introduced, but the team hasn’t changed its ways. Tickets are still being passed to colleagues verbally or via email.
This list could easily go on, but it only addresses the ‘before’, not the ‘after’. Once the raw data is available, the following questions need to be considered: Who is responsible for interpreting and defining measures? What processes can be implemented to ensure data is not misinterpreted? What might be the reason for the logistics service provider’s long response time (e.g., poor network coverage at its remote location)?

Once these questions have been clarified, you can use process mining to align company processes with your specific business requirements.