Real-Time for Business Intelligence(RTBI)

DEFINITION

The Concept of Real-Time Business Intelligence(RTBI) [1] refers to the ability to provide information on business operations as soon as they occur.

Although RTBI is oriented to real-time operations, it does not mean that it cannot provide the classic strategic functions and Data Warehouses. It supports both perspectives perfectly. Therefore, it serves as a replacement for these, since it replaces the classic data warehouse with one with real-time response capabilities, and enterprise application integration (EAI).

It is very important to decide before its implementation if it is really what the company in question needs. The cost of setting up an RTBI system is very high and it is more than likely that the company does not need responses in real-time. If it is possible to work correctly with a system that operates somewhat less quickly than in real-time, the implementation of a system with these characteristics should be ruled out.

The main function of any RTBI is to offer information that can be used to improve key aspects of business processes as well as take advantage as soon as events occur.

CHARACTERISTICS COMMON TO RTBI SYSTEMS

Typically, a traditional Business Intelligence system requires a few hours to implement the actions that were decided based on the events that were received. But this is not a characteristic of a real-time Business Intelligence system, where speed is taken to the extreme of real-time is required. The actions to be executed by a BI system must be carried out in a second, or even less time, and for this to be carried out, the data warehouses can’t be updated every several hours. 

For these reasons, for a system to be 100% pure RTBI, it must be constantly updated, and this is achieved by reducing the latency with which operational information is accessed at a time close to 0 seconds. Generally, the type of information update is carried out by dripping operational transactions, as they occur, they are stored in the warehouse, and from that moment they are already useful for use. In addition to real-time updating, it is convenient for a decision-making system to be able to analyze all the information as it arrives at the warehouse while generating a decision quickly and effectively. All this is considered an RTBI system.

GOAL OF AN RTBI SYSTEM

Every RTBI system aspires to reduce as much as possible the latency with which the system processes business operations until a notification or corrective action is produced from the Business Intelligence system in question. In RTBI systems, 3 types of latency are:

• Data latency: The time it takes to collect and store information.
• Analysis Latency – The time spent deciding how to act based on the information received.
• Action latency: time used to perform the action decided by the system.

REAL APPLICATIONS

There are various fields in which an RTBI system is very useful for the business to which it is directed. That is why every day more and more efforts are invested in making these systems as fast as possible, to save costs and other greater evils, due to inaction or action out of time. Areas in which they are widely used:

• Fraud detection
• Monitoring systems
• CRM software
• Operational intelligence
• Control of payments and collections
• Data security monitoring
• Call center optimization
• Transport industries
• Sales forecasts
• Portfolio management

RTBI ARCHITECTURES

There are four large, widely extended families of RTBI systems, which are the following:

• Event-based: The main feature of this branch is to employ techniques that allow events to be parsed without first being stored in the data store. Because of this, the responsiveness is typically in the millisecond range.
• Highly updated data stores: This branch of RTBI is based on performing a large number of refreshes of the information that is in the data stores so that the data in them is as recent as possible. Typically, data latency is anywhere from a few minutes to not much more than a couple of hours.
• It does not imply a completely real-time system, but in its favor, it has the greatest simplicity and the lowest cost compared to a pure real-time system.
• Serverless Technology: Recently, an RTBI system model MSSO/SD4E has been developed, allowing to dispense with the server where the data warehouse information was normally stored, as well as intermediate servers. This is so because the sources from which the system is fed are accessed in real-time. In addition, these data sources can be completely different from each other, and must not strictly be in the same place. By directly accessing the information, the latency time is close to zero.
• Process-Aware – The latest type of RTBI system allows processes to be monitored, metrics to be visualized, compared to historical data from data warehouses, and all in real-time. This type is closely related to business activity monitoring (BAM) software and Operational Intelligence systems, which is a form of real-time analytics.

This architecture focuses on offering a quick response as events occur that may require a response from company administrators. This requirement arises due to today’s need to maintain profitability and competitiveness in companies. That is why software is needed that can monitor changes and trends that indicate opportunities or problems and that allow the right people in said companies to take action to change the course of the company. For all these reasons, the Gartner company coined the term Business  

ActivityMonitoring or simply BAM. [2] BAM systems work by capturing events (predetermined in advance, which can be of many types, depending on the type of company or its department) from operational systems. For a system of these characteristics to work as expected, it is essential to be able to delve into the details of the operations, as well as quickly obtain the information of said events. Otherwise, this could incur problems of various kinds for the organization, ranging from the loss of customers key or malfunction of the system or part of it, to give a few examples.

However, BAM systems are not a replacement for data warehouses but are used in conjunction with them, each doing its job. Data warehouses are intended to store historical information to subsequently perform analysis on said information and obtain relevant data for the improvement of business activity.

The architecture of a BAM system. On the other hand, BAM systems are used to provide updated information and a set of business rules that direct the actions to be taken for the events of the operational functions of organizations (supply chains, logistics, etc.). For the BAM system and the data warehouse to work in coordination, both parties must be highly coupled with each other, so that the warehouse offers historical information and that the BAM system uses this information in the framework of the event-based intelligence system.

BAM systems are suitable for those organizations and companies that need operational information in real-time. Some examples are:

• Financial services: they range from portfolio management, systematic trading, fraud detection, risk management and control, real-time or interactive customer marketing, and finally, anti-terrorism laws that exist in various countries
• Manufacturing: management and shipping forecasts, sales forecasts, product recalls from a market, and quality control.
• Retailers: Real-time inventory analysis, real-time marketing and promotions, and product recall.

The main characteristics that make up the operation of a BAM system are the following:

• Capture real-time data that is event-driven and obtainable from a wide variety of sources.
• Calculate temporary information by comparing information over time from event streams (this historical information can be used as thresholds to determine whether or not to perform certain actions).
• Develop dynamic modeling by integrating contextual and event information on the fly and producing suites of analytical models.
• These models can be updated through continuous interaction with the system.
• Perform business rules for the creation of thresholds based on KPIs and other determining information.
• Provide a system that has a clear, simple, and friendly interface that constantly updates the organization’s metrics.

A highly updated data store is nothing more than a data store, although with the proviso that the information it contains is expanded each time the associated information from its sources changes. That is why it is useful in the field of RTBI. The basic uses of a data warehouse are well known, and they are:

• Reducing the load on transactional databases
• Data simplification
• Storage of historical data of the company in question

The reasons that can lead an organization to maintain a data warehouse are several, among them, is the centralization of the information for its access from different departments of the same. And it is this reason that forces us to update the information in the warehouse in real-time since it is necessary for the proper functioning of the company that the information is correct and up-to-date in all systems.

To achieve this challenge, it is essential that the data warehouse can access the different sources of information and check the changes when they occur to update it.

There are various alternatives for this, but the most widespread is updating after a successful transaction. Therefore, the system will supervise the transactions that are carried out in the data sources and will modify the data warehouse as it deems necessary, and since there is little information to update, the transactional systems will not suffer in terms of efficiency. Of course, there is a latency time, which will depend on the requirements of the company and/or the characteristics of the systems that interact with it.

Typically, refreshments are produced in the range of a few minutes to an hour.

Scheme of a highly updated data warehouse

As has been commented on serverless technology is oriented to Business Intelligence, its objective is to dispense with servers to access data directly from where it is stored. For this, a technology called Multiple Source Single Output / Structured Data for Excel (or MSSO/SD4E) was developed, and all this is possible from any computer with the Excel application installed.

This technology aims to show the benefits that Excel can have, as a real tool oriented to Business Intelligence, so that it can manipulate and analyze data in real-time, from the same sources, in a simple and fast way. The scalability is virtually unlimited and the data flow can be controlled seamlessly as well as being secure.

MSSO is the architecture that is dedicated to obtaining information from different sources, for later, and thanks to the use of the SD4E application, is presented to the Microsoft spreadsheet application and it recognizes said information so that I can process it with no problem. Until the arrival of this combination of applications, the fundamental problem that Excel had as an application for RTBI was obtaining it easily and easily understandable by Excel, in addition to the difficulty in maintaining the integrity of the data and the security of access to the data. themselves.

MSSO/SD4E architecture

The simplicity with which the MSSO/SD4E technology works in such a way that it allows enormous savings for reporting, getting to do in hours what other tools do in days, and minutes what others do in hours. All these functions have added the possibility of including much more information from other sources whenever you want, without having to pay for these modifications to be made, since technology allows it in a completely natural way, but above all, simple.

Data corruption is prevented thanks to MSSO DataGuard technology, which stores source information without any alteration, as well as marking it to easily recognize which source it came from and thus keep it up to date. All the modifications that the users on Excel produce will be safe and will not be transmitted to the databases so that there will be no data corruption.

Regarding the security of access to information, and given the nature of the technology in question, it takes advantage of the security that databases have and do not require any additional security layer to prevent unauthorized access, since security credentials are identical. Not to mention the lack of need to introduce additional layers of security on intermediate servers, since the data is always accessed from the same sources. In short, the MSSO architecture and its associated technologies make a combination that allows any user with Business Intelligence needs to use an environment as familiar as Excel, as well as completely secure and reliable, as well as powerful and versatile. Thanks also to the unlimited number of fully customizable dashboards

Information Systems – PAIS, are not exclusively dedicated to Business Intelligence systems, but cover much more, from workflow control systems, case management systems, business information systems, etc.

Broadly speaking, a PAIS system is dedicated to providing support to business processes and the context of the organization that surrounds them, and they are the evolution of workflow management systems. PAIS systems are closely related to data mining processes they are based on these processes to discover new business models or to verify the consistency of existing ones. This is because the use of data mining has shown that there are large differences between the ideal models used to configure business systems and the actual processes of the companies. companies. 

To a large extent, this occurs because there is no single model applicable to a specific situation, rather there are several completely valid ones. Traditionally, the software has been developed in such a way that it is oriented to perform a set of tasks, however, PAIS systems have been developed to support processes. This change in point of view offers a series of advantages over the old paradigm, and they are:

• The use of explicit process models provides a way of communication between people.
• Model-driven systems rather than code-driven systems have fewer change issues.
• For example, if an information system is driven by process models, it is only necessary to change the models.
• Explicit representation of processes under an organization allows for automated disclosure, therefore it could have an impact on better performance.
• The explicit representation of processes allows management support at the design level as well as at the control level.

It should be noted that PAIS systems can be designed taking into account who or what will control the processes. In other words, depending on whether the system interacts with people or with devices, the system will be classified into one group or another. These groups are:

• Person-to-Person (P2P ) Processes: the parties that interact with the processes are mainly humans. 
• Examples of P2P processes are job tracking, project management, or collaboration tools. Generally in these processes, the supported processes do not use any automated task.
• Application to Application (A2A ) Processes: As the name suggests, these processes are fully automated and machine-driven.
• System transaction processes, EAI platforms, and web servers are common of this type.
• Person-to-Application Processes (P2A ): this group of processes requires both machines and people for their proper performance.
• Workflow systems are in this category since it is precisely intended that people and applications work together.

Distinctions between different types of COUNTRY.

To carry out this segmentation between the different types, it is necessary to define the variables that are analyzed. A process is said to be unframed, when there is no process model associated with it, as is the case of collaborative processes supported by collaborative systems that do not offer the possibility of defining process models.

A process is ad hoc framed if there is a process model defined a priori but it is only executed once or a few times before being deleted or modified. The case that occupies this category encompasses project management environments, as well as distributed computing environments. 

A loosely framed process is one in which a process model and a set of constraints have been defined, such that the predefined model describes the normal way of doing things while, albeit with certain liberties, defining limits in advance (the constraints ). And finally, a tightly framed process is one in which a process model exists and does not allow modifications to it.

Finally, and as already mentioned, data mining processes are very useful for verifying the reliability of business processes, since the difference between what should happen in a process and what happens seems to be quite noticeable. For this, it is common to use the event logs produced by the information systems and extract the information for said validation from them. This is thanks to the fact that it is supposed to be possible to discern to which activity and to which particular process each of the events collected in the log files refers.

Which, the objective of data mining for PAIS systems is to monitor and improve processes, and for them, there are 3 ways to do it:

• Discovery method: there is no a priori model, so it might be necessary to generate it through the logs using algorithms for it.
• Conformance Method: there is a defined model and it is used to verify that what is extracted from the records is indeed what should happen and vice versa.
• Extension method: a model exists and it is extended with a new perspective since the objective is not to check what the Conformance method does but to enrich the model itself.

As a general rule, the models do not completely agree with what happens in reality, which is why the use of data mining is insisted on to corroborate the correct functioning of the processes. In addition, multiple factors affect the development and use of PAIS systems, such as the difficulty of capturing without errors the characteristics of the people who interact with these systems, to name one of them.

CONCLUSIONS

Throughout this investigation, it has been tried to publicize the benefits of Real-Time systems oriented to Business Intelligence, however, it is not easy to obtain the problems that each of the different architectures proposed by the various associated companies entails. So, from a completely subjective point of view, we are going to proceed with a comparison of them.

Broadly speaking, the architecture could be
defined as shown in the following table:

Through this comparison of advantages and disadvantages, it is easy to conclude that not everything is as simple as it seems at first glance. In other words, RTBI systems are not easily implementable and, of course, they have problems that must be taken into account when preparing the chosen system.

It should be noted that the choice of architecture does not prevent the adhesion of certain characteristics (but all in some cases) of others. For example, architectures based on events and those of highly updated warehouses could coexist, with hardly any problems.

Although it may be necessary to make certain aspects of them independent for their proper performance.

Therefore, we have before us the dilemma of choosing what type of architecture is suitable for the company to which we are going to implement the RTBI system. To make such a decision, it would be necessary to obtain an estimate of implementation costs, and above all, difficulty when installing it.

Looking at serverless technology, we might think that it is the panacea for our problems since it is supported by a widely extended application, especially by people related to Business Intelligence. But on the other hand, we have the great drawback of using a program like Excel, and that is that even though it looks like it is one of the perfect alternatives, it must be admitted that it has certain limitations that are not easy to solve since it is not possible to modify its code.

Thus, we end the conclusion by noting that there is no perfect system for everyone because it is essential to make a correct report that shows all the variables that affect given the implementation of said systems. And it will be only after the analysis of said report that it will be possible to affirm which is the most appropriate system for a specific company.

REFERENCES

• Gravic, Inc.
• http://www.gravic.com/shadowbase/uses/real-timebusinessintelligenceintroduction.html
• Event-Driven Architecture (BAM):
• http://www.information-management.com/issues/20040301/8177-1.html
• Architecture based on
• highly updated data warehouses:
• http://www.uintelligence.cl/attachments/1163_5814_sieteestilos.pdf
• MSSO Technology:
• http://research.tomsitpro.com/whitepaper760
• Conscious processes:
• http://wwwis.win.tue.nl/~wvdaalst/publications/p522.pdf
• Real-Time Business Intelligence:
• http://www.gravic.com/shadowbase/uses/realtimebusinessintelligence.html

Author:

Carlos Alberto Martinez