Key Performance Indicators

Business Alignment

Do the BI asset components reflect the goals and plans for an organization? Does Management understand what can be achieved? How can the BI product be carved to fit the specific needs of an organization?

In addition to clear and widespread communication of business goals, there are other important business alignment factors:

1. Understanding potential uses and forms for BI products is a prerequisite for realizing the greatest benefits. By this we mean, "If you don’t know you can get it, you won’t ask for it."
2. BI is, in most cases, aligned to the needs of the business. However, only a fraction of the potential benefits will be realized if there is only a one-way alignment. Traditional operations can be transformed into streamlined, information-based processes if the organization is open to revising these activities to conform to the realities of the BI product.

Assessing business alignment issues can be more complex than other parts of TBIA. In part, rating an organization will be dependent on the comprehension and creativity of TBIA team.

Business IT Partnership

Knowledgeable business resources must work hand-in-hand with IT specialists. If one or the other tries to go it alone, the consequences are often severe. Silos of data too often result from isolated business efforts. If IT builds alone, then the product often does not reflect the business needs and no one comes to the party. Assessing this partnership requires review of the BI development activities from the initial requirements brainstorming through the implementation and support. The success of the final BI product will reflect the extent of this partnership.

Integration

Integration may refer to any number of subjects: from data and applications to user interfaces. Data is of most interest in the BI arena. Data integration for BI is usually addressed using federation and/or a data warehouse. Data federation refers to a virtual integration of several different data sources, usually managed by overlaying a map of the data over the physical data sources and treating them as a single entity. One of the major limitations here is the collection and storage of historical data. A data warehouse solves the history issue. The data warehouse includes the platforms and processes that collect data from multiple sources and unifies it into a single library of information. Assessing how the integration occurs and how well it is accomplished are primary goals of TBIA.

Scalability

BI systems must support large numbers of users who interactively access the same reports and data. BI systems must support massive amounts of data. Scalability refers to the capability to meet the requirements of the organization in terms of the number of users, type and number of reports, applications and data volumes. It also refers the capability to support rapid increases in data, queries and users – quickly, effectively, efficiently and with no downtime to the systems. Assessing scalability requires that the audit team review previous results of changes to the numbers of users and data volumes. For example, have there been system slowdowns and/or breaks and are there pain points which can be identified? This audit category also will require close assessment of the technical infrastructure, including hardware and tools. What are the vendor guarantees and suggestions? How are we estimating for the future and how close are we measuring related statistics?

Adaptability

Much like the scalability KPI, Adaptability refers to the capability for rapid response to the changing needs for BI. All the BI asset components must be designed to be flexible. They must have the ability to grow and evolve along with the organization understanding and requirements for BI, as well as, for changing technologies. This means, for example, that a data warehouse staging library should be able to accommodate new files and categories of files as technology changes. It means that, not only should we be able to quickly react to changes in volumes of users and data (i.e., scalability), all the BI asset structures should be designed for rapid changes due to new and evolving technologies and product requirements. We should be factoring in such possibilities as new acquisitions and other changing business requirements. New paradigms for creating and managing BI are difficult to plan into a current system architecture. However, much of the standard growth and change can and should be anticipated in the design and development of the BI assets. In TBIA, we need to review specifics of the infrastructures as well as how the BI applications are designed and developed. Are requirements focused only on specific current requirements? Are broader business goals and forecasted requirements also included?