Business Intelligence Architecture Notes

COMPONENTS OF A BUSINESS INTELLIGENCE ARCHITECTURE

• Operational Systems: Starting points for most quantitative data in a company.

  • Also known as:

  • Transaction processing systems

  • Source systems

  • Enterprise Resource Planning (ERP) systems

Operational Systems in Detail

• Operational systems record data from daily tasks, including:

  • Supply Chain System: Records when products are shipped and order fulfillment details.

  • Sales System: Captures customer order details in an order entry system.

  • Manufacturing System: Logs production orders, quantities of raw material, and finished products.

  • Accounting System: Manages invoicing and payments, possibly in different operational systems.

Data Input Process

• Each operational task creates data for business intelligence (BI) usage.

• Users may need BI to complete tasks, e.g., ensuring product availability for orders.

  • Embedded BI: Displays options if a desired product combination is unavailable.

Operational Business Intelligence (OBI)

• Operational BI: Integrated with operational systems to support tasks.

• Embedded BI: Refers to reports or insights directly utilized in operational tasks.

• Operational systems can be custom-built or purchased from vendors like:

  • Oracle (e.g., e-Business Suite)

  • SAP

  • Microsoft (Dynamics GP)Operational systems can be custom-built or purchased from vendors like:

Oracle (e.g., e-Business Suite)

SAP

Microsoft (Dynamics GP)

Data Consistency and ERP Systems

• Systems should enable smooth data transfer between operational systems.

• Inconsistent data (e.g., differing customer IDs in systems) can pose issues.

• ERP Systems: Help standardize processes, reducing duplicate data entry and improving data quality.

  • Shared reference tables across modules are known as master data.

From Operational to Data Warehouse

• Operational systems provide the initial data for analysis; errors at this stage propagate into BI.

• Additional sources of data may include:

  • Distributors, advertisers, web logs, market research, social data, machine-generated data.

Extract, Transform, Load (ETL) Process

• ETL is the process of extracting data from operational systems, transforming it, and loading it into a data warehouse.

• ELT: New approach where data is loaded before transforming.

• Transformation involves cleaning inconsistent codes and incomplete data handling.

Enterprise Information Management (EIM)

• EIM encompasses various tools beyond ETL, including data modeling, data quality, and master data management (MDM).

• Metadata: Data about data, crucial for understanding and consistency in BI usage.

  • Technical metadata refers to where data is stored, while business metadata refers to definitions and calculations.

Master Data Management (MDM)

• MDM ensures consistent definitions of entities like customers and products across IT systems.

• It addresses product IDs across multiple applications to avoid discrepancies.

Need for a Data Warehouse

• A separate data warehouse is preferred when:

  • Cross-subject or cross-functional analysis is required.

  • Fast reporting and analysis are needed, as operational systems can hinder performance.

Data Marts

• A data mart can be a subset of the central data warehouse or a single subject area from various sources.

• Data marts cater to specific business requirements, sometimes bypassing the data warehouse.

Data Storage in Data Warehouses

• Data warehouses store data in relational database tables.

• They typically consist of:

  • Fact Tables: Contain numeric data for analysis (e.g., sales).

  • Dimension Tables: Allow analysis from various perspectives (e.g., time, product).

Big Data Technologies

• Big data utilizes various technologies, including:

  • Traditional data warehouses

  • Analytic appliances

  • Hadoop

  • NoSQL databases

• Hadoop: An open-source framework for handling big data, requiring files and pointers rather than structured tables.

Conclusion

• The evolution of technologies continues to shape the BI landscape, prompting organizations to adapt and innovate their data management strategies for optimal performance and insights.

• Understanding the components of BI architecture and data handling processes is vital for successful data-driven decision-making.