Chapter 6: Foundation of Business Intelligence - Databases and Information Management

Bit

The smallest unit of data representing a 0 or 1

On is a bit off is a bit

Byte

A collection of 8 bits

Binary

A collection of 0 and 1

Takes 8 bits and there is a combination for each letter of the alphabet

Nibble

A collection of 4 bits (half a byte)

ASCII

American Standard Code for Information InterChange

File Environment data Management

Byte

Field

Record

File

Database

Entity

Attribute

Byte

A single character of data

Field

Group of characters as word(s) or number(s)

Record

Group of related fields

File

Group of records of same type

Database

Group of related files

Entity

Person, place, thing on which we store information

Attribute

Each characteristic, or quality, describing entity

Traditional file processing

Database management systems (DBMS)

Software that enables an organization to:

• Centralize data

• Manage that data efficiently and

• Provide access to the stored data by application programs

DBMS and Relational DBMS

A table is a grid of columns and rows

The structure of a database includes:

• rows: Records for different entities

• fields (columns): Represents attribute for entity

• key field: Field used to uniquely identify each record

• primary key: Field in table used for key fields

• foreign key: Primary key used in second table as look-up field to identify records from original table

DBMS Capabilities

3 basic operations of relational DBMS

1. SELECT- creates subset of data of all records that meet stated criteria

2. JOIN- combine relational tables to provide user with more information than available in individual tables

3. PROJECT- creates subset of columns in table, creating tables with only the information specified

SQL query writes code - AI can write code now for people

Access query - easier than SQL

Cloud databases

• Appeal to start-ups, smaller businesses

• Amazon relational database service, Microsoft SQL azure

• Private clouds

Distributed Databases

• Stored in multiple physical locations

• example: google spanner

Blockchain

Transaction recording: when a new transaction occurs, it is recorded as a data block containing details like sender, receiver, and timestamp

Verification (consensus): before a block is added, the network of computers must agree it is valid using a consensus mechanism (ex: proof of work)

Harshing: each block contains a unique digital fingerprint called a “harsh”, plus the hash of the block before it

Decentralized

No single entity controls the data. It is maintained collectively by the network, removing single points of failure

Immutability

Once data is confirmed and added to the chain, it is mathematically nearly impossible to delete or change

Transparency

In public blockchains, all transactions are visible to anyone, though the identities of the participants remain encrypted and private

Big Data

• Massive sets of unstructured/semi structured data from web traffic, social media, sensors, and so on

• 3 V’s: volumes, variety, velocity

• Value: benefits of fata and veracity, quality and accuracy

Big data and AI synergistic Relationship

• AI requires a massive amount of data for training

• Machine learning is able to use the information provided by big data to generate more valuable business insights

Data Warehouse

Database that stores current and historical data from many core operational transaction systems, data cannot be altered

Data Mart

Subset of data warehouse, single subject or line of business

Data Lake

Repository for raw unstructured or structured data not analyzed

Hadoop

Open-source software framework managed by the Apache Software foundation. Enables distributed parallel processing of big data across inexpensive computers

In-Memory Computing

Uses computers main memory (RAM) for data storage to avoid delays in retrieving data from disk storage. Reduces processing time

Data-analytic Platform

High-speed platform using both relational and non-relational tools optimized for large datasets

Online analytical processing (OLAP)

Software technology used to rapidly analyze large volumes of business data from multiple perspectives

Data Mining

The process of using software, algorithms, and statistical methods to analyze large datasets to discover patterns, correlations, and anomalies

Classification- clustering - association rule learning - regression - anomaly detection

Text Mining

Tools to extract key elements from unstructured natural language text, discover patterns and relationships, and summarize the information

• Sentiment analysis: involves mining comments in an email, to detect favorable and unfavorable opinions about specific subjects

Web Mining

Discovery and analysis of useful patterns and information from the web

Data Governance

• Encompasses policies and procedures through which data can be managed as an organizational resource

• Identifying personnel who are responsible for sharing, disseminating, acquiring, standardizing, classifying and inventorying information

• EXAMPLE: firm information policy that specifies that only selected members of a particular department can view certain information

Data Quality

Data should be accurate, timely and consistent

More than 25% of critical data in Fortune 10000 company database are inaccurate or incomplete

Data Quality Audit

A structured survey of the accuracy and level of completeness of the data in an information system

Data Cleansing

Also known as data scrubbing, consists of processes of detecting and correcting data in a database that are incorrect, incomplete, improperly formatted, or redundant