Data Collection and Storage

Data Collection and Storage

Learning Objective

• After completing this topic, you should be able to identify the considerations involved in collecting and storing data.

Introduction

• Previous topic focus: Definition of data and various types available to businesses.

• Current focus: Techniques for data storage and management.

• Importance of data accessibility and quality for informed decision-making.

• Topic structure:

  • Data life cycle in a business.

  • Methods of storing data.

  • Detailed examination of data quality and associated challenges.

The Data Life Cycle

• Understanding the stages of the data life cycle is crucial for recognizing challenges in data collection and storage.

• Fundamental stages include:

  1. Identify: Recognizing necessary data for achieving business objectives.

  • Settlement on data sources, stemming from internal and external environments.

  • Involvement of a cross-disciplinary team:

    • Participants: Functional heads, end users, managers, information security specialists, business intelligence specialists.

    • Their task: Identify data needs and document decisions.

  1. Capture: Collecting and securely storing the identified data.

  • Data integrity is crucial: Ensure data is properly collected, up-to-date, and accurately replicated.

  • Trustworthy data ensures accessibility and usability by authorized users.

  1. Manage: Ongoing management practices to maintain data integrity.

  • Utilization involves turning raw data into meaningful information:

    • Analysis of raw facts leads to knowledge.

    • Knowledge is evaluated for actionable business insights.

  1. Utilise: Applying data to gain business intelligence.

  2. Archive: Safely storing data that is no longer actively used but may still be relevant.

  3. Destroy: Disposing of data that has surpassed its useful life to mitigate storage costs and legal liabilities.

Information Life Cycle Management (ILCM)

• Definition: Management of data from initial collection to final destruction.

• Policy integration into overall business strategy is essential.

Benefits of a Strong ILCM Policy

• Fewer risks and associated costs, including:

  • Reduced risks of using outdated or corrupt data.

  • Enhanced operational efficiency.

  • Improved accuracy in decision-making.

  • Cost savings through optimized data management.

  • Full data utilization enhances quality of management information.

Data Storage and Access Systems

• Database: An organized collection of structured data.

  • Facilitated by a Database Management System (DBMS).

• Data Warehouse: Curated database optimized for analysis of structured data across systems.

• Data Lake: Generalized storage for both structured and unstructured data.

  • Allows for creative analytics and faster query processing despite the possibility of slower processing times.

Data Modelling

• Practice of defining data types and their relationships for efficient processing.

• Finance professionals may assist in providing context for data models but do not model data themselves.

Cloud Computing

• Definition: Delivery of computing services through remote servers hosted on the internet.

• Benefits: Scalability and elimination of infrastructure costs, backup options, and resources relocation during outages.

• Software as a Service (SaaS): Accessing software products through web and mobile platforms.

Data Management and Governance

• Data Governance: Ensures accountability and integrity in data handling.

• Core objectives:

  • Improve decision-making and promote efficient data management.

  • Governance roles must be well-defined and processes standardized.

• Essential considerations within data governance include:

  • Data quality, ownership, roles, and responsibilities.

• A Chief Data Officer (CDO) is often appointed to oversee governance and strategic use of data.

Features of Data Quality

• High-quality data (data integrity) is characterized by:

  • Accuracy, accessibility, completeness, consistency, relevance, timeliness, uniqueness, validity, correct formatting.

• Preprocessing or data cleaning often occurs before data is processed into information (to eliminate errors).

Master Data Management (MDM)

• MDM focuses on maintaining authoritative versions of master data.

• Utilizes data owners and stewards to ensure quality and compliance with criteria.

Data Complexity

• Complexity affects storage and management efforts.

• Examples of datasets:

  • Low complexity: Simple records (student names, attendance).

  • High complexity: Comprehensive records (attendance, extracurricular details, GPAs, social media accounts).

Challenges in Capturing and Managing Data

Bias and Unintended Consequences

• Bias inherent in datasets can lead to skewed outcomes in analyses and predictions.

• Examples:

  • Amazon's algorithm favoring certain demographics due to biased initial data.

  • Issues arising from inadequate datasets leading to ineffective analytics.

Data Silos

• Definition: Isolated data held by specific departments, limiting inter-departmental access.

• Consequences:

  • Incomplete data utilization, redundancy in datasets, reduced organizational collaboration.

  • Recommendations include integration software and cultural shifts toward data sharing.

Cultural Resistance

• Challenge of encouraging collaborative practices among employees.

• Importance of top-down leadership in fostering a data sharing culture.

Costs vs. Benefits

• Analyzing costs versus benefits before advancing data initiatives is necessary, including:

  • Ensuring insights generated are actionable and valuable for the organization.

Conclusion

• Reviewed the data life cycle stages: capturing and managing data.

• Discussed storage techniques (data warehouses vs. data lakes), data quality governance, and challenges such as bias and silos.

• Emphasized creating a collaborative data culture for maximum insight utilization.

• Upcoming focus will be on maintaining ethical practices in data analytics, critical for finance professionals.