Enterprise Architecture Notes

Data Architecture Recap

• DMBOK (Data Management Book of Knowledge):

  • Considered obsolete for modern data environments.

  • Suitable for internal, structured data in early computing days.

  • Fails to address the complexity of diverse data types, management tools, and database structures today.

  • Modern environments involve topologies within and between organizations, extending across company borders with open data and distributed databases.

• Early Data Structures:

  • Emphasized simplicity with metadata describing database data.

  • Sought to maintain persistent customer data with restricted access.

• Modern Data Landscape:

  • Metadata is multifaceted, requiring multiple views for effective data utilization.

  • Internal data can be structured (standardized format) or unstructured (free format).

  • Free-format data is increasingly prevalent, often from external sources.

  • Metadata is essential for each data element.

  • Crucial to identify master and reference data for usage or maintenance.

  • Data originates from transactions, structured for reporting, and various content formats like streaming data.

• Information Governance:

  • Critical and challenging due to the complexity of modern data.

Data to Knowledge

• Standard Understanding of Data:

  • Elementary data is structured, defining meaning, format, timeframe, and relevance.

  • Information is interpreted data within a specific context (e.g., temperature data from Nordic countries).

  • Knowledge is derived from patterns, trends, and relationships within the data (e.g., winds, pressure systems).

• Data Creation Cycle:

  • Turning knowledge into structured, streamable data formats is crucial.

  • Data architecture design involves choices considering the customer, data producer, and subcontractors, aiming for a "win-win-win" scenario.

• Functional Perspective:

  • Data can be stand-alone or interoperable, requiring data matching.

  • Security, privacy, and ethical concerns are paramount in data design, considering data movement and associated risks.

• Data Reporting:

  • Involves maintaining and reporting data, assessing its usefulness.

  • Challenges include ESG, copyright, intellectual property, access, and use rights.

  • Modern data administration is more complex than early conceptions of simple data possession.

Components and Capabilities in Enterprise Architecture

• Focus on Business Purpose:

  • Align architectural design with business objectives.

  • Utilize reusable components for multiple purposes to enhance capabilities.

• Data Streaming Example:

  • Data pipelines stream data from various sources (cloud, streaming data, sensors, structured sources) to databases for querying.

  • Requires careful attention to data integrity during the extraction and transformation stages.

• Data Lifecycle Considerations:

  • Traditional thinking separates data from processes (as in DMBOK), leading to complexities. The life cycles of data are very different.

Data and Processes

• Encapsulation: Data and Processes should belong together.

• Traditional Approach:

  • Services/information systems linked to multiple databases, creating, retrieving, updating, and deleting data.

  • Changes require careful maintenance of interdependencies between databases.

• Modern Approach:

  • Data and processes are encapsulated, with data creation, maintenance, and archiving done on the fly.

• Data Flow:

  • Streams from e-commerce, data lakes, and unstructured sources are transformed to compatible formats.

  • Data warehousing ensures data correctness for reporting and machine learning algorithms (AI, predictive analytics).

  • Direct connections enable on-the-fly analytics and data exploration.

  • Data lakes process data before loading into data warehouses.

  • Parallel processes handle data storage, restoration, business intelligence, and dashboards.

• Maintaining Data Integrity:

  • Essential due to constant data streaming from multiple sources.

• Emerging Roles:

  • Data engineers: Responsible for data cataloging.

  • Big data engineers: Maintain connections between cloud services, databases, and streams.

  • Business intelligence analysts: Interpret data for actionable insights.

Building Blocks for Data Architecture

• Focus: Data to intelligence/knowledge.

• Context is Key: Data interpretation is challenging outside its original context.

• AI Considerations: Specialized AI systems require contextual understanding for data interpretation.

• Large language models generate text based on probability, without genuine understanding.

London, Weber, and Good Information Systems

• Criteria for Good Information Systems:

  • Domain describes the actual area of interest.

  • Data describes the phenomenon and changes in the environment one-to-one.

  • Changes in the environment are immediately reflected in the information system.

  • The system can report its status in a meaningful way.

• The Role of Data Architecture: Data ensures meaningful description of phenomena.

• Pragmatic Approach: Solutions should solve business problems and improve profits.

• Example: Master Thesis Status System:

  • The initial design reported status at year-shift, which was too late for corrective actions. Intervention is needed earlier (March, April).

  • Timing is a critical detail that can render a design obsolete.

Understanding Data

• Context is crucial: Citing Dahlberg et al. (2016) on using context in medical applications.

• Breast Cancer Example: Illustrates the complexity of processes and the need to leverage data effectively.

• Componentization: Requires simplifying architecture and relating building blocks to business sense.

• Shift Left and Security:

  • Components and artifacts should be embedded early in enterprise architecture.

  • Security, privacy, and compliance are fundamental aspects of enterprise architecting.

  • Shift left architecture aligns with infrastructure, microservices, and DevOps (continuous integration, testing).

  • Shortens the period to utilize data in an agile manner.

  • Emphasizes reading and understanding articles on this topic.

Distributed Ledger Technology (Blockchain)

• Example Use: Ensuring data originality and immutability.

• Addresses the Issue: Verifying the origin of data, especially in the context of AI-generated content.

• Applications: Healthcare record management, voting systems.

  • Enables building applications in a trustable way even when authorities change.

The Architect's Job

• Objective: Make more business and obtain better profits through architecture (02/2012).

• Architects Aim To:

  • Maintain development objectives.

  • Exceed set targets.

  • Create new business by combining assets innovatively.
    *In order to know if you are getting value of what you have you need enterprise data.

• Matching Requirements: Align strategy with business processes and develop necessary capabilities to optimize business performance.

• Business Models: Differentiate design to create diverse business models for different operating models and customer segments.

• Capabilities: Divided into subsets, with architects ensuring responsibility, ICT integration, and capability enhancement.

• Governance: Capabilities are developed to meet market requirements and strategic initiatives.

• Enterprise Architecture Facilitates:

  • Capability building.

  • Leveraging information systems.

  • Cross-disciplinary knowledge sharing.

• Architect Roles: Enterprise, data, solution, business, and security architects.

• Security Focus: Security is a top concern for enterprise architects design, according to recent reports.

Maturity Level Assessment for Enterprise Architecture

• Elements:

  • Strategy connection.

  • Governance.

  • Methods and processes.

  • Visibility of architecture.

  • Business alignment.

• CMMI (Capability Maturity Model):

  • Levels range from ad hoc (level 1) to optimized (level 5).

  • Level 5 integrates feedback loops for continuous improvement. Not so rare to have today.

EA Practice in Essence

• Holistic Approach: Translates strategy into defined execution, using analysis, planning, design, and implementation methods.

• Architect's Role: Mediates changes, aligns with benefits management, and coordinates with vendors.

• Key Views: Business infrastructure and roadmaps to implement strategies.

• Agile Integration: Value management office maintains alignment with objectives.

• Requirements:

  • Conceptualization.

  • Understanding complex concepts.

  • Simplifying problems.

  • Fluent communication.

  • Joint efforts in defining solutions.

Deliverables

• EA Document: Consists of diagrams explaining operations in reusable components, current state, future state, and design changes.

• Necessary Artifacts (Kotushev):Principles, capabilities, and technology reference models.
  *Models starts from data and from there process that data.

EA Tools

• **Functionalities: Process modeling, object/component management, connections to existing systems, dependency matrices, project portfolio management, and support for multinational teams.

• Repository: Stores metamodels and models.

• Meta Models Described Primarily in ArchiMate.

Evolution of EA Tools

• First Generation: Automated drawings and repositories.

• Second Generation: Business rules management, workflow monitoring, KPI integration, and standardized processes.

• Modern Tools: Expanding functionality with project management, risk assessment, and enterprise architecture management procedures.

• Gartner's Recommendations: Focus on the vendor's ability to execute and maintain the tool, with clear vision and product analysis.

• Interoperability is Key: Due to potential mergers or acquisitions.

• Standard Compliance: Togaf ADM provides starting points.

• Meta-Level Transformation: Ability to export descriptions in ArchiMate format.

• Agreed Practices and Responsibilities: Aligned with governance and maturity models.

Usefulness to Stakeholders

• Tool is obsolete if it cannot serve its usefulness, and provide value for customers or to the management.

• Reporting Needs: Providing roadmaps, and relevant information for the management.

• Training on the tools must be done with Peer to peer group support, because they are user groups that helps you.

• Adding functionalities on the tools can be a leverage.

Compliance

• Compliance Analysis and Management.

• If you are unable to perform them the best is to contract advisors or consultants if you have a lot of Money.
  *Those reports now could be generated from the data architecture designs. AI could also be used to generate reports.

Certificates, you need the proof to show they are being compliant

*Because you need to somehow show. And if you are doing this manually by hand, finding it out from the your existing systems, you better to have a very good relationships with universities and in order to get some extra workforce to go or consultants if you have money. Tons of money.

But now these can be generated from the AI descriptions. And I put a video on this, how it is done on the website. You can check that. Yeah. It says see video.

Shift Left with Security

• Security by design.
  Consultants provide these kind of, methods. Integrated risk management is yet yet another thing that puts more emphasis on the organizational risk management and cloud security architecture framework, which are provided by the cloud providers and platforms.

And, and there's also a kind of joint alliance, setting the standards for that. And then if you do these development processes, then you have to think the security also from that perspective. That if we are developing something, our plans and trials and test environments don't leak. It has to be designed in advance.

• Adaptive security architecture:Detect intrude intrusions and and and breaks of integrity or or GDPR or whatever.

• You need to choose what you should put in the architecture, and try to keep it as simple as possible.
  And here is an example of the same thing then. We already talked about this corporate governance, how IT governance and EA governance are related with it. But here is now a new kid on the block, AI governance.

AI Governance

• Models, algorithms, solutions. What it plays in the corporate, and at what role does it play.

The Profession of Enterprise Architects

• How they should make it simpler and communicate with number of people so they can communicate across.

*It means that you have to be open to all sorts of discussions and negotiations the architect.

Architecting is very interpersonal profession.

*Its better to take it from strategic point of view of point, by analyzing the capabilities, and the gaps in order to design the point from data, implementing that for data architecture.

*Using existing building blocks, you also try to find ALTERNATIVE solutions before engaging in the actual implementations, to prevent redundancy, that you are using existing building blocks so the changes wont happen more frequently.

Architecture

• The architecture makes easy the communications, and manage the complexity of the the solution.

• The streaming of data into report, to solve the open agile architecture or SAF ( scaled agile framework).

• Agility is better rather that lead to worse. It helps give piecemeal solutions it also means that its directly to enterprise

• Because you are providing piecemeal solutions, and now you can do it faster than it. Yeah.

  • Avoid redundancy by utilizing existing building blocks.

  • Evaluate alternative solutions before development.

  • Understand the impact of changes across the system.

  • Communicate and manage complexity to avoid side effects.

Not Easy. You must know what is going to be changed in the future. To find a solution and avoid trouble.

Conclusion

*Data federation and the, streaming data coming to dashboards going for reporting in data marts, data lake houses.

Nobody said this would be easy.