Enterprise Data Fundamentals

Outline

Data Management Introduction

• States of Digital Data
• What is Data Management
• The Core Components
• Objectives
• Timeliness
• Data Management and Data Governance Relationship
• Metadata
• Information About Processes
• Data Management Systems
• Data Warehouses, Data Marts, and Data Lakes
• High-Level Traditional Enterprise Data Flow
• The Conceptual DW/BI Diagram
• The Enterprise Data Problem Domain
• ETL
• Workflow (Pipeline) Orchestration Systems
• Data Engineering
• Data Management Best Practices

Data Governance

• Data Governance
• The DAMA-DMBOK Framework
• Key Artifacts of Data Governance
• Shared Environment Governance Controls
• Best Practices
• The Goldilocks Principle
• Common Issues That Can be Prevented through Effective Governance
• Ethics of Data Handling
• Ethical AI

Data Architecture and Data Modeling

• Data Architecture Defined
• Data Modeling Defined
• Data Architecture vs Data Modeling
• A Data Model
• Data Modeling and Design in Practice
• Conceptual Data Models
• The Entity-Relationship Model
• Logical Data Models
• Normalization
• Normalization Forms
• Physical Data Models
• The Physical Data Model and DDL
• The Star Schema
• The Fact and Dimension Tables
• Master Data Management

Data Storage Options

• Storage Options
• Which One Should I Choose?
• Storage Location Options
• Deciding on Database Type
• Data Models
• NoSQL Database Storage Types
• A Key-Value Storage Type Example
• Efficient Storage with Columnar Formats
• Scalability
• ACID Compliance
• Cloud-Based Database Services
• Creating a MySQL Database Instance Dialog
• Key Concepts of Object Storage
• Accessing Data in Object Stores
• Content Delivery Networks
• BigQuery
• BigQuery Data Source Integrations
• BigQuery Use Case: Migrating Data from Teradata
• The Repository Type
• Access Timeliness

Data Security

• Security Domains
• The CIAs of Security
• Common Areas, Concerns, and Considerations
• NIST Risk Management Framework
• Vulnerability and Exploits
• Ways to Eliminate (Mitigate) Vulnerabilities
• The Inputs, Activities, and Deliverables Flow
• Inputs
• Activities
• Deliverables
• Distributed Identity Management
• Access Control: Authentication & Authorization
• Authorization and Data Access Constraints
• Working Environments
• Access Control: Auditing
• Cloud Shared Responsibility Model (SRM)
• The AWS SRM: The AWS Side of the Deal
• The AWS SRM: Your Side of the Deal
• Cloud Compliance Programs
• DevOps Security Concerns
• Agile Programming Concerns
• Protecting Sensitive Data at Rest
• Hashing
• Secure Hashing Algorithm Family
• Symmetric and Asymmetric (Public) Key Encryption
• Security Best Practices
• Be Aware of Spear-phishing Attacks

Data Quality Introduction

• Data Quality Defined
• An Opinion on Data Quality
• The Great, Fast, and Cheap Quality Diagram
• Data Quality Dimensions/Properties
• Interpreting Data Quality Properties
• Data Flow Potential Points of Failure
• Data Quality Assurance
• Common Factors Contributing to Poor Data Quality
• Data Quality is a Shared Concern
• Data Governance
• Common Steps to Overcome Data Quality Issues
• Data Observability
• Application Performance Monitoring (APM) and Observability Magic Quadrant
• Data Quality and Data Observability Relationship
• A Glossary of Business Terms
• Data Dictionaries
• SLAs
• SLAs and Non-Functional Requirements
• SLAs Types
• Data Integration and Data Integrity
• IT Systems' Woes
• Unified Data Platform
• The Methods and Techniques to Ensure Data Quality
• Maintenance
• Automation
• Data Formats
• Interoperable Data
• Data Validation
• DDL-based Data Validation
• The Schema Production and Consumption Diagram
• Regular Expressions
• Industry-Standard Data Models

Lab Exercises

• Lab 1. Learning the Colab Jupyter Notebook Environment
• Lab 2. Metadata
• Lab 3. The Star Schema Project
• Lab 4. Understanding the Normalization and MDM Connection
• Lab 5. Evidence-Based SLA Metrics