1/32
Vocabulary practice flashcards covering Databricks Data Engineer Associate exam concepts, including Asset Bundles, Lakeflow Jobs, Auto Loader, Medallion Architecture, Spark optimizations, and Unity Catalog.
Declarative Automation Bundle (Databricks Asset Bundle)
A configuration-driven tool used to manage and deploy Databricks assets (such as jobs, pipelines, and dashboards) as code across different environments (dev, test, prod) via CI/CD.
databricks.yml
The primary configuration file located at the repository root of a Databricks Asset Bundle that specifies the bundle name, target environments, variables, permissions, and included resource files.
Fan-in Control Flow
A DAG-based task dependency pattern where a single downstream task (such as a dashboard refresh) is configured to wait for and depend upon the completion of multiple parallel upstream tasks.
Lakeflow Jobs Run History
A built-in Databricks observability view used to evaluate execution trends against a baseline, inspect cluster metrics, and identify whether performance slowdowns are intermittent or consistent.
Auto Loader
A Databricks streaming file source that incrementally and efficiently processes new data files as they arrive in cloud object storage using internal checkpointing to track processed files.
File Notification Mode
An Auto Loader operating mode that utilizes cloud storage event notifications (e.g., SNS/SQS or Event Grid) rather than directory listing, scaling efficiently for continuous high-volume file arrivals.
File Arrival Trigger
A Lakeflow Jobs event-driven trigger that initiates a job execution automatically when new files land in a monitored cloud storage location.
Repair Run
A feature in Databricks Workflows that reruns only the failed or skipped tasks of a multi-task job using updated job or task settings while preserving execution lineage and run history.
Serverless Compute for Workflows
A fully managed compute configuration for Databricks jobs that automatically selects instance types, manages auto-scaling, applies Photon acceleration, and performs runtime upgrades with minimal operational overhead.
Continuous Pipeline Mode (DLT)
A Delta Live Tables execution mode where datasets update continuously at set intervals until stopped, maintaining active compute resources until manually shut down.
COPY INTO
An idempotent SQL command used to load data from cloud storage into Delta tables that skips previously loaded files unless modified or reloaded.
Lakeflow Connect Managed Connector
A low-code, fully managed serverless connector that provides built-in Change Data Capture (CDC), automated schema evolution, OAuth authentication, and direct ingestion into Unity Catalog tables.
WHEN NOT MATCHED BY SOURCE THEN DELETE
A Delta Lake MERGE INTO clause used to atomically remove rows from a target table that no longer exist in the incoming source dataset.
trigger(availableNow=True)
A Structured Streaming trigger setting that processes all currently available data across multiple micro-batches and then shuts down the stream.
trigger(processingTime="…")
A Structured Streaming trigger configuration that executes micro-batches at a defined periodic time interval (e.g., every 5 seconds).
Null-Safe Equality Operator (
A comparison predicate in Databricks SQL that returns true if both expressions evaluate to NULL or if both evaluate to identical non-NULL values.
Materialized View
A Unity Catalog object that precomputes and stores complex aggregations on a defined schedule to provide high-concurrency and low-latency performance for BI dashboards.
Broadcast Join
A Spark join optimization where a small lookup dataset is copied to all worker nodes to avoid data shuffling across the cluster.
spark.sql.autoBroadcastJoinThreshold
The Spark configuration parameter that sets the maximum table size in bytes for broadcasting across worker nodes during joins.
Left Anti-Join
A join operation between two tables that outputs only rows from the left table that have no matching key in the right table, excluding right-side columns.
dropDuplicates()
A PySpark DataFrame method used to eliminate duplicate records across all columns or a specified subset of key columns.
dropna(subset=[…])
A PySpark DataFrame method used to drop rows containing NULL values across specified columns in a single method call.
Bronze Layer
The raw ingestion layer in the medallion architecture where data from diverse sources is stored append-only in Delta format while preserving original schemas without complex transformations.
Silver Layer
The refined layer of the medallion architecture where raw data is cleaned, filtered, enriched, deduplicated, and conformed for downstream analytics.
Gold Layer
The final layer of the medallion architecture composed of denormalized, business-level aggregates, star schemas, or materialized views tailored for reporting and BI applications.
Predictive Optimization
A Databricks Unity Catalog service that automatically determines when and how to run OPTIMIZE file compaction and VACUUM cleanups on managed tables without manual scheduling.
Databricks Connect
A client library that enables developers to write Spark code and step through business logic locally in their preferred IDE while proxying DataFrame execution to a remote Databricks cluster.
Lakehouse Federation
A Databricks data virtualization feature that allows users to query external databases (such as PostgreSQL and Azure Synapse) directly in place through Unity Catalog without copying or moving data.
Control Plane
The core infrastructure layer managed by Databricks that hosts the web application, encrypted notebook storage, compute orchestration, and Unity Catalog metadata services.
Compute Plane
The infrastructure layer where virtual machines reside, Spark clusters execute queries, and actual customer data transformations are processed.
Serverless SQL Warehouse
A fully managed compute resource that provides near-instant compute startup, automatic scaling based on SQL query concurrency, and charges solely for execution time.
External Table
A table registered in Databricks whose metadata is governed by Unity Catalog while its underlying data files remain in a user-specified cloud storage path via the LOCATION clause.
count_if
A Databricks SQL aggregate function that counts the number of rows for which a given boolean condition evaluates to true.