Databricks Data Engineer Associate Exam Review

0.0(0)
Studied by 0 people
call kaiCall Kai
learnLearn
examPractice Test
spaced repetitionSpaced Repetition
heart puzzleMatch
flashcardsFlashcards
GameKnowt Play
Card Sorting

1/32

flashcard set

Earn XP

Description and Tags

Vocabulary practice flashcards covering Databricks Data Engineer Associate exam concepts, including Asset Bundles, Lakeflow Jobs, Auto Loader, Medallion Architecture, Spark optimizations, and Unity Catalog.

33 Terms

1
New cards

Declarative Automation Bundle (Databricks Asset Bundle)

A configuration-driven tool used to manage and deploy Databricks assets (such as jobs, pipelines, and dashboards) as code across different environments (dev, test, prod) via CI/CD.

2
New cards

databricks.yml

The primary configuration file located at the repository root of a Databricks Asset Bundle that specifies the bundle name, target environments, variables, permissions, and included resource files.

3
New cards

Fan-in Control Flow

A DAG-based task dependency pattern where a single downstream task (such as a dashboard refresh) is configured to wait for and depend upon the completion of multiple parallel upstream tasks.

4
New cards

Lakeflow Jobs Run History

A built-in Databricks observability view used to evaluate execution trends against a baseline, inspect cluster metrics, and identify whether performance slowdowns are intermittent or consistent.

5
New cards

Auto Loader

A Databricks streaming file source that incrementally and efficiently processes new data files as they arrive in cloud object storage using internal checkpointing to track processed files.

6
New cards

File Notification Mode

An Auto Loader operating mode that utilizes cloud storage event notifications (e.g., SNS/SQS or Event Grid) rather than directory listing, scaling efficiently for continuous high-volume file arrivals.

7
New cards

File Arrival Trigger

A Lakeflow Jobs event-driven trigger that initiates a job execution automatically when new files land in a monitored cloud storage location.

8
New cards

Repair Run

A feature in Databricks Workflows that reruns only the failed or skipped tasks of a multi-task job using updated job or task settings while preserving execution lineage and run history.

9
New cards

Serverless Compute for Workflows

A fully managed compute configuration for Databricks jobs that automatically selects instance types, manages auto-scaling, applies Photon acceleration, and performs runtime upgrades with minimal operational overhead.

10
New cards

Continuous Pipeline Mode (DLT)

A Delta Live Tables execution mode where datasets update continuously at set intervals until stopped, maintaining active compute resources until manually shut down.

11
New cards

COPY INTO

An idempotent SQL command used to load data from cloud storage into Delta tables that skips previously loaded files unless modified or reloaded.

12
New cards

Lakeflow Connect Managed Connector

A low-code, fully managed serverless connector that provides built-in Change Data Capture (CDC), automated schema evolution, OAuth authentication, and direct ingestion into Unity Catalog tables.

13
New cards

WHEN NOT MATCHED BY SOURCE THEN DELETE

A Delta Lake MERGE INTO clause used to atomically remove rows from a target table that no longer exist in the incoming source dataset.

14
New cards

trigger(availableNow=True)

A Structured Streaming trigger setting that processes all currently available data across multiple micro-batches and then shuts down the stream.

15
New cards

trigger(processingTime="…")

A Structured Streaming trigger configuration that executes micro-batches at a defined periodic time interval (e.g., every 5 seconds).

16
New cards

Null-Safe Equality Operator (

A comparison predicate in Databricks SQL that returns true if both expressions evaluate to NULL or if both evaluate to identical non-NULL values.

17
New cards

Materialized View

A Unity Catalog object that precomputes and stores complex aggregations on a defined schedule to provide high-concurrency and low-latency performance for BI dashboards.

18
New cards

Broadcast Join

A Spark join optimization where a small lookup dataset is copied to all worker nodes to avoid data shuffling across the cluster.

19
New cards

spark.sql.autoBroadcastJoinThreshold

The Spark configuration parameter that sets the maximum table size in bytes for broadcasting across worker nodes during joins.

20
New cards

Left Anti-Join

A join operation between two tables that outputs only rows from the left table that have no matching key in the right table, excluding right-side columns.

21
New cards

dropDuplicates()

A PySpark DataFrame method used to eliminate duplicate records across all columns or a specified subset of key columns.

22
New cards

dropna(subset=[…])

A PySpark DataFrame method used to drop rows containing NULL values across specified columns in a single method call.

23
New cards

Bronze Layer

The raw ingestion layer in the medallion architecture where data from diverse sources is stored append-only in Delta format while preserving original schemas without complex transformations.

24
New cards

Silver Layer

The refined layer of the medallion architecture where raw data is cleaned, filtered, enriched, deduplicated, and conformed for downstream analytics.

25
New cards

Gold Layer

The final layer of the medallion architecture composed of denormalized, business-level aggregates, star schemas, or materialized views tailored for reporting and BI applications.

26
New cards

Predictive Optimization

A Databricks Unity Catalog service that automatically determines when and how to run OPTIMIZE file compaction and VACUUM cleanups on managed tables without manual scheduling.

27
New cards

Databricks Connect

A client library that enables developers to write Spark code and step through business logic locally in their preferred IDE while proxying DataFrame execution to a remote Databricks cluster.

28
New cards

Lakehouse Federation

A Databricks data virtualization feature that allows users to query external databases (such as PostgreSQL and Azure Synapse) directly in place through Unity Catalog without copying or moving data.

29
New cards

Control Plane

The core infrastructure layer managed by Databricks that hosts the web application, encrypted notebook storage, compute orchestration, and Unity Catalog metadata services.

30
New cards

Compute Plane

The infrastructure layer where virtual machines reside, Spark clusters execute queries, and actual customer data transformations are processed.

31
New cards

Serverless SQL Warehouse

A fully managed compute resource that provides near-instant compute startup, automatic scaling based on SQL query concurrency, and charges solely for execution time.

32
New cards

External Table

A table registered in Databricks whose metadata is governed by Unity Catalog while its underlying data files remain in a user-specified cloud storage path via the LOCATION clause.

33
New cards

count_if

A Databricks SQL aggregate function that counts the number of rows for which a given boolean condition evaluates to true.