1.1.1 Configure Spark Pools

What are the two types of Spark pools in Fabric?

Starter pools (pre-warmed, shared, fast startup) and custom pools (user-configured with specific node sizes, counts, and auto-scale settings).

What is a starter pool in Fabric Spark?

A pre-warmed, shared Spark pool that provides fast session startup with no configuration required. Best for ad-hoc exploration and development.

When should you use a custom Spark pool instead of a starter pool?

When you need dedicated resources, specific node sizes, guaranteed compute, or custom auto-scaling configurations for heavy production workloads.

Where do you configure Spark pool settings?

Workspace Settings > Data Engineering/Science > Spark Settings. You can also configure pools at the Environment item level.

What is the minimum and maximum node range for a custom Spark pool?

You define a min and max node count. Fabric auto-scales executors within this range. Minimum is 1 node; maximum depends on capacity SKU.

How does Spark pool selection affect capacity consumption?

Spark pools consume Fabric Capacity Units (CUs). Larger node sizes and more nodes consume more CUs from the workspace's assigned capacity.

Can different notebooks use different Spark pools?

Yes. You can attach different Environments to different notebooks, and each Environment can reference a different Spark pool configuration.

What happens when all Spark pool resources are in use?

New session requests are queued until resources free up. If the queue timeout is exceeded, the session request fails.

What is high concurrency mode for Spark pools?

A mode where multiple users share a single Spark session to reduce resource consumption. Useful for interactive exploration with many concurrent users.

How do you set a default Spark pool for the entire workspace?

In Workspace Settings > Spark Settings, select the default pool. All notebooks and Spark jobs use this unless overridden by an Environment.