The module compares traditional storage systems (scale-up Isilon/PowerScale style) with Pure Storage FlashBlade, emphasizing a design built for modern big data workloads rather than retrofitting legacy architectures.
Key idea: simplicity at scale, high parallelism, and no traditional serial bottlenecks by design.
Analogy: front end vs back end is like restaurant service — clients interact with a waiter (front end) who passes requests to kitchen staff (back end) without needing to know how data is stored or orchestrated.
Front end vs back end layers
Front end handles client requests, talks to the client, and passes generic requests to the back end.
Back end (storage management) handles the filesystem, metadata, and data placement across the system.
The client remains unaware of the internal FlashBlade architecture; it appears as a normal NFS server to the client.
In a traditional scale-up system, there are two controllers for redundancy, but the system still divides front end and back end processing. The front end may adapt to new protocols (e.g., upgrading from NFS to NFS 4.1) while the back end remains responsible for the filesystem.
Traditional scale-up systems (e.g., Isilon/PowerScale in this context) rely on a centralized metadata table that can become a serial bottleneck, affects lookups, and can limit scale when many small files and rapid lookups are involved.
Traditional scale-up vs FlashBlade (high level)
FlashArray/FlashBlade are contrasted: FlashArray is a scale-up block+file system designed around traditional architectures; FlashBlade is built for modern big data workloads with parallelism and scale-out characteristics.
Traditional, scale-up systems typically split work into a front end (client-facing) and back end (filesystem management) with a central metadata table, leading to potential bottlenecks for high numbers of small file lookups.
FlashBlade uses a different architecture to avoid serial bottlenecks and to enable true parallelism across many operations.
FlashBlade architecture: authorities, DFMs, and storage manager
Core concept: back-end processes are called authorities; there are exactly 128 authorities in any configuration (FlashBlade S, FlashBlade E, etc.).
Authorities run on blades and are designed to operate independently; data can be accessed across blades via the internal network.
Each blade runs a set of authorities; data is distributed evenly to balance load and storage usage.
128 authorities allow fine-grained parallelism and load balancing across blades.
Partitions: Every DFM (disk flash module) partitions its NVRAM and NAND flash into 128 partitions, one per authority. This design prevents cross-authority locking and reduces contention.
Storage manager: A lightweight piece of software per DFM that handles low-level reads/writes, encryption, and exposing the DFM to authorities.
Data at rest is encrypted using hardware-accelerated encryption; storage manager ensures encryption during write and reads during retrieval.
The system architecture is designed around big data workloads, enabling parallelism and avoiding traditional serial bottlenecks.
Data path: write path (client to flash)
Client makes a write request via NFS; the request goes through LACP load balancing across FIOMs/PHIOMs and protocol endpoints on blades.
Protocol endpoint role: lightweight, handles the NFS file handle (inode as identity), request length, and offset. It performs a deterministic hash to decide which authority will manage this file operation.
Hashing: protocol endpoint computes a hash from inputs (client IP, client TCP port, destination VIP, destination port 2049). The result is a number in the range 1..128, deterministically mapping to an authority.
Deterministic mapping: The same inputs produced by any protocol endpoint yield the same authority, ensuring consistent routing without centralized state.
Authority handling: The selected authority has its own metadata table and manages the location of the file data on erasure-coded stripes across the backend.
NVRAM buffering: When a write is accepted, the authority buffers data into NVRAM across at least three blades (NVRAM partitions are per authority). This creates a write buffer that is power-protected and can be flushed later to flash.
No per-file immediate write to flash: the authority writes full stripes once enough data accumulates, not individual small writes.
Stripe construction: Stripes are formed as full stripes (e.g., around 16extMB per stripe, though exact block composition can vary). Data is coalesced into full stripes, and a parity/erasure coding scheme is applied across stripes stored on flash.
Erasure coding: Data is written to flash as erasure-coded stripes across multiple DFMs. This provides redundancy without RAID, since there is no disk-level RAID in FlashBlade.
Multi-block writes: Even if a single NFS write is small (e.g., 512 KB), the system aggregates into full stripes, coalescing small writes into larger stripes for efficient storage and I/O.
Parallelism: Multiple write operations can be in flight simultaneously across multiple authorities. In one example, four writes can occur in parallel with zero hardware/software queuing.
Hardware parallelism: The system can perform up to 128 operations at the same time on an S-series blade setup, with very thin queues if more than that many operations arrive (e.g., when there are 256 ops, queues are still shallow).
The hashing distribution is designed to balance load across the 128 authorities so that per-authority work and per-authority NAND/NVRAM usage are evenly balanced.
Why blades are the same: uniform blade capabilities ensure even distribution of work; having blades of different capabilities would not yield a performance advantage because the hashing distributes workload evenly.
Data path: read path (authority to client)
Client requests a read; protocol endpoint hashes the request to determine which authority manages the file.
Authority location: The authority consults its own metadata table to locate the file’s stripes and how to reconstruct the data from erasure-coded shards.
Data retrieval: Storage managers on the authority coordinate with their own NVRAM/flash to fetch the required stripes; the data is read from the correct stripes and sent back through the protocol endpoint to the client.
The client again sees a normal NFS server; no exposure of the FlashBlade internals to the client.
Parallel reads: Because data may be distributed across multiple authorities, multiple reads can occur in parallel, maintaining high throughput without cross-authority serialization.
Metadata, bottlenecks, and design decisions
Central metadata bottleneck in traditional Isilon/PowerScale architecture: a single (or serial) metadata table can become a performance/scale bottleneck as the number of small-file lookups increases.
FlashBlade avoids a single central metadata bottleneck by distributing metadata across authorities; each authority maintains its own metadata table.
This distribution supports high-scale, parallel workloads typical of modern big data, AI, and analytics workloads.
Data locality vs distribution: Authorities can access data across blades; data locality is not required. This enables better failover, load balancing, and resilience because authorities are not tied to a single blade.
The system uses high-performance Ethernet for inter-blade communication: 100 Gbps Ethernet with 50 Gbps reserved for front-end communications (client-facing) and the remainder for cross-blade communication.
Total interconnect: 100extGbps; front-end per blade: 50extGbps; cross-blade interconnect: 50extGbps
File and block handling details
Block sizing: FlashBlade uses a variable block size rather than fixed 4 KB or 8 KB blocks common in traditional file systems. This minimizes waste and fragmentation when writing small files.
Why variable block sizes matter: With fixed blocks, small files would waste space (partial blocks). The variable approach allows coalescing into full stripes regardless of file size.
Stripe formation and coalescing: Data writes are buffered in NVRAM, then written to flash as full stripes (e.g., around 16 MB), with erasure coding across the stripes.
Parity and erasure coding: The system uses erasure coding to protect data across multiple DFMs. This eliminates reliance on RAID and provides redundancy across the distributed architecture.
Data distribution: The hashing, NVRAM partitioning, and erasure coding work together to distribute data evenly across all blades and DFMs, enabling linear scaling and avoiding hotspots.
Efficiency, performance, and real-world implications
The architecture targets modern big data workloads with many small to medium files and also large files, providing consistent efficiency across file sizes without tuning.
Small file performance: Because the system can map to many authorities and perform parallel lookups/writes, small-file workloads are handled efficiently without the bottlenecks of centralized metadata.
Large file performance: Even with large files, the protocol endpoint breaks the operation into multiple pieces that can be directed to different authorities in parallel, maintaining high throughput.
Stripe efficiency on QLC flash: The system can fill up each NAND array block's pages, avoiding inefficiencies with QLC writes by shaping data to fill pages fully, which improves write performance and endurance.
No need for extra buffering strategies like tiering with storage-class memory; the architecture itself provides parallelism and efficient I/O at the hardware and software layers.
Real-world scalability: Adding a blade triggers a rebalance of authorities across blades, which increases per-blade CPU, RAM, and I/O capacity, maintaining linear scaling as you add resources.
Scaling out: adding blades and rebalancing
When a new blade is added, the system checks hardware versions and may auto-sync software to match other blades.
New blade adds capacity: 128 authorities are rebalanced across the full set of blades. Authorities stop on some blades and start on the new blade; authorities are stateless, so redistribution is fast.
After adding a blade, the per-blade load of authorities decreases (e.g., from 16 authorities per blade to 14–15 on most blades in a simple example), increasing available RAM and CPU time per authority, and enabling the front-end (PHIOME) to balance across more blades.
The system thus achieves linear scaling: each new blade adds proportional improvements in performance, barring network or client-side bottlenecks.
Blade failures: If a blade fails, purity detects the failure and rebalances authorities onto the remaining blades, maintaining service continuity.
Break and wrap-up ideas
The FlashBlade architecture is designed specifically for big data workloads, emphasizing parallelism, distributed metadata, erasure coding, and no central bottlenecks.
The design aims to simplify sizing and deployment: there is no need to tune for small vs large files; the system handles all sizes with consistent efficiency.
The architecture avoids traditional RAID and leverages high-bandwidth, low-latency cross-blade networking to enable true parallelism across 128 authorities.
For sizing: the FlashBlade Sizer helps predict how many blades/authorities are needed for a given workload; scaling involves adding blades and rebalancing authorities with minimal disruption.
Quick reference notes (key numbers and concepts)
Authorities: 128 authorities per FlashBlade configuration (always).
Authorities per blade (example): with N<em>blades=8, authorities per blade = 128/8=16; with N</em>blades=10, authorities per blade ≈ 128/10=12.8 (distributed as evenly as possible, e.g., some blades with 13, some with 12).
File limit: real-world practical limit is driven by clients and storage capacity, with an approximate theoretical limit of 4×1015 files.
Stripe size: around 16 MB per stripe; writes are buffered in NVRAM until full stripe is ready.
Write length example: a write can be split into a 512 KB segment for NFS operation, later aggregated into full stripes for flash writes.
NVRAM partitions: each DFM partitions its NVRAM into 128 partitions, one per authority, preventing cross-authority locking.
Flash interconnect: 100 Gbps Ethernet overall; front-end (client-facing) communications use 50extGbps per blade; the remainder 50extGbps is used for cross-blade communication.
Parallelism: up to 128 operations can be processed in parallel (no serial bottlenecks under typical big data workloads).
Block size: variable block size (no fixed 4 KB/8 KB minimum); reduces fragmentation and improves efficiency for small and large files alike.
Data distribution: data and workload are balanced across all blades and authorities to maximize throughput and avoid hotspots.
Data protection: no RAID; data is protected via erasure coding across stripes and across multiple blades.
Data at rest: all data encrypted with hardware-accelerated encryption.
Breakpoints: the presenter notes that a deeper dive into EC chassis/EC management will occur later; the EC chassis acts as a control plane for the fabric, with emphasis on scalable management of the FlashBlade environment.
Connections to prior principles and real-world relevance
Aligns with distributed systems principles: horizontal scaling, stateless components (authorities), and distributed metadata to avoid single points of contention.
Demonstrates practical tradeoffs between traditional file-system architectures and modern, parallel, scale-out designs used for AI/big data, where latency and throughput under massive concurrency matter more than single-node capacity.
Ethical/practical implications: by design, FlashBlade aims to simplify deployment and operation for large data workloads, potentially reducing human error and maintenance overhead; it emphasizes performance consistency and data integrity through erasure coding and encryption.