machine learning engineer - ML Data Storage Flashcards

AWS ML storage options and best practices

what data access pattern does FSx support?

random access

what data access pattern does EFS support?

random access

what data access pattern does EBS storage support?

sequential streaming

what storage type is best for dataset storage?

S3

what storage type is best for real-time and streaming workloads?

Amazon EFS, as it supports low-latency access and concurrent access from multiple instances.

what storage type is best for training large ML workloads?

EBS as it provides random I/O access to EC2 instances with high throughput.

what storage type is best for ML inference workloads?

Amazon EBS or Amazon EFS, depending on the latency and concurrency requirements.

What is FSx ?

Amazon FSx is a fully managed file storage service that provides file systems optimized for specific workloads, such as high-performance computing or machine learning.

When is FSx used?

FSx is used when high-performance file storage is required for workloads like machine learning, big data analytics, or high-performance computing.

What are the storage options for ML?

The storage options for machine learning include Amazon S3 for object storage, Amazon EBS for block storage, and Amazon EFS or FSx for file storage, each catering to different performance and scalability needs.

what are the object notation data types for ML and how are they used?

JSON and JSONL. They are used on non-tabular hierarchical data

what data access pattern does S3 support?

copy and load