dp-203 Practice Question 1942
Exam: dp-203
Domain: Design and implement data storage
Topic: Implement physical data storage structures
Difficulty: medium
You are tasked with optimizing the storage and query performance of a dataset in Azure Data Lake Storage Gen2. The dataset contains daily transaction records and needs to be processed in near real-time. Which of the following strategies would best meet these requirements while leveraging the capabilities of Azure Data Lake Storage Gen2 and Delta Lake?
Answer Options
A
Store the data in CSV files and use MapReduce for processing.
B
Use Azure Data Lake Storage Gen2 and implement Delta Lake with partitioning by date and storing data in Parquet format.
C
Store the data in JSON files and use Spark SQL for processing.
D
Use Azure Blob Storage and implement custom partitioning logic in code.
Correct Answer
B: Use Azure Data Lake Storage Gen2 and implement Delta Lake with partitioning by date and storing data in Parquet format.
Explanation
Implementing Delta Lake on top of Azure Data Lake Storage Gen2 allows for efficient handling of incremental changes and supports ACID transactions, which is crucial for near real-time processing. Partitioning the data by date ensures that only relevant partitions need to be scanned during queries, improving performance. Storing the data in Parquet format within Delta Lake further optimizes storage and query performance.