Average Score In Real Data-Engineer-Associate Exam
98%
Questions came from our Data-Engineer-Associate dumps.
Choosing the Right Path for Your Data-Engineer-Associate Exam Preparation
Welcome to PassExamHub's comprehensive study guide for the AWS Certified Data Engineer - Associate (DEA-C01) exam. Our Data-Engineer-Associate dumps is designed to equip you with the knowledge and resources you need to confidently prepare for and succeed in the Data-Engineer-Associate certification exam.
What Our Amazon Data-Engineer-Associate Study Material Offers
PassExamHub's Data-Engineer-Associate dumps PDF is carefully crafted to provide you with a comprehensive and effective learning experience. Our study material includes:
In-depth Content: Our study guide covers all the key concepts, topics, and skills you need to master for the Data-Engineer-Associate exam. Each topic is explained in a clear and concise manner, making it easy to understand even the most complex concepts.
Online Test Engine: Test your knowledge and build your confidence with a wide range of practice questions that simulate the actual exam format. Our test engine cover every exam objective and provide detailed explanations for both correct and incorrect answers.
Exam Strategies: Get valuable insights into exam-taking strategies, time management, and how to approach different types of questions.
Real-world Scenarios: Gain practical insights into applying your knowledge in real-world scenarios, ensuring you're well-prepared to tackle challenges in your professional career.
Why Choose PassExamHub?
Expertise: Our Data-Engineer-Associate exam questions answers are developed by experienced Amazon certified professionals who have a deep understanding of the exam objectives and industry best practices.
Comprehensive Coverage: We leave no stone unturned in covering every topic and skill that could appear on the Data-Engineer-Associate exam, ensuring you're fully prepared.
Engaging Learning: Our content is presented in a user-friendly and engaging format, making your study sessions enjoyable and effective.
Proven Success: Countless students have used our study materials to achieve their Data-Engineer-Associate certifications and advance their careers.
Start Your Journey Today!
Embark on your journey to AWS Certified Data Engineer - Associate (DEA-C01) success with PassExamHub. Our study material is your trusted companion in preparing for the Data-Engineer-Associate exam and unlocking exciting career opportunities.
A data engineer needs Amazon Athena queries to finish faster. The data engineer noticesthat all the files the Athena queries use are currently stored in uncompressed .csv format.The data engineer also notices that users perform most queries by selecting a specificcolumn.Which solution will MOST speed up the Athena query performance?
A. Change the data format from .csvto JSON format. Apply Snappy compression. B. Compress the .csv files by using Snappy compression. C. Change the data format from .csvto Apache Parquet. Apply Snappy compression. D. Compress the .csv files by using gzjg compression.
Answer: C
Explanation: Amazon Athena is a serverless interactive query service that allows you to
analyze data in Amazon S3 using standard SQL. Athena supports various data formats,
such as CSV, JSON, ORC, Avro, and Parquet. However, not all data formats are equally
efficient for querying. Some data formats, such as CSV and JSON, are row-oriented,
meaning that they store data as a sequence of records, each with the same fields. Roworiented
formats are suitable for loading and exporting data, but they are not optimal for
analytical queries that often access only a subset of columns. Row-oriented formats also
do not support compression or encoding techniques that can reduce the data size and
improve the query performance.
On the other hand, some data formats, such as ORC and Parquet, are column-oriented,
meaning that they store data as a collection of columns, each with a specific data type.
Column-oriented formats are ideal for analytical queries that often filter, aggregate, or join
data by columns. Column-oriented formats also support compression and encoding
techniques that can reduce the data size and improve the query performance. For
example, Parquet supports dictionary encoding, which replaces repeated values with
numeric codes, and run-length encoding, which replaces consecutive identical values with
a single value and a count. Parquet also supports various compression algorithms, such as
Snappy, GZIP, and ZSTD, that can further reduce the data size and improve the query
performance. Therefore, changing the data format from CSV to Parquet and applying Snappy
compression will most speed up the Athena query performance. Parquet is a columnoriented
format that allows Athena to scan only the relevant columns and skip the rest,
reducing the amount of data read from S3. Snappy is a compression algorithm that reduces
the data size without compromising the query speed, as it is splittable and does not require
decompression before reading. This solution will also reduce the cost of Athena queries, as
Athena charges based on the amount of data scanned from S3.
The other options are not as effective as changing the data format to Parquet and applying
Snappy compression. Changing the data format from CSV to JSON and applying Snappy
compression will not improve the query performance significantly, as JSON is also a roworiented
format that does not support columnar access or encoding techniques.
Compressing the CSV files by using Snappy compression will reduce the data size, but it
will not improve the query performance significantly, as CSV is still a row-oriented format
that does not support columnar access or encoding techniques. Compressing the CSV files
by using gzjg compression will reduce the data size, but it willdegrade the query
performance, as gzjg is not a splittable compression algorithm and requires decompression
before reading. References:
Amazon Athena
Choosing the Right Data Format
AWS Certified Data Engineer - Associate DEA-C01 Complete Study Guide,
Chapter 5: Data Analysis and Visualization, Section 5.1: Amazon Athena
Question # 2
A company stores data in a data lake that is in Amazon S3. Some data that the company stores in the data lake contains personally identifiable information (PII). Multiple usergroups need to access the raw data. The company must ensure that user groups canaccess only the PII that they require.Which solution will meet these requirements with the LEAST effort?
A. Use Amazon Athena to query the data. Set up AWS Lake Formation and create datafilters to establish levels of access for the company's IAM roles. Assign each user to theIAM role that matches the user's PII access requirements. B. Use Amazon QuickSight to access the data. Use column-level security features inQuickSight to limit the PII that users can retrieve from Amazon S3 by using AmazonAthena. Define QuickSight access levels based on the PII access requirements of theusers. C. Build a custom query builder UI that will run Athena queries in the background to accessthe data. Create user groups in Amazon Cognito. Assign access levels to the user groupsbased on the PII access requirements of the users. D. Create IAM roles that have different levels of granular access. Assign the IAM roles toIAM user groups. Use an identity-based policy to assign access levels to user groups at thecolumn level.
Answer: A
Explanation:
Amazon Athena is a serverless, interactive query service that enables you to analyze data
in Amazon S3 using standard SQL. AWS Lake Formation is a service that helps you build,
secure, and manage data lakes on AWS. You can use AWS Lake Formation to create data
filters that define the level of access for different IAM roles based on the columns, rows, or
tags of the data. By using Amazon Athena to query the data and AWS Lake Formation to
create data filters, the company can meet the requirements of ensuring that user groups
can access only the PII that they require with the least effort. The solution is to use Amazon
Athena to query the data in the data lake that is in Amazon S3. Then, set up AWS Lake
Formation and create data filters to establish levels of access for the company’s IAM roles.
For example, a data filter can allow a user group to access only the columns that contain
the PII that they need, such as name and email address, and deny access to the columns
that contain the PII that they do not need, such as phone number and social security
number. Finally, assign each user to the IAM role that matches the user’s PII access
requirements. This way, the user groups can access the data in the data lake securely and
efficiently. The other options are either not feasible or not optimal. Using Amazon
QuickSight to access the data (option B) would require the company to pay for the
QuickSight service and to configure the column-level security features for each user.
Building a custom query builder UI that will run Athena queries in the background to access
the data (option C) would require the company to develop and maintain the UI and to
integrate it with Amazon Cognito. Creating IAM roles that have different levels of granular
access (option D) would require the company to manage multiple IAM roles and policies and to ensure that they are aligned with the data schema. References:
Amazon Athena
AWS Lake Formation
AWS Certified Data Engineer - Associate DEA-C01 Complete Study Guide,
Chapter 4: Data Analysis and Visualization, Section 4.3: Amazon Athena
Question # 3
A company receives call logs as Amazon S3 objects that contain sensitive customerinformation. The company must protect the S3 objects by using encryption. The companymust also use encryption keys that only specific employees can access.Which solution will meet these requirements with the LEAST effort?
A. Use an AWS CloudHSM cluster to store the encryption keys. Configure the process thatwrites to Amazon S3 to make calls to CloudHSM to encrypt and decrypt the objects.Deploy an IAM policy that restricts access to the CloudHSM cluster. B. Use server-side encryption with customer-provided keys (SSE-C) to encrypt the objectsthat contain customer information. Restrict access to the keys that encrypt the objects. C. Use server-side encryption with AWS KMS keys (SSE-KMS) to encrypt the objects thatcontain customer information. Configure an IAM policy that restricts access to the KMSkeys that encrypt the objects. D. Use server-side encryption with Amazon S3 managed keys (SSE-S3) to encrypt theobjects that contain customer information. Configure an IAM policy that restricts access tothe Amazon S3 managed keys that encrypt the objects.
Answer: C
Explanation: Option C is the best solution to meet the requirements with the least effort
because server-side encryption with AWS KMS keys (SSE-KMS) is a feature that allows
you to encrypt data at rest in Amazon S3 using keys managed by AWS Key Management
Service (AWS KMS). AWS KMS is a fully managed service that enables you to create and
manage encryption keys for your AWS services and applications. AWS KMS also allows
you to define granular access policies for your keys, such as who can use them to encrypt
and decrypt data, and under what conditions. By using SSE-KMS, you canprotect your S3
objects by using encryption keys that only specific employees can access, without having to manage the encryption and decryption process yourself.
Option A is not a good solution because it involves using AWS CloudHSM, which is a
service that provides hardware security modules (HSMs) in the AWS Cloud. AWS
CloudHSM allows you to generate and use your own encryption keys on dedicated
hardware that is compliant with various standards and regulations. However, AWS
CloudHSM is not a fully managed service and requires more effort to set up and maintain
than AWS KMS. Moreover, AWS CloudHSM does not integrate with Amazon S3, so you
have to configure the process that writes to S3 to make calls to CloudHSM to encrypt and
decrypt the objects, which adds complexity and latency to the data protection process.
Option B is not a good solution because it involves using server-side encryption with
customer-provided keys (SSE-C), which is a feature that allows you to encrypt data at rest
in Amazon S3 using keys that you provide and manage yourself. SSE-C requires you to
send your encryption key along with each request to upload or retrieve an object. However,
SSE-C does not provide any mechanism to restrict access to the keys that encrypt the
objects, so you have to implement your own key management and access control system,
which adds more effort and risk to the data protection process.
Option D is not a good solution because it involves using server-side encryption with
Amazon S3 managed keys (SSE-S3), which is a feature that allows you to encrypt data at
rest in Amazon S3 using keys that are managed by Amazon S3. SSE-S3 automatically
encrypts and decrypts your objects as they are uploaded and downloaded from S3.
However, SSE-S3 does not allow you to control who can access the encryption keys or
under what conditions. SSE-S3 uses a single encryption key for each S3 bucket, which is
shared by all users who have access to the bucket. This means that you cannot restrict
access to the keys that encrypt the objects by specific employees, which does not meet the
requirements.
References:
AWS Certified Data Engineer - Associate DEA-C01 Complete Study Guide
Protecting Data Using Server-Side Encryption with AWS KMS–Managed
Encryption Keys (SSE-KMS) - Amazon Simple Storage Service
What is AWS Key Management Service? - AWS Key Management Service
What is AWS CloudHSM? - AWS CloudHSM
Protecting Data Using Server-Side Encryption with Customer-Provided Encryption
Keys (SSE-C) - Amazon Simple Storage Service
Protecting Data Using Server-Side Encryption with Amazon S3-Managed
Encryption Keys (SSE-S3) - Amazon Simple Storage Service
Question # 4
A data engineer needs to maintain a central metadata repository that users access throughAmazon EMR and Amazon Athena queries. The repository needs to provide the schemaand properties of many tables. Some of the metadata is stored in Apache Hive. The dataengineer needs to import the metadata from Hive into the central metadata repository.Which solution will meet these requirements with the LEAST development effort?
A. Use Amazon EMR and Apache Ranger. B. Use a Hive metastore on an EMR cluster. C. Use the AWS Glue Data Catalog. D. Use a metastore on an Amazon RDS for MySQL DB instance.
Answer: C
Explanation: The AWS Glue Data Catalog is an Apache Hive metastore-compatible
catalog that provides a central metadata repository for various data sources and formats.
You can use the AWS Glue Data Catalog as an external Hive metastore for Amazon EMR
and Amazon Athena queries, and import metadata from existing Hive metastores into the Data Catalog. This solution requires the least development effort, as you can use AWS
Glue crawlers to automatically discover and catalog the metadata from Hive, and use the
AWS Glue console, AWS CLI, or Amazon EMR API to configure the Data Catalog as the
Hive metastore. The other options are either more complex or require additional steps,
such as setting up Apache Ranger for security, managing a Hive metastore on an EMR
cluster or an RDS instance, or migrating the metadata manually. References:
Using the AWS Glue Data Catalog as the metastore for Hive (Section: Specifying
AWS Glue Data Catalog as the metastore)
Metadata Management: Hive Metastore vs AWS Glue (Section: AWS Glue Data
Catalog)
AWS Glue Data Catalog support for Spark SQL jobs (Section: Importing metadata
from an existing Hive metastore)
AWS Certified Data Engineer - Associate DEA-C01 Complete Study Guide
(Chapter 5, page 131)
Question # 5
A company is planning to use a provisioned Amazon EMR cluster that runs Apache Sparkjobs to perform big data analysis. The company requires high reliability. A big data teammust follow best practices for running cost-optimized and long-running workloads onAmazon EMR. The team must find a solution that will maintain the company's current levelof performance.Which combination of resources will meet these requirements MOST cost-effectively?(Choose two.)
A. Use Hadoop Distributed File System (HDFS) as a persistent data store. B. Use Amazon S3 as a persistent data store. C. Use x86-based instances for core nodes and task nodes. D. Use Graviton instances for core nodes and task nodes. E. Use Spot Instances for all primary nodes.
Answer: B,D
Explanation: The best combination of resources to meet the requirements of high
reliability, cost-optimization, and performance for running Apache Spark jobs on Amazon
EMR is to use Amazon S3 as a persistent data store and Graviton instances for core nodes
and task nodes.
Amazon S3 is a highly durable, scalable, and secure object storage service that can store
any amount of data for a variety of use cases, including big data analytics1. Amazon S3 is
a better choice than HDFS as a persistent data store for Amazon EMR, as it decouples the
storage from the compute layer, allowing for more flexibility and cost-efficiency. Amazon S3
also supports data encryption, versioning, lifecycle management, and cross-region
replication1. Amazon EMR integrates seamlessly with Amazon S3, using EMR File System
(EMRFS) to access data stored in Amazon S3 buckets2. EMRFS also supports consistent
view, which enables Amazon EMR to provide read-after-write consistency for Amazon S3
objects that are accessed through EMRFS2.
Graviton instances are powered by Arm-based AWS Graviton2 processors that deliver up
to 40% better price performance over comparable current generation x86-based
instances3. Graviton instances are ideal for running workloads that are CPU-bound,
memory-bound, or network-bound, such as big data analytics, web servers, and opensource
databases3. Graviton instances are compatible with Amazon EMR, and can beused
for both core nodes and task nodes. Core nodes are responsible for running the data processing frameworks, such as Apache Spark, and storing data in HDFS or the local file
system. Task nodes are optional nodes that can be added to a cluster to increase the
processing power and throughput. By using Graviton instances for both core nodes and
task nodes, you can achieve higher performance and lower cost than using x86-based
instances.
Using Spot Instances for all primary nodes is not a good option, as it can compromise the
reliability and availability of the cluster. Spot Instances are spare EC2 instances that are
available at up to 90% discount compared to On-Demand prices, but they can be
interrupted by EC2 with a two-minute notice when EC2 needs the capacity back. Primary
nodes are the nodes that run the cluster software, such as Hadoop, Spark, Hive, and Hue,
and are essential for the cluster operation. If a primary node is interrupted by EC2, the
cluster will fail or become unstable. Therefore, it is recommended to use On-Demand
Instances or Reserved Instances for primary nodes, and use Spot Instances only for task
nodes that can tolerate interruptions. References:
A company wants to implement real-time analytics capabilities. The company wants to useAmazon Kinesis Data Streams and Amazon Redshift to ingest and process streaming dataat the rate of several gigabytes per second. The company wants to derive near real-timeinsights by using existing business intelligence (BI) and analytics tools.Which solution will meet these requirements with the LEAST operational overhead?
A. Use Kinesis Data Streams to stage data in Amazon S3. Use the COPY command toload data from Amazon S3 directly into Amazon Redshift to make the data immediatelyavailable for real-time analysis. B. Access the data from Kinesis Data Streams by using SQL queries. Create materializedviews directly on top of the stream. Refresh the materialized views regularly to query themost recent stream data. C. Create an external schema in Amazon Redshift to map the data from Kinesis DataStreams to an Amazon Redshift object. Create a materialized view to read data from thestream. Set the materialized view to auto refresh. D. Connect Kinesis Data Streams to Amazon Kinesis Data Firehose. Use Kinesis DataFirehose to stage the data in Amazon S3. Use the COPY command to load the data fromAmazon S3 to a table in Amazon Redshift.
Answer: C
Explanation: This solution meets the requirements of implementing real-time analytics
capabilities with the least operational overhead. By creating an external schema in Amazon
Redshift, you can access the data from Kinesis Data Streams using SQL queries without
having to load the data into the cluster. By creating a materialized view on top of the
stream, you can store the results of the query in the cluster and make them available for
analysis. By setting the materialized view to auto refresh, you can ensure that the view is
updated with the latest data from the stream at regular intervals. This way, you can derive
near real-time insights by using existing BI and analytics tools. References:
Amazon Redshift streaming ingestion
Creating an external schema for Amazon Kinesis Data Streams
Creating a materialized view for Amazon Kinesis Data Streams
Question # 7
A company stores details about transactions in an Amazon S3 bucket. The company wantsto log all writes to the S3 bucket into another S3 bucket that is in the same AWS Region.Which solution will meet this requirement with the LEAST operational effort?
A. Configure an S3 Event Notifications rule for all activities on the transactions S3 bucket toinvoke an AWS Lambda function. Program the Lambda function to write the event toAmazon Kinesis Data Firehose. Configure Kinesis Data Firehose to write the event to thelogs S3 bucket. B. Create a trail of management events in AWS CloudTraiL. Configure the trail to receivedata from the transactions S3 bucket. Specify an empty prefix and write-only events.Specify the logs S3 bucket as the destination bucket. C. Configure an S3 Event Notifications rule for all activities on the transactions S3 bucket toinvoke an AWS Lambda function. Program the Lambda function to write the events to thelogs S3 bucket. D. Create a trail of data events in AWS CloudTraiL. Configure the trail to receive data fromthe transactions S3 bucket. Specify an empty prefix and write-only events. Specify the logsS3 bucket as the destination bucket.
Answer: D
Explanation: This solution meets the requirement of logging all writes to the S3 bucket
into another S3 bucket with the least operational effort. AWS CloudTrail is a service that
records the API calls made to AWS services, including Amazon S3. By creating a trail of
data events, you can capture the details of the requests that are made to the transactions
S3 bucket, such as the requester, the time, the IP address, and the response elements. By
specifying an empty prefix and write-only events, you can filter the data events to only
include the ones that write to the bucket. By specifying the logs S3 bucket as the
destination bucket, you can store the CloudTrail logs in another S3 bucket that is in the
same AWS Region. This solution does not require any additional coding or configuration,
and it is more scalable and reliable than using S3 Event Notifications and Lambda
functions. References:
Logging Amazon S3 API calls using AWS CloudTrail
Creating a trail for data events
Enabling Amazon S3 server access logging
Question # 8
A data engineer has a one-time task to read data from objects that are in Apache Parquetformat in an Amazon S3 bucket. The data engineer needs to query only one column of thedata.Which solution will meet these requirements with the LEAST operational overhead?
A. Confiqure an AWS Lambda function to load data from the S3 bucket into a pandasdataframe- Write a SQL SELECT statement on the dataframe to query the requiredcolumn. B. Use S3 Select to write a SQL SELECT statement to retrieve the required column fromthe S3 objects. C. Prepare an AWS Glue DataBrew project to consume the S3 objects and to query the required column. D. Run an AWS Glue crawler on the S3 objects. Use a SQL SELECT statement in AmazonAthena to query the required column.
Answer: B
Explanation: Option B is the best solution to meet the requirements with the least
operational overhead because S3 Select is a feature that allows you to retrieve only a
subset of data from an S3 object by using simple SQL expressions. S3 Select works on
objects stored in CSV, JSON, or Parquet format. By using S3 Select, you can avoid the
need to download and process the entire S3 object, which reduces the amount of data
transferred and the computation time. S3 Select is also easy to use and does not require
any additional services or resources.
Option A is not a good solution because it involves writing custom code and configuring an
AWS Lambda function to load data from the S3 bucket into a pandas dataframe and query
the required column. This option adds complexity and latency to the data retrieval process
and requires additional resources and configuration.Moreover, AWS Lambda has
limitations on the execution time, memory, and concurrency, which may affect the
performance and reliability of the data retrieval process.
Option C is not a good solution because it involves creating and running an AWS Glue
DataBrew project to consume the S3 objects and query the required column. AWS Glue
DataBrew is a visual data preparation tool that allows you to clean, normalize, and
transform data without writing code. However, in this scenario, the data is already in
Parquet format, which is a columnar storage format that is optimized for analytics.
Therefore, there is no need to use AWS Glue DataBrew to prepare the data. Moreover,
AWS Glue DataBrew adds extra time and cost to the data retrieval process and requires
additional resources and configuration.
Option D is not a good solution because it involves running an AWS Glue crawler on the S3
objects and using a SQL SELECT statement in Amazon Athena to query the required
column. An AWS Glue crawler is a service that can scan data sources and create metadata
tables in the AWS Glue Data Catalog. The Data Catalog is a central repository that stores
information about the data sources, such as schema, format, and location. Amazon Athena
is a serverless interactive query service that allows you to analyze data in S3 using
standard SQL. However, in this scenario, the schema and format of the data are already
known and fixed, so there is no need to run a crawler to discover them. Moreover, running
a crawler and using Amazon Athena adds extra time and cost to the data retrieval process
and requires additional services and configuration.
References:
AWS Certified Data Engineer - Associate DEA-C01 Complete Study Guide
S3 Select and Glacier Select - Amazon Simple Storage Service
AWS Lambda - FAQs
What Is AWS Glue DataBrew? - AWS Glue DataBrew
Populating the AWS Glue Data Catalog - AWS Glue What is Amazon Athena? - Amazon Athena
Question # 9
A retail company has a customer data hub in an Amazon S3 bucket. Employees from manycountries use the data hub to support company-wide analytics. A governance team mustensure that the company's data analysts can access data only for customers who arewithin the same country as the analysts.Which solution will meet these requirements with the LEAST operational effort?
A. Create a separate table for each country's customer data. Provide access to eachanalyst based on the country that the analyst serves. B. Register the S3 bucket as a data lake location in AWS Lake Formation. Use the LakeFormation row-level security features to enforce the company's access policies. C. Move the data to AWS Regions that are close to the countries where the customers are.Provide access to each analyst based on the country that the analyst serves. D. Load the data into Amazon Redshift. Create a view for each country. Create separate1AM roles for each country to provide access to data from each country. Assign theappropriate roles to the analysts.
Answer: B
Explanation: AWS Lake Formation is a service that allows you to easily set up, secure,
and manage data lakes. One of the features of Lake Formation is row-level security, which
enables you to control access to specific rows or columns of data based on the identity or
role of the user. This feature is useful for scenarios where you need to restrict access to
sensitive or regulated data, such as customer data from different countries. By registering
the S3 bucket as a data lake location in Lake Formation, you can use the Lake Formation
console or APIs to define and apply row-level security policies to the data in the bucket.
You can also use Lake Formation blueprints to automate the ingestion and transformation
of data from various sources into the data lake. This solution requires the least operational
effort compared to the other options, as it does not involve creating or moving data, or
managing multiple tables, views, or roles. References:
AWS Lake Formation
Row-Level Security
AWS Certified Data Engineer - Associate DEA-C01 Complete Study Guide,
Chapter 4: Data Lakes and Data Warehouses, Section 4.2: AWS Lake Formation
Question # 10
A company uses Amazon RDS to store transactional data. The company runs an RDS DBinstance in a private subnet. A developer wrote an AWS Lambda function with defaultsettings to insert, update, or delete data in the DB instance.The developer needs to give the Lambda function the ability to connect to the DB instanceprivately without using the public internet.Which combination of steps will meet this requirement with the LEAST operationaloverhead? (Choose two.)
A. Turn on the public access setting for the DB instance. B. Update the security group of the DB instance to allow only Lambda function invocationson the database port. C. Configure the Lambda function to run in the same subnet that the DB instance uses. D. Attach the same security group to the Lambda function and the DB instance. Include aself-referencing rule that allows access through the database port. E. Update the network ACL of the private subnet to include a self-referencing rule thatallows access through the database port.
Answer: C,D
Explanation: To enable the Lambda function to connect to the RDS DB instance privately
without using the public internet, the best combination of steps is to configure the Lambda
function to run in the same subnet that the DB instance uses, and attach the same security
group to the Lambda function and the DB instance. This way, the Lambda function and the
DB instance can communicate within the same private network, and the security group can
allow traffic between them on the database port. This solution has the least operational
overhead, as it does not require any changes to the public access setting, the network
ACL, or the security group of the DB instance.
The other options are not optimal for the following reasons:
A. Turn on the public access setting for the DB instance. This option is not
recommended, as it would expose the DB instance to the public internet, which
can compromise the security and privacy of the data. Moreover, this option would
not enable the Lambda function to connect to the DB instance privately, as it would
still require the Lambda function to use the public internet to access the DB
instance.
B. Update the security group of the DB instance to allow only Lambda function
invocations on the database port. This option is not sufficient, as it would only
modify the inbound rules of the security group of the DB instance, but not the
outbound rules of the security group of the Lambda function. Moreover, this option would not enable the Lambda function to connect to the DB instance privately, as it
would still require the Lambda function to use the public internet to access the DB
instance.
E. Update the network ACL of the private subnet to include a self-referencing rule
that allows access through the database port. This option is not necessary, as the
network ACL of the private subnet already allows all traffic within the subnet by
default. Moreover, this option would not enable the Lambda function to connect to
the DB instance privately, as it would still require the Lambda function to use the
public internet to access the DB instance.
References:
1: Connecting to an Amazon RDS DB instance
2: Configuring a Lambda function to access resources in a VPC
3: Working with security groups
: Network ACLs
Question # 11
A company has five offices in different AWS Regions. Each office has its own humanresources (HR) department that uses a unique IAM role. The company stores employeerecords in a data lake that is based on Amazon S3 storage. A data engineering team needs to limit access to the records. Each HR department shouldbe able to access records for only employees who are within the HR department's Region.Which combination of steps should the data engineering team take to meet thisrequirement with the LEAST operational overhead? (Choose two.)
A. Use data filters for each Region to register the S3 paths as data locations. B. Register the S3 path as an AWS Lake Formation location. C. Modify the IAM roles of the HR departments to add a data filter for each department'sRegion. D. Enable fine-grained access control in AWS Lake Formation. Add a data filter for eachRegion. E. Create a separate S3 bucket for each Region. Configure an IAM policy to allow S3access. Restrict access based on Region.
Answer: B,D
Explanation: AWS Lake Formation is a service that helps you build, secure, and manage
data lakes on Amazon S3. You can use AWS Lake Formation to register the S3 path as a
data lake location, and enable fine-grained access control to limit access to the records
based on the HR department’s Region. You can use data filters to specify which S3
prefixes or partitions each HR department can access, and grant permissions to the IAM
roles of the HR departments accordingly. This solution will meet the requirement with the
least operational overhead, as it simplifies the data lake management and security, and
leverages the existing IAM roles of the HR departments12.
The other options are not optimal for the following reasons:
A. Use data filters for each Region to register the S3 paths as data locations. This
option is not possible, as data filters are not used to register S3 paths as data
locations, but to grant permissions to access specific S3 prefixes or partitions
within a data location. Moreover, this option does not specify how to limit access to
the records based on the HR department’s Region.
C. Modify the IAM roles of the HR departments to add a data filter for each
department’s Region. This option is not possible, as data filters are not added to
IAM roles, but to permissions granted by AWS Lake Formation. Moreover, this
option does not specify how to register the S3 path as a data lake location, or how
to enable fine-grained access control in AWS Lake Formation.
E. Create a separate S3 bucket for each Region. Configure an IAM policy to allow
S3 access. Restrict access based on Region. This option is not recommended, as
it would require more operational overhead to create and manage multiple S3
buckets, and to configure and maintain IAM policies for each HR department.
Moreover, this option does not leverage the benefits of AWS Lake Formation, such
as data cataloging, data transformation, and data governance.
References:
1: AWS Lake Formation
2: AWS Lake Formation Permissions
: AWS Identity and Access Management
: Amazon S3
Question # 12
A healthcare company uses Amazon Kinesis Data Streams to stream real-time health datafrom wearable devices, hospital equipment, and patient records.A data engineer needs to find a solution to process the streaming data. The data engineerneeds to store the data in an Amazon Redshift Serverless warehouse. The solution must support near real-time analytics of the streaming data and the previous day's data.Which solution will meet these requirements with the LEAST operational overhead?
A. Load data into Amazon Kinesis Data Firehose. Load the data into Amazon Redshift. B. Use the streaming ingestion feature of Amazon Redshift. C. Load the data into Amazon S3. Use the COPY command to load the data into AmazonRedshift. D. Use the Amazon Aurora zero-ETL integration with Amazon Redshift.
Answer: B
Explanation: The streaming ingestion feature of Amazon Redshift enables you to ingest
data from streaming sources, such as Amazon Kinesis Data Streams, into Amazon
Redshift tables in near real-time. You can use the streaming ingestion feature to process
the streaming data from the wearable devices, hospital equipment, and patient records.
The streaming ingestion feature also supports incremental updates, which means you can
append new data or update existing data in the Amazon Redshift tables. This way, you can
store the data in an Amazon Redshift Serverless warehouse and support near real-time
analytics of the streaming data and the previous day’s data. This solution meets the
requirements with the least operational overhead, as it does not require any additional
services or components to ingest and process the streaming data. The other options are
either not feasible or not optimal. Loading data into Amazon Kinesis Data Firehose and
then into Amazon Redshift (option A) would introduce additional latency and cost, as well
as require additional configuration and management. Loading data into Amazon S3 and
then using the COPY command to load the data into Amazon Redshift (option C) would
also introduce additional latency and cost, as well as require additional storage space and
ETL logic. Using the Amazon Aurora zero-ETL integration with Amazon Redshift (option D)
would not work, as it requires the data to be stored in Amazon Aurora first, which is not the
case for the streaming data from the healthcare company. References:
Using streaming ingestion with Amazon Redshift
AWS Certified Data Engineer - Associate DEA-C01 Complete Study Guide,
Chapter 3: Data Ingestion and Transformation, Section 3.5: Amazon Redshift
Streaming Ingestion
Question # 13
A company is migrating a legacy application to an Amazon S3 based data lake. A dataengineer reviewed data that is associated with the legacy application. The data engineerfound that the legacy data contained some duplicate information.The data engineer must identify and remove duplicate information from the legacyapplication data.Which solution will meet these requirements with the LEAST operational overhead?
A. Write a custom extract, transform, and load (ETL) job in Python. Use theDataFramedrop duplicatesf) function by importingthe Pandas library to perform datadeduplication. B. Write an AWS Glue extract, transform, and load (ETL) job. Usethe FindMatchesmachine learning(ML) transform to transform the data to perform data deduplication. C. Write a custom extract, transform, and load (ETL) job in Python. Import the Pythondedupe library. Use the dedupe library to perform data deduplication. D. Write an AWS Glue extract, transform, and load (ETL) job. Import the Python dedupelibrary. Use the dedupe library to perform data deduplication.
Answer: B
Explanation: AWS Glue is a fully managed serverless ETL service that can handle data
deduplication with minimal operational overhead. AWS Glue provides a built-in ML
transform called FindMatches, which can automatically identify and group similar records in
a dataset. FindMatches can also generate a primary key for each group of records and
remove duplicates. FindMatches does not require any coding or prior ML experience, as it
can learn from a sample of labeled data provided by the user. FindMatches can also scale
to handle large datasets and optimize the cost and performance of the ETL job.
References:
AWS Glue
FindMatches ML Transform
AWS Certified Data Engineer - Associate DEA-C01 Complete Study Guide
Question # 14
A company needs to build a data lake in AWS. The company must provide row-level dataaccess and column-level data access to specific teams. The teams will access the data byusing Amazon Athena, Amazon Redshift Spectrum, and Apache Hive from Amazon EMR.Which solution will meet these requirements with the LEAST operational overhead?
A. Use Amazon S3 for data lake storage. Use S3 access policies to restrict data access byrows and columns. Provide data access throughAmazon S3. B. Use Amazon S3 for data lake storage. Use Apache Ranger through Amazon EMR torestrict data access byrows and columns. Providedata access by using Apache Pig. C. Use Amazon Redshift for data lake storage. Use Redshift security policies to restrictdata access byrows and columns. Provide data accessby usingApache Spark and AmazonAthena federated queries. D. UseAmazon S3 for data lake storage. Use AWS Lake Formation to restrict data accessby rows and columns. Provide data access through AWS Lake Formation.
Answer: D
Explanation: Option D is the best solution to meet the requirements with the least
operational overhead because AWS Lake Formation is a fully managed service that
simplifies the process of building, securing, and managing data lakes. AWS Lake Formation allows you to define granular data access policies at the row and column level
for different users and groups. AWS Lake Formation also integrates with Amazon Athena,
Amazon Redshift Spectrum, and Apache Hive on Amazon EMR, enabling these services to
access the data in the data lake through AWS Lake Formation.
Option A is not a good solution because S3 access policies cannot restrict data access by
rows and columns. S3 access policies are based on the identity and permissions of the
requester, the bucket and object ownership, and the object prefix and tags. S3 access
policies cannot enforce fine-grained data access control at the row and column level.
Option B is not a good solution because it involves using Apache Ranger and Apache Pig,
which are not fully managed services and require additional configuration and
maintenance. Apache Ranger is a framework that provides centralized security
administration for data stored in Hadoop clusters, such as Amazon EMR. Apache Ranger
can enforce row-level and column-level access policies for Apache Hive tables. However,
Apache Ranger is not a native AWS service and requires manual installation and
configuration on Amazon EMR clusters. Apache Pig is a platform that allows you to analyze
large data sets using a high-level scripting language called Pig Latin. Apache Pig can
access data stored in Amazon S3 and process it using Apache Hive. However,Apache Pig
is not a native AWS service and requires manual installation and configuration on Amazon
EMR clusters.
Option C is not a good solution because Amazon Redshift is not a suitable service for data
lake storage. Amazon Redshift is a fully managed data warehouse service that allows you
to run complex analytical queries using standard SQL. Amazon Redshift can enforce rowlevel
and column-level access policies for different users and groups. However, Amazon
Redshift is not designed to store and process large volumes of unstructured or semistructured
data, which are typical characteristics of data lakes. Amazon Redshift is also
more expensive and less scalable than Amazon S3 for data lake storage.
References:
AWS Certified Data Engineer - Associate DEA-C01 Complete Study Guide
What Is AWS Lake Formation? - AWS Lake Formation
Using AWS Lake Formation with Amazon Athena - AWS Lake Formation
Using AWS Lake Formation with Amazon Redshift Spectrum - AWS Lake
Formation
Using AWS Lake Formation with Apache Hive on Amazon EMR - AWS Lake
Formation
Using Bucket Policies and User Policies - Amazon Simple Storage Service
Apache Ranger
Apache Pig
What Is Amazon Redshift? - Amazon Redshift
Question # 15
A company uses an Amazon Redshift provisioned cluster as its database. The Redshiftcluster has five reserved ra3.4xlarge nodes and uses key distribution.A data engineer notices that one of the nodes frequently has a CPU load over 90%. SQLQueries that run on the node are queued. The other four nodes usually have a CPU loadunder 15% during daily operations.The data engineer wants to maintain the current number of compute nodes. The dataengineer also wants to balance the load more evenly across all five compute nodes.Which solution will meet these requirements?
A. Change the sort key to be the data column that is most often used in a WHERE clauseof the SQL SELECT statement. B. Change the distribution key to the table column that has the largest dimension. C. Upgrade the reserved node from ra3.4xlarqe to ra3.16xlarqe. D. Change the primary key to be the data column that is most often used in a WHEREclause of the SQL SELECT statement.
Answer: B
Explanation: Changing the distribution key to the table column that has the largest
dimension will help to balance the load more evenly across all five compute nodes. The
distribution key determines how the rows of a table are distributed among the slices of the
cluster. If the distribution key is not chosen wisely, it can cause data skew, meaning some
slices will have more data than others, resulting in uneven CPU load and query
performance. By choosing the table column that has the largest dimension, meaning the
column that has the most distinct values, as the distribution key, the data engineer can
ensure that the rows are distributed more uniformly across the slices, reducing data skew
and improving query performance.
The other options are not solutions that will meet the requirements. Option A, changing the
sort key to be the data column that is most often used in a WHERE clause of the SQL
SELECT statement, will not affect the data distribution or the CPU load. The sort key
determines the order in which the rows of a table are stored on disk, which can improve the
performance of range-restricted queries, but not the load balancing. Option C, upgrading
the reserved node from ra3.4xlarge to ra3.16xlarge, will not maintain the current number of
compute nodes, as it will increase the cost and the capacity of the cluster. Option D,
changing the primary key to be the data column that is most often used in a WHERE
clause of the SQL SELECT statement, will not affect the data distribution or the CPU load
either. The primary key is a constraint that enforces the uniqueness of the rows in a table,
but it does not influence the data layout or the query optimization. References:
Choosing a data distribution style
Choosing a data sort key
Working with primary keys
Question # 16
A company is developing an application that runs on Amazon EC2 instances. Currently, thedata that the application generates is temporary. However, the company needs to persistthe data, even if the EC2 instances are terminated.A data engineer must launch new EC2 instances from an Amazon Machine Image (AMI)and configure the instances to preserve the data.Which solution will meet this requirement?
A. Launch new EC2 instances by using an AMI that is backed by an EC2 instance storevolume that contains the application data. Apply the default settings to the EC2 instances. B. Launch new EC2 instances by using an AMI that is backed by a root Amazon ElasticBlock Store (Amazon EBS) volume that contains the application data. Apply the defaultsettings to the EC2 instances. C. Launch new EC2 instances by using an AMI that is backed by an EC2 instance storevolume. Attach an Amazon Elastic Block Store (Amazon EBS) volume to contain theapplication data. Apply the default settings to the EC2 instances. D. Launch new EC2 instances by using an AMI that is backed by an Amazon Elastic BlockStore (Amazon EBS) volume. Attach an additional EC2 instance store volume to containthe application data. Apply the default settings to the EC2 instances.
Answer: C
Explanation: Amazon EC2 instances can use two types of storage volumes: instance
store volumes and Amazon EBS volumes. Instance store volumes are ephemeral, meaning
they are only attached to the instance for the duration of its life cycle. If the instance is
stopped, terminated, or fails, the data on the instance store volume is lost. Amazon EBS
volumes are persistent, meaning they can be detached from the instance and attached to
another instance, and the data on the volume is preserved. To meet the requirement of
persisting the data even if the EC2 instances are terminated, the data engineer must use
Amazon EBS volumes to store the application data. The solution is to launch new EC2
instances by using an AMI that is backed by an EC2 instance store volume, which is the
default option for most AMIs. Then, the data engineer must attach an Amazon EBS volume
to each instance and configure the application to write the data to the EBS volume. This
way, the data will be saved on the EBS volume and can be accessed by another instance if
needed. The data engineer can apply the default settings to the EC2 instances, as there is
no need to modify the instance type, security group, or IAM role for this solution. The other
options are either not feasible or not optimal. Launching new EC2 instances by using an
AMI that is backed by an EC2 instance store volume that contains the application data
(option A) or by using an AMI that is backed by a root Amazon EBS volume that contains
the application data (option B) would not work, as the data on the AMI would be outdated
and overwritten by the new instances. Attaching an additional EC2 instance store volume
to contain the application data (option D)would not work, as the data on the instance store
volume would be lost if the instance is terminated. References:
Amazon EC2 Instance Store
Amazon EBS Volumes
AWS Certified Data Engineer - Associate DEA-C01 Complete Study Guide,
Chapter 2: Data Store Management, Section 2.1: Amazon EC2
Question # 17
A data engineer must ingest a source of structured data that is in .csv format into anAmazon S3 data lake. The .csv files contain 15 columns. Data analysts need to runAmazon Athena queries on one or two columns of the dataset. The data analysts rarelyquery the entire file.Which solution will meet these requirements MOST cost-effectively?
A. Use an AWS Glue PySpark job to ingest the source data into the data lake in .csvformat. B. Create an AWS Glue extract, transform, and load (ETL) job to read from the .csvstructured data source. Configure the job to ingest the data into the data lake in JSONformat.C. Use an AWS Glue PySpark job to ingest the source data into the data lake in ApacheAvro format. D. Create an AWS Glue extract, transform, and load (ETL) job to read from the .csvstructured data source. Configure the job to write the data into the data lake in ApacheParquet format.
Answer: D
Explanation: Amazon Athena is a serverless interactive query service that allows you to
analyze data in Amazon S3 using standard SQL. Athena supports various data formats,
such as CSV,JSON, ORC, Avro, and Parquet. However, not all data formats are equally
efficient for querying. Some data formats, such as CSV and JSON, are row-oriented,
meaning that they store data as a sequence of records, each with the same fields. Roworiented
formats are suitable for loading and exporting data, but they are not optimal for
analytical queries that often access only a subset of columns. Row-oriented formats also
do not support compression or encoding techniques that can reduce the data size and
improve the query performance.
On the other hand, some data formats, such as ORC and Parquet, are column-oriented,
meaning that they store data as a collection of columns, each with a specific data type.
Column-oriented formats are ideal for analytical queries that often filter, aggregate, or join
data by columns. Column-oriented formats also support compression and encoding
techniques that can reduce the data size and improve the query performance. For
example, Parquet supports dictionary encoding, which replaces repeated values with
numeric codes, and run-length encoding, which replaces consecutive identical values with
a single value and a count. Parquet also supports various compression algorithms, such as
Snappy, GZIP, and ZSTD, that can further reduce the data size and improve the query
performance.
Therefore, creating an AWS Glue extract, transform, and load (ETL) job to read from the
.csv structured data source and writing the data into the data lake in Apache Parquet
format will meet the requirements most cost-effectively. AWS Glue is a fully managed service that provides a serverless data integration platform for data preparation, data
cataloging, and data loading. AWS Glue ETL jobs allow you to transform and load data
from various sources into various targets, using either a graphical interface (AWS Glue
Studio) or a code-based interface (AWS Glue console or AWS Glue API). By using AWS
Glue ETL jobs, you can easily convert the data from CSV to Parquet format, without having
to write or manage any code. Parquet is a column-oriented format that allows Athena to
scan only the relevant columns and skip the rest, reducing the amount of data read from
S3. This solution will also reduce the cost of Athena queries, as Athena charges based on
the amount of data scanned from S3.
The other options are not as cost-effective as creating an AWS Glue ETL job to write the
data into the data lake in Parquet format. Using an AWS Glue PySpark job to ingest the
source data into the data lake in .csv format will not improve the query performance or
reduce the query cost, as .csv is a row-oriented format that does not support columnar
access or compression. Creating an AWS Glue ETL job to ingest the data into the data
lake in JSON format will not improve the query performance or reduce the query cost, as
JSON is also a row-oriented format that does not support columnar access or compression.
Using an AWS Glue PySpark job to ingest the source data into the data lake in Apache
Avro format will improve the query performance, as Avro is a column-oriented format that
supports compression and encoding, but it will require more operational effort, as you will
need to write and maintain PySpark code to convert the data from CSV to Avro format.
References:
Amazon Athena
Choosing the Right Data Format
AWS Glue
[AWS Certified Data Engineer - Associate DEA-C01 Complete Study Guide],
Chapter 5: Data Analysis and Visualization, Section 5.1: Amazon Athena
Question # 18
A data engineer uses Amazon Redshift to run resource-intensive analytics processes onceevery month. Every month, the data engineer creates a new Redshift provisioned cluster.The data engineer deletes the Redshift provisioned cluster after the analytics processesare complete every month. Before the data engineer deletes the cluster each month, thedata engineer unloads backup data from the cluster to an Amazon S3 bucket.The data engineer needs a solution to run the monthly analytics processes that does notrequire the data engineer to manage the infrastructure manually.Which solution will meet these requirements with the LEAST operational overhead?
A. Use Amazon Step Functions to pause the Redshift cluster when the analytics processesare complete and to resume the cluster to run new processes every month. B. Use Amazon Redshift Serverless to automatically process the analytics workload. C. Use the AWS CLI to automatically process the analytics workload. D. Use AWS CloudFormation templates to automatically process the analytics workload.
Answer: B
Explanation: Amazon Redshift Serverless is a new feature of Amazon Redshift that
enables you to run SQL queries on data in Amazon S3 without provisioning or managing
any clusters. You can use Amazon Redshift Serverless to automatically process the
analytics workload, as it scales up and down the compute resources based on the query
demand, and charges you only for the resources consumed. This solution will meet the
requirements with the least operational overhead, as it does not require the data engineer
to create, delete, pause, or resume any Redshift clusters, or to manage any infrastructure
manually. You can use the Amazon Redshift Data API to run queries from the AWS CLI,
AWS SDK, or AWS Lambda functions12.
The other options are not optimal for the following reasons:
A. Use Amazon Step Functions to pause the Redshift cluster when the analytics
processes are complete and to resume the cluster to run new processes every
month. This option is not recommended, as it would still require the data engineer
to create and delete a new Redshift provisioned cluster every month, which can
incur additional costs and time. Moreover, this option would require the data
engineer to use Amazon Step Functions to orchestrate the workflow of pausing
and resuming the cluster, which can add complexity and overhead.
C. Use the AWS CLI to automatically process the analytics workload. This option
is vague and does not specify how the AWS CLI is used to process the analytics
workload. The AWS CLI can be used to run queries on data in Amazon S3 using
Amazon Redshift Serverless, Amazon Athena, or Amazon EMR, but each of these
services has different features and benefits. Moreover, this option does not
address the requirement of not managing the infrastructure manually, as the data
engineer may still need to provision and configure some resources, such as
Amazon EMR clusters or Amazon Athena workgroups.
D. Use AWS CloudFormation templates to automatically process the analytics
workload. This option is also vague and does not specify how AWS
CloudFormation templates are used to process the analytics workload. AWS
CloudFormation is a service that lets you model and provision AWS resources
using templates. You can use AWS CloudFormation templates to create and
delete a Redshift provisioned cluster every month, or to create and configure other
AWS resources, such as Amazon EMR, Amazon Athena, or Amazon Redshift
Serverless. However, this option does not address the requirement of not
managing the infrastructure manually, as the data engineer may still need to write
and maintain the AWS CloudFormation templates, and to monitor the status and
performance of the resources.
References:
1: Amazon Redshift Serverless
2: Amazon Redshift Data API
: Amazon Step Functions
: AWS CLI
: AWS CloudFormation
Question # 19
A financial company wants to use Amazon Athena to run on-demand SQL queries on apetabyte-scale dataset to support a business intelligence (BI) application. An AWS Glue jobthat runs during non-business hours updates the dataset once every day. The BIapplication has a standard data refresh frequency of 1 hour to comply with companypolicies. A data engineer wants to cost optimize the company's use of Amazon Athena withoutadding any additional infrastructure costs.Which solution will meet these requirements with the LEAST operational overhead?
A. Configure an Amazon S3 Lifecycle policy to move data to the S3 Glacier Deep Archivestorage class after 1 day B. Use the query result reuse feature of Amazon Athena for the SQL queries. C. Add an Amazon ElastiCache cluster between the Bl application and Athena. D. Change the format of the files that are in the dataset to Apache Parquet.
Answer: B
Explanation: The best solution to cost optimize the company’s use of Amazon Athena
without adding any additional infrastructure costs is to use the query result reuse feature of
AmazonAthena for the SQL queries. This feature allows you to run the same query multiple
times without incurring additional charges, as long as the underlying data has not changed
and the query results are still in the query result location in Amazon S31. This feature is
useful for scenarios where you have a petabyte-scale dataset that is updated infrequently,
such as once a day, and you have a BI application that runs the same queries repeatedly,
such as every hour. By using the query result reuse feature, you can reduce the amount of
data scanned by your queries and save on the cost of running Athena. You can enable or
disable this feature at the workgroup level or at the individual query level1.
Option A is not the best solution, as configuring an Amazon S3 Lifecycle policy to move
data to the S3 Glacier Deep Archive storage class after 1 day would not cost optimize the
company’s use of Amazon Athena, but rather increase the cost and complexity. Amazon
S3 Lifecycle policies are rules that you can define to automatically transition objects
between different storage classes based on specified criteria, such as the age of the
object2. S3 Glacier Deep Archive is the lowest-cost storage class in Amazon S3, designed
for long-term data archiving that is accessed once or twice in a year3. While moving data to
S3 Glacier Deep Archive can reduce the storage cost, it would also increase the retrieval
cost and latency, as it takes up to 12 hours to restore the data from S3 Glacier Deep
Archive3. Moreover, Athena does not support querying data that is in S3 Glacier or S3
Glacier Deep Archive storage classes4. Therefore, using this option would not meet the
requirements of running on-demand SQL queries on the dataset.
Option C is not the best solution, as adding an Amazon ElastiCache cluster between the BI
application and Athena would not cost optimize the company’s use of Amazon Athena, but
rather increase the cost and complexity. Amazon ElastiCache is a service that offers fully
managed in-memory data stores, such as Redis and Memcached, that can improve the
performance and scalability of web applications by caching frequently accessed data.
While using ElastiCache can reduce the latency and load on the BI application, it would not
reduce the amount of data scanned by Athena, which is the main factor that determines the
cost of running Athena. Moreover, using ElastiCache would introduce additional infrastructure costs and operational overhead, as you would have to provision, manage,
and scale the ElastiCache cluster, and integrate it with the BI application and Athena.
Option D is not the best solution, as changing the format of the files that are in the dataset
to Apache Parquet would not cost optimize the company’s use of Amazon Athena without
adding any additional infrastructure costs, but rather increase the complexity. Apache
Parquet is a columnar storage format that can improve the performance of analytical
queries by reducing the amount of data that needs to be scanned and providing efficient
compression and encoding schemes. However,changing the format of the files that are in
the dataset to Apache Parquet would require additional processing and transformation
steps, such as using AWS Glue or Amazon EMR to convert the files from their original
format to Parquet, and storing the converted files in a separate location in Amazon S3. This
would increase the complexity and the operational overhead of the data pipeline, and also
incur additional costs for using AWS Glue or Amazon EMR. References:
Query result reuse
Amazon S3 Lifecycle
S3 Glacier Deep Archive
Storage classes supported by Athena
[What is Amazon ElastiCache?]
[Amazon Athena pricing]
[Columnar Storage Formats]
AWS Certified Data Engineer - Associate DEA-C01 Complete Study Guide
Question # 20
A company uses an Amazon Redshift cluster that runs on RA3 nodes. The company wantsto scale read and write capacity to meet demand. A data engineer needs to identify asolution that will turn on concurrency scaling.Which solution will meet this requirement?
A. Turn on concurrency scaling in workload management (WLM) for Redshift Serverlessworkgroups. B. Turn on concurrency scaling at the workload management (WLM) queue level in theRedshift cluster. C. Turn on concurrency scaling in the settings duringthe creation of andnew Redshiftcluster. D. Turn on concurrency scaling for the daily usage quota for the Redshift cluster.
Answer: B
Explanation: Concurrency scaling is a feature that allows you to support thousands of
concurrent users and queries, with consistently fast query performance. When you turn on
concurrency scaling, Amazon Redshift automatically adds query processing power in
seconds to process queries without any delays. You can manage which queries are sent to
the concurrency-scaling cluster by configuring WLM queues. To turn on concurrency
scaling for a queue, set the Concurrency Scaling mode value to auto. The other options are
either incorrect or irrelevant, as they do not enable concurrency scaling for the existing
Redshift cluster on RA3 nodes. References:
Working with concurrency scaling - Amazon Redshift
Amazon Redshift Concurrency Scaling - Amazon Web Services
AWS Certified Data Engineer - Associate DEA-C01 Complete Study Guide
(Chapter 6, page 163)c
Question # 21
A company has a production AWS account that runs company workloads. The company'ssecurity team created a security AWS account to store and analyze security logs from theproduction AWS account. The security logs in the production AWS account are stored inAmazon CloudWatch Logs. The company needs to use Amazon Kinesis Data Streams to deliver the security logs tothe security AWS account.Which solution will meet these requirements?
A. Create a destination data stream in the production AWS account. In the security AWSaccount, create an IAM role that has cross-account permissions to Kinesis Data Streams inthe production AWS account. B. Create a destination data stream in the security AWS account. Create an IAM role and atrust policy to grant CloudWatch Logs the permission to put data into the stream. Create asubscription filter in the security AWS account. C. Create a destination data stream in the production AWS account. In the production AWSaccount, create an IAM role that has cross-account permissions to Kinesis Data Streams inthe security AWS account. D. Create a destination data stream in the security AWS account. Create an IAM role and atrust policy to grant CloudWatch Logs the permission to put data into the stream. Create asubscription filter in the production AWS account.
Answer: D
Explanation: Amazon Kinesis Data Streams is a service that enables you to collect,
process, and analyze real-time streaming data. You can use Kinesis Data Streams to
ingest data from various sources, such as Amazon CloudWatch Logs, and deliver it to
different destinations, such as Amazon S3 or Amazon Redshift. To use Kinesis Data
Streams to deliver the security logs from the production AWS account to the security AWS
account, you need to create a destination data stream in the security AWS account. This
data stream will receive the log data from the CloudWatch Logs service in the production
AWS account. To enable this cross-account data delivery, you need to create an IAM role
and a trust policy in the security AWS account. The IAM role defines the permissions that
the CloudWatch Logs service needs to put data into the destination data stream. The trust
policy allows the production AWS account to assume the IAM role. Finally, you need to
create a subscription filter in the production AWS account. A subscription filter defines the
pattern to match log events and the destination to send the matching events. In this case,
the destination is the destination data stream in the security AWS account. This solution
meets the requirements of using Kinesis Data Streams to deliver the security logs to the
security AWS account. The other options are either not possible or not optimal. You cannot
create a destination data stream in the production AWS account, as this would not deliver
the data to the security AWS account. You cannot create a subscription filter in the security
AWS account, as this would not capture the log events from the production AWS account.
References:
Using Amazon Kinesis Data Streams with Amazon CloudWatch Logs
AWS Certified Data Engineer - Associate DEA-C01 Complete Study Guide,
Chapter 3: Data Ingestion and Transformation, Section 3.3: Amazon Kinesis Data
Streams
Question # 22
A company is migrating on-premises workloads to AWS. The company wants to reduceoverall operational overhead. The company also wants to explore serverless options.The company's current workloads use Apache Pig, Apache Oozie, Apache Spark, ApacheHbase, and Apache Flink. The on-premises workloads process petabytes of data inseconds. The company must maintain similar or better performance after the migration toAWS.Which extract, transform, and load (ETL) service will meet these requirements?
A. AWS Glue B. Amazon EMR C. AWS Lambda D. Amazon Redshift
Answer: A
Explanation: AWS Glue is a fully managed serverless ETL service that can handle
petabytes of data in seconds. AWS Glue can run Apache Spark and Apache Flink jobs
without requiring any infrastructure provisioning or management. AWS Glue can also
integrate with Apache Pig, Apache Oozie, and Apache Hbase using AWS Glue Data
Catalog and AWS Glue workflows. AWS Glue can reduce the overall operational overhead
by automating the data discovery, data preparation, and data loading processes. AWS
Glue can also optimize the cost and performance of ETL jobs by using AWS Glue Job
Bookmarking, AWS Glue Crawlers, and AWS Glue Schema Registry. References:
AWS Glue
AWS Glue Data Catalog
AWS Glue Workflows
[AWS Glue Job Bookmarking]
[AWS Glue Crawlers]
[AWS Glue Schema Registry]
[AWS Certified Data Engineer - Associate DEA-C01 Complete Study Guide]
Question # 23
A data engineering team is using an Amazon Redshift data warehouse for operationalreporting. The team wants to prevent performance issues that might result from longrunningqueries. A data engineer must choose a system table in Amazon Redshift to recordanomalies when a query optimizer identifies conditions that might indicate performanceissues.Which table views should the data engineer use to meet this requirement?
A. STL USAGE CONTROL B. STL ALERT EVENT LOG C. STL QUERY METRICS D. STL PLAN INFO
Answer: B
Explanation: The STL ALERT EVENT LOG table view records anomalies when the query
optimizer identifies conditions that might indicate performance issues. These conditions
include skewed data distribution, missing statistics, nested loop joins, and broadcasted
data. The STL ALERT EVENT LOG table view can help the data engineer to identify and
troubleshoot the root causes of performance issues and optimize the query execution plan.
The other table views are not relevant for this requirement. STL USAGE CONTROL
records the usage limits and quotas for Amazon Redshift resources. STL QUERY
METRICS records the execution time and resource consumption of queries. STL PLAN
INFO records the query execution plan and the steps involved in each query. References:
STL ALERT EVENT LOG
System Tables and Views
AWS Certified Data Engineer - Associate DEA-C01 Complete Study Guide
Question # 24
A media company wants to improve a system that recommends media content to customerbased on user behavior and preferences. To improve the recommendation system, thecompany needs to incorporate insights from third-party datasets into the company'sexisting analytics platform.The company wants to minimize the effort and time required to incorporate third-partydatasets. Which solution will meet these requirements with the LEAST operational overhead?
A. Use API calls to access and integrate third-party datasets from AWS Data Exchange. B. Use API calls to access and integrate third-party datasets from AWS C. Use Amazon Kinesis Data Streams to access and integrate third-party datasets fromAWS CodeCommit repositories. D. Use Amazon Kinesis Data Streams to access and integrate third-party datasets fromAmazon Elastic Container Registry (Amazon ECR).
Answer: A
Explanation: AWS Data Exchange is a service that makes it easy to find, subscribe to,
and use third-party data in the cloud. It provides a secure and reliable way to access and
integrate data from various sources, such as data providers, public datasets, or AWS
services. Using AWS Data Exchange, you can browse and subscribe to data products that
suit your needs, and then use API calls or the AWS Management Console to export the
data to Amazon S3, where you can use it with your existing analytics platform. This solution
minimizes the effort and time required to incorporate third-party datasets, as you do not
need to set up and manage data pipelines, storage, or access controls. You also benefit
from the data quality and freshness provided by the data providers, who can update their
data products as frequently as needed12.
The other options are not optimal for the following reasons:
B. Use API calls to access and integrate third-party datasets from AWS. This
option is vague and does not specify which AWS service or feature is used to
access and integrate third-party datasets. AWS offers a variety of services and
features that can help with data ingestion, processing, and analysis, but not all of
them are suitable for the given scenario. For example, AWS Glue is a serverless
data integration service that can help you discover, prepare, and combine data
from various sources, but it requires you to create and run data extraction,
transformation, and loading (ETL) jobs, which can add operational overhead3.
C. Use Amazon Kinesis Data Streams to access and integrate third-party datasets
from AWS CodeCommit repositories. This option is not feasible, as AWS
CodeCommit is a source control service that hosts secure Git-based repositories,
not a data source that can be accessed by Amazon Kinesis Data Streams.
Amazon Kinesis Data Streams is a service that enables you to capture, process,
and analyze data streams in real time, suchas clickstream data, application logs,
or IoT telemetry. It does not support accessing and integrating data from AWS
CodeCommit repositories, which are meant for storing and managing code, not
data .
D. Use Amazon Kinesis Data Streams to access and integrate third-party datasets
from Amazon Elastic Container Registry (Amazon ECR). This option is also not
feasible, as Amazon ECR is a fully managed container registry service that stores,
manages, and deploys container images, not a data source that can be accessed
by Amazon Kinesis Data Streams. Amazon Kinesis Data Streams does not
support accessing and integrating data from Amazon ECR, which is meant for
storing and managing container images, not data .
References: 1: AWS Data Exchange User Guide
2: AWS Data Exchange FAQs
3: AWS Glue Developer Guide
: AWS CodeCommit User Guide
: Amazon Kinesis Data Streams Developer Guide
: Amazon Elastic Container Registry User Guide
: Build a Continuous Delivery Pipeline for Your Container Images with Amazon
ECR as Source
Question # 25
A company uses an on-premises Microsoft SQL Server database to store financialtransaction data. The company migrates the transaction data from the on-premisesdatabase to AWS at the end of each month. The company has noticed that the cost tomigrate data from the on-premises database to an Amazon RDS for SQL Server databasehas increased recently.The company requires a cost-effective solution to migrate the data to AWS. The solutionmust cause minimal downtown for the applications that access the database.Which AWS service should the company use to meet these requirements?
A. AWS Lambda B. AWS Database Migration Service (AWS DMS) C. AWS Direct Connect D. AWS DataSync
Answer: B
Explanation: AWS Database Migration Service (AWS DMS) is a cloud service that makes
it possible to migrate relational databases, data warehouses, NoSQL databases, and other
types of data stores to AWS quickly, securely, and with minimal downtime and zero data loss1. AWS DMS supports migration between 20-plus database and analytics engines,
such as Microsoft SQL Server to Amazon RDS for SQL Server2. AWS DMS takes
overmany of the difficult or tedious tasks involved in a migration project, such as capacity
analysis, hardware and software procurement, installation and administration, testing and
debugging, and ongoing replication and monitoring1. AWS DMS is a cost-effective solution,
as you only pay for the compute resources and additional log storage used during the
migration process2. AWS DMS is the best solution for the company to migrate the financial
transaction data from the on-premises Microsoft SQL Server database to AWS, as it meets
the requirements of minimal downtime, zero data loss, and low cost.
Option A is not the best solution, as AWS Lambda is a serverless compute service that lets
you run code without provisioning or managing servers, but it does not provide any built-in
features for database migration. You would have to write your own code to extract,
transform, and load the data from the source to the target, which would increase the
operational overhead and complexity.
Option C is not the best solution, as AWS Direct Connect is a service that establishes a
dedicated network connection from your premises to AWS, but it does not provide any
built-in features for database migration. You would still need to use another service or tool
to perform the actual data transfer, which would increase the cost and complexity.
Option D is not the best solution, as AWS DataSync is a service that makes it easy to
transfer data between on-premises storage systems and AWS storage services, such as
Amazon S3, Amazon EFS, and Amazon FSx for Windows File Server, but it does not
support Amazon RDS for SQL Server as a target. You would have to use another service
or tool to migrate the data from Amazon S3 to Amazon RDS for SQL Server, which would
increase the latency and complexity. References:
Database Migration - AWS Database Migration Service - AWS
What is AWS Database Migration Service?
AWS Database Migration Service Documentation
AWS Certified Data Engineer - Associate DEA-C01 Complete Study Guide
Question # 26
A company has used an Amazon Redshift table that is named Orders for 6 months. Thecompany performs weekly updates and deletes on the table. The table has an interleavedsort key on a column that contains AWS Regions.The company wants to reclaim disk space so that the company will not run out of storagespace. The company also wants to analyze the sort key column.Which Amazon Redshift command will meet these requirements?
A. VACUUM FULL Orders B. VACUUM DELETE ONLY Orders C. VACUUM REINDEX Orders D. VACUUM SORT ONLY Orders
Answer: C
Explanation:
Amazon Redshift is a fully managed, petabyte-scale data warehouse service that enables
fast and cost-effective analysis of large volumes of data. Amazon Redshift uses columnar
storage, compression, and zone maps to optimize the storage and performance of data.
However, over time, as data is inserted, updated, or deleted, the physical storage of data
can become fragmented, resulting in wasted disk space and degraded query
performance. To address this issue, Amazon Redshift provides the VACUUM command,
which reclaims disk space and resorts rows in either a specified table or all tables in the
current schema1.
The VACUUM command has four options: FULL, DELETE ONLY, SORT ONLY, and
REINDEX. The option that best meets the requirements of the question is VACUUM
REINDEX, which re-sorts the rows in a table that has an interleaved sort key and
rewritesthe table to a new location on disk. An interleaved sort key is a type of sort key that
gives equal weight to each column in the sort key, and stores the rows in a way that
optimizes the performance of queries that filter by multiple columns in the sort key.
However, as data is added or changed, the interleaved sort order can become skewed,
resulting in suboptimal query performance. The VACUUM REINDEX option restores the
optimal interleaved sort order and reclaims disk space by removing deleted rows. This
option also analyzes the sort key column and updates the table statistics, which are used
by the query optimizer to generate the most efficient query execution plan23. The other options are not optimal for the following reasons:
A. VACUUM FULL Orders. This option reclaims disk space by removing deleted
rows and resorts the entire table. However, this option is not suitable for tables that
have an interleaved sort key, as it does not restore the optimal interleaved sort
order. Moreover, this option is the most resource-intensive and time-consuming,
as it rewrites the entire table to a new location on disk.
B. VACUUM DELETE ONLY Orders. This option reclaims disk space by removing
deleted rows, but does not resort the table. This option is not suitable for tables
that have any sort key, as it does not improve the query performance by restoring
the sort order. Moreover, this option does not analyze the sort key column and
update the table statistics.
D. VACUUM SORT ONLY Orders. This option resorts the entire table, but does
not reclaim disk space by removing deleted rows. This option is not suitable for
tables that have an interleaved sort key, as it does not restore the optimal
interleaved sort order. Moreover, this option does not analyze the sort key column
and update the table statistics.
References:
1: Amazon Redshift VACUUM
2: Amazon Redshift Interleaved Sorting
3: Amazon Redshift ANALYZE
Question # 27
A company extracts approximately 1 TB of data every day from data sources such as SAPHANA, Microsoft SQL Server, MongoDB, Apache Kafka, and Amazon DynamoDB. Someof the data sources have undefined data schemas or data schemas that change.A data engineer must implement a solution that can detect the schema for these datasources. The solution must extract, transform, and load the data to an Amazon S3 bucket.The company has a service level agreement (SLA) to load the data into the S3 bucketwithin 15 minutes of data creation.Which solution will meet these requirements with the LEAST operational overhead?
A. Use Amazon EMR to detect the schema and to extract, transform, and load the data intothe S3 bucket. Create a pipeline in Apache Spark. B. Use AWS Glue to detect the schema and to extract, transform, and load the data intothe S3 bucket. Create a pipeline in Apache Spark. C. Create a PvSpark proqram in AWS Lambda to extract, transform, and load the data intothe S3 bucket. D. Create a stored procedure in Amazon Redshift to detect the schema and to extract,transform, and load the data into a Redshift Spectrum table. Access the table from AmazonS3.
Answer: B
Explanation:
AWS Glue is a fully managed service that provides a serverless data integration platform.
It can automatically discover and categorize data from various sources, including SAP
HANA, Microsoft SQL Server, MongoDB, Apache Kafka, and Amazon DynamoDB. It can
also infer the schema of the data and store it in the AWS Glue Data Catalog, which is a
central metadata repository. AWS Glue can then use the schema information to generate
and run Apache Spark code to extract, transform, and load the data into an Amazon S3
bucket. AWS Glue can also monitor and optimize the performance and cost of the data
pipeline, and handle any schema changes that may occur in the source data. AWS Glue
can meet the SLA of loading the data into the S3 bucket within 15 minutes of data creation,
as it can trigger the data pipeline based on events, schedules, or on-demand. AWS Glue
has the least operational overhead among the options, as it does not require provisioning,
configuring, or managing any servers or clusters. It also handles scaling, patching, and
security automatically. References:
AWS Glue
[AWS Glue Data Catalog]
[AWS Glue Developer Guide]
AWS Certified Data Engineer - Associate DEA-C01 Complete Study Guide
Question # 28
A company stores data from an application in an Amazon DynamoDB table that operates inprovisioned capacity mode. The workloads of the application have predictable throughputload on a regular schedule. Every Monday, there is an immediate increase in activity earlyin the morning. The application has very low usage during weekends.The company must ensure that the application performs consistently during peak usagetimes. Which solution will meet these requirements in the MOST cost-effective way?
A. Increase the provisioned capacity to the maximum capacity that is currently presentduring peak load times. B. Divide the table into two tables. Provision each table with half of the provisionedcapacity of the original table. Spread queries evenly across both tables. C. Use AWS Application Auto Scaling to schedule higher provisioned capacity for peakusage times. Schedule lower capacity during off-peak times. D. Change the capacity mode from provisioned to on-demand. Configure the table to scaleup and scale down based on the load on the table.
Answer: C
Explanation: Amazon DynamoDB is a fully managed NoSQL database service that
provides fast and predictable performance with seamless scalability. DynamoDB offers two
capacity modes for throughput capacity: provisioned and on-demand. In provisioned
capacity mode, you specify the number of read and write capacity units per second that
you expect your application to require. DynamoDB reserves the resources to meet your
throughput needs with consistent performance. In on-demand capacity mode, you pay per
request and DynamoDB scales the resources up and down automatically based on the
actual workload. On-demand capacity mode is suitable for unpredictable workloads that
can vary significantly over time1.
The solution that meets the requirements in the most cost-effective way is to use AWS
Application Auto Scaling to schedule higher provisioned capacity for peak usage times and
lower capacity during off-peak times. This solution has the following advantages:
It allows you to optimize the cost and performance of your DynamoDB table by
adjusting the provisioned capacity according to your predictable workload patterns.
You can use scheduled scaling to specify the date and time for the scaling actions,
and the new minimum and maximum capacity limits. For example, you can
schedule higher capacity for every Monday morning and lower capacity for
weekends2.
It enables you to take advantage of the lower cost per unit of provisioned capacity
mode compared to on-demand capacity mode. Provisioned capacity mode
charges a flat hourly rate for the capacity you reserve, regardless of how much you
use. On-demand capacity mode charges for each read and write request you
consume, with nominimum capacity required. For predictable workloads,
provisioned capacity mode can be more cost-effective than on-demand capacity
mode1.
It ensures that your application performs consistently during peak usage times by
having enough capacity to handle the increased load. You can also use auto
scaling to automatically adjust the provisioned capacity based on the actual
utilization of your table, and set a target utilization percentage for your table or
global secondary index. This way, you can avoid under-provisioning or overprovisioning
your table2.
Option A is incorrect because it suggests increasing the provisioned capacity to the
maximum capacity that is currently present during peak load times. This solution has the
following disadvantages: It wastes money by paying for unused capacity during off-peak times. If you
provision the same high capacity for all times, regardless of the actual workload,
you are over-provisioning your table and paying for resources that you don’t
need1.
It does not account for possible changes in the workload patterns over time. If your
peak load times increase or decrease in the future, you may need to manually
adjust the provisioned capacity to match the new demand. This adds operational
overhead and complexity to your application2.
Option B is incorrect because it suggests dividing the table into two tables and provisioning
each table with half of the provisioned capacity of the original table. This solution has the
following disadvantages:
It complicates the data model and the application logic by splitting the data into two
separate tables. You need to ensure that the queries are evenly distributed across
both tables, and that the data is consistent and synchronized between them. This
adds extra development and maintenance effort to your application3.
It does not solve the problem of adjusting the provisioned capacity according to the
workload patterns. You still need to manually or automatically scale the capacity of
each table based on the actual utilization and demand. This may result in underprovisioning
or over-provisioning your tables2.
Option D is incorrect because it suggests changing the capacity mode from provisioned to
on-demand. This solution has the following disadvantages:
It may incur higher costs than provisioned capacity mode for predictable
workloads. On-demand capacity mode charges for each read and write request
you consume, with no minimum capacity required. For predictable workloads,
provisioned capacity mode can be more cost-effective than on-demand capacity
mode, as you can reserve the capacity you need at a lower rate1.
It may not provide consistent performance during peak usage times, as ondemand
capacity mode may take some time to scale up the resources to meet the
sudden increase in demand. On-demand capacity mode uses adaptive capacity to
handle bursts of traffic, but it may not be able to handle very large spikes or
sustained high throughput. In such cases, you may experience throttling or
increased latency.
References:
1: Choosing the right DynamoDB capacity mode - Amazon DynamoDB
2: Managing throughput capacity automatically with DynamoDB auto scaling -
Amazon DynamoDB
3: Best practices for designing and using partition keys effectively - Amazon
DynamoDB
[4]: On-demand mode guidelines - Amazon DynamoDB
[5]: How to optimize Amazon DynamoDB costs - AWS Database Blog
[6]: DynamoDB adaptive capacity: How it works and how it helps - AWS Database
Blog
[7]: Amazon DynamoDB pricing - Amazon Web Services (AWS)
Question # 29
A company is planning to migrate on-premises Apache Hadoop clusters to Amazon EMR.The company also needs to migrate a data catalog into a persistent storage solution.The company currently stores the data catalog in an on-premises Apache Hive metastoreon the Hadoop clusters. The company requires a serverless solution to migrate the datacatalog.Which solution will meet these requirements MOST cost-effectively?
A. Use AWS Database Migration Service (AWS DMS) to migrate the Hive metastore intoAmazon S3. Configure AWS Glue Data Catalog to scan Amazon S3 to produce the datacatalog. B. Configure a Hive metastore in Amazon EMR. Migrate the existing on-premises Hivemetastore into Amazon EMR. Use AWS Glue Data Catalog to store the company's datacatalog as an external data catalog. C. Configure an external Hive metastore in Amazon EMR. Migrate the existing on-premisesHive metastore into Amazon EMR. Use Amazon Aurora MySQL to store the company'sdata catalog. D. Configure a new Hive metastore in Amazon EMR. Migrate the existing on-premises Hivemetastore into Amazon EMR. Use the new metastore as the company's data catalog.
Answer: A
Explanation: AWS Database Migration Service (AWS DMS) is a service that helps you
migrate databases to AWS quickly and securely. You can use AWS DMS to migrate the
Hive metastore from the on-premises Hadoop clusters into Amazon S3, which is a
highlyscalable, durable, and cost-effective object storage service. AWS Glue Data Catalog
is a serverless, managed service that acts as a central metadata repository for your data
assets. You can use AWS Glue Data Catalog to scan the Amazon S3 bucket that contains
the migrated Hive metastore and create a data catalog that is compatible with Apache Hive
and other AWS services. This solution meets the requirements of migrating the data
catalog into a persistent storage solution and using a serverless solution. This solution is
also the most cost-effective, as it does not incur any additional charges for running Amazon
EMR or Amazon Aurora MySQL clusters. The other options are either not feasible or not
optimal. Configuring a Hive metastore in Amazon EMR (option B) or an external Hive
metastore in Amazon EMR (option C) would require running and maintaining Amazon EMR
clusters, which would incur additional costs and complexity. Using Amazon Aurora MySQL
to store the company’s data catalog (option C) would also incur additional costs and
complexity, as well as introduce compatibility issues with Apache Hive. Configuring a new
Hive metastore in Amazon EMR (option D) would not migrate the existing data catalog, but
create a new one, which would result in data loss and inconsistency. References:
Using AWS Database Migration Service
Populating the AWS Glue Data Catalog
AWS Certified Data Engineer - Associate DEA-C01 Complete Study Guide,
Chapter 4: Data Analysis and Visualization, Section 4.2: AWS Glue Data Catalog
