Ensure AWS Elastic MapReduce (EMR) clusters capture detailed log data to Amazon S3


EMR cluster can be configured to periodically archive the log files stored on the master node to Amazon S3. This ensures that the log files are available after the cluster terminates, whether this is through normal shut down or due to an error. Amazon EMR archives the log files to Amazon S3 at 5 minute intervals.


From Console :
1. Sign in to the Amazon EMR console at
2. Choose Create cluster.
3. Choose Go to advanced options.
4. In the General options section, in the Logging field, accept the default option: Enabled.
This determines whether Amazon EMR captures detailed log data to Amazon S3. You can only set this when the cluster is created. For more information, see View Log Files.
5. In the S3 folder field, type (or browse to) an Amazon S3 path to store your logs. You may also allow the console to generate an Amazon S3 path for you. If you type the name of a folder that does not exist in the bucket, it is created.
When this value is set, Amazon EMR copies the log files from the EC2 instances in the cluster to Amazon S3. This prevents the log files from being lost when the cluster ends and the EC2 instances hosting the cluster are terminated. These logs are useful for troubleshooting purposes.
For more information, see View Log Files.
6. In the Log encryption field, select Encrypt logs stored in S3 with an AWS KMS customer managed key. Then select an AWS KMS key from the list or enter a key ARN. You may also create a new KMS key.
This option is only available with Amazon EMR version 5.30.0 and later. To use this option, add permission to KMS for your EC2 instance profile and EMR role. For more information, see To encrypt log files stored in Amazon S3 with an AWS KMS customer managed key.
7. Proceed with creating the cluster as described in Plan and Configure Clusters.

From TF:

resource aws_emr_cluster” “example” {
+ log_uri = “s3n://my-emr-logs/test/example”

Reference :







We are now live on AWS Marketplace.
The integrated view of your cloud infrastructure is now easier than ever!