Amazon Web Services - Configuring Amazon S3 Event Notifications

The Amazon S3 notification feature enables you to receive notifications when a certain event occurs inside your bucket. To get notifications, first, add a notification configuration that reads the event you want Amazon S3 to publish and the destinations where Amazon S3 will send the notifications. This configuration is stored in the notification sub-resource that is associated with a bucket. 

Types of Event Notifications:

Currently, Amazon S3 can publish notifications for the following supported events:

1. New object created events — Amazon S3 sends a notification when an object is created. It supports multiple APIs to create objects such as Put, Post, Copy, and Multipart Upload. We can also use a wildcard (s3:ObjectCreated:*) if any of the objects create an event happens.
2. Object removal events — Amazon S3 sends a notification upon deletion of an object. It supports two delete options. One is Permanently Delete and the other is Delete Marker Created. We can also use a wildcard (s3:ObjectRemoved:*) if any of the objects delete event happens.
3. Restore object events — Amazon S3 allows restoration of objects archived to the S3 Glacier storage classes. Your request to notified upon completion of object restoration. It is of two types. The first is Restore Initiated and other is Restore Completed. we can also use a wildcard (s3:ObjectRestore:*) if any of the objects restore events occurs.
4. Reduced Redundancy Storage (RRS) object lost events — Amazon S3 notifies by delivering a message when it detects that an object of the RRS storage class has been lost.
5. Replication events — Amazon S3 sends two event notifications. One, when an object fails replication when an object exceeds the 15-minute threshold, when an object is replicated after the 15-minute threshold, and when an object is no longer tracked by replication metrics. Another when that object replicates to the destination Region. We can also use a wildcard (s3:Replication:*) if any of the object replication events happens.

The following image shows the type of events available in AWS.

Supported Destinations:

1. Amazon Simple Notification Service (Amazon SNS) topic - Amazon SNS is a fully managed, flexible push messaging service. Using this service, you can push messages to mobile devices, emails or distributed services. SNS can publish a message once and can deliver it one or more times.
2. Amazon Simple Queue Service (Amazon SQS) queue - Amazon SQS is a scalable and fully managed message queuing service. SQS can be used to transmit any volume of data without requiring other services to be always available. It is used to decouple services.
3. AWS Lambda - AWS Lambda is a server less compute service that makes it easy for you to build applications that respond quickly to new information. AWS Lambda runs written code in response to events such as image uploads, in-app activity, website clicks, or outputs from connected devices.

The supported destinations have been shown below in the image.

Granting Permissions to publish event notification messages to a Destination

For Amazon S3 to publish event notification messages to a destination, you must grant the Amazon S3 principal the required permissions to call the relevant API to publish messages to an SNS topic, an SQS queue, or a Lambda function.

• Granting permissions to invoke an AWS Lambda function - Amazon S3 invokes a Lambda function and provide the event message as an argument to publish event messages to AWS Lambda. When setting up lambda as the destination to receive event notification messages in the Amazon S3 console, the console sets up the required permissions on the Lambda function so that Amazon S3 bucket has permissions to invoke the function.
• Granting permissions to publish messages to an SNS topic or an SQS queue - To grant Amazon S3 bucket permissions to publish messages to the SNS topic or SQS queue, you attach an AWS Identity and Access Management (IAM) policy to the destination SNS topic or SQS queue. The policy is in JSON format. Examples of SNS topic policy and SQS Policy are given below:

SNS Policy Example:

{
 "Version": "2012-10-17",
 "Id": "SNS Topic Policy",
 "Statement": [
  {
   "Sid": "Geeksforgeeks SNS",
   "Effect": "Allow",
   "Principal": {
     "Service": "s3.amazonaws.com"  
   },
   "Action": [
    "SNS:Publish"
   ],
   "Resource": "arn:aws:sns:Region:amazon-account-id:geeksforgeeksSNS",
   "Condition": {
      "ArnLike": { "aws:SourceArn": "arn:aws:s3:::geeksforgeeks" },
      "StringEquals": { "aws:SourceAccount": "bucket-owner-account-id" }
   }
  }
 ]
}

SQS Policy Example:

{
 "Version": "2012-10-17",
 "Id": "SQS Policy",
 "Statement": [
  {
   "Sid": "Geeksforgeeks SQS",
   "Effect": "Allow",
   "Principal": {
     "Service": "s3.amazonaws.com"  
   },
   "Action": [
    "SQS:SendMessage"
   ],
   "Resource": "arn:aws:sqs:Region:amazon-account-id:geeksforgeeksSQS",
   "Condition": {
      "ArnLike": { "aws:SourceArn": "arn:aws:s3:*:*:geeksforgeeks" },
      "StringEquals": { "aws:SourceAccount": "bucket-owner-account-id" }
   }
  }
 ]
}

Enabling Event Notifications: 

Enabling notifications is a bucket-level operation. You store notification configuration information in the event notification sub resource associated with a bucket.

Event Notifications can be set up by two ways:

1. Amazon S3 Console - You can simply choose the bucket for which you want to receive messages for any kind of activity. Direct to the properties tab of the bucket and there you can set up notifications in the event notifications section.
2. Programmatically using the AWS SDKs - Amazon S3 stores the notification configuration as XML in the notification sub resource associated with a bucket.