Lambda function to monitor EC2 snapshots

A snapshot is a point-in-time copy of data. The best thing about snapshot over a normal backup is it is an effortlessness to rollback.

Prerequisites

  • Ec2 instances
  • Snapshots with tags “eg – backup”
  • Creation of IAM role and policy that will allow Lamda to interact with EC2.

Check the below policy for reference
{
“Version”: “2012-10-17”,
“Statement”: [
{
“Effect”: “Allow”,
“Action”: [
“logs:*”
],
“Resource”: “arn:aws:logs:*:*:*”
},
{
“Effect”: “Allow”,
“Action”: “ec2:Describe*”,
“Resource”: “*”
}
]
}

Steps to create IAM role and to attach policy is as below 

  • Go to Services, IAM, Create a new Role
  • Select the option Lambda and not any policy
  • Click Next and Create a Role
  • Enter the role name (Eg:ebs-lambda-worker)
  • Select the new role, and click attach policies
  • Click the option Create Policy
  • Select the option JSON and insert the content of the above snippet
  • Click the button “Review Policy button”
  • Provide a name for the policy and click the “button create policy”
  • Now select the policy that you have created and click the “button policy actions” and select the option to attach

Steps to create a Lambda function

  • Go to Services, Lambda, and click Create a Lambda Function
  • Write a name for it
  • Select Python 2.7 as a Runtime option
  • Select the previously created IAM role
  • Click Create Function
  • Paste the code below in the inline editor

import boto3
from botocore.exceptions import ClientError
from datetime import datetime,timedelta

def lambda_handler(event, context):
filters = [{‘Name’: ‘tag-key’, ‘Values’: [‘backup’]}]
#define retention period(in days)
retention_days = 10
now = datetime.now()
#create EC2 client
ec2 = boto3.client(‘ec2’)
#list of regions
regions = ec2.describe_regions().get(‘Regions’,[] )
numberofsnapshot = 0
old = 0
# search in regions for instances
for region in regions:
print “Checking region %s ” % region[‘RegionName’]
reg=region[‘RegionName’]
ec2 = boto3.client(‘ec2’, region_name=reg)
result = ec2.describe_volumes( Filters=[{‘Name’: ‘status’, ‘Values’: [‘in-use’]}])
for volume in result[‘Volumes’]:#get the volume ID of the instance
result1 = ec2.describe_snapshots(Filters=filters)
for snapshot in result1[‘Snapshots’]:#get the snapshot details and store it in to the variable snapshot
print “Checking snapshot %s which was created on %s” % (snapshot[‘SnapshotId’],snapshot[‘StartTime’])
numberofsnapshot = numberofsnapshot + 1
time = snapshot[‘StartTime’].replace(tzinfo=None)
if (now – time) > timedelta(retention_days):#check if the timedelta is greater than retention days
old = old + 1
print “for volume %s found %s snapshots number of snapshots older than the retention date is %s “% (volume[‘VolumeId’],numberofsnapshot,old)

 

  • Make sure that timeout value is more than 1 minute
  • Please be noted that this lambda function will check for the snapshots which are having a tag “backup”. Hence if you are using any other scripts to generate snapshots please do make sure that the snapshot is having the same tag
  • Now click on the Test to execute the function
  • Once the execution is completed you will be able to see the result just below the lambda function

 

Conclusion

Lambda functions currently support the following languages: Node.js, Java, C#, and Python

AWS offers the run-time platform for Lamda to execute the “serverless” code.  Hence it is extremely profitable in terms of server space and cost.

Integrating Amazon CDN (CloudFront) with CMS

Cloudfront is an Amazon web service that helps us to speed up the distribution of both static and dynamic web content. CloudFront delivers web content through a worldwide network of data centers called edge locations. This web service conjointly offers businesses and web application developers a simple and cost-effective way to distribute content with low latency and high data transfer speeds.

Enabling CloudFront in wordpress

CloudFront speeds up the wordpress website in the following ways 

  • Anycast DNS route the customer to the nearest edge location. 
  • Cached content are delivered to the users from the edge location. 
  • When data needs to be fetched from the site CloudFront optimizes network throughput by managing the transfers between Edge Locations and the website.  This traffic runs over the Amazon Global Backbone, where optimized TCP configuration ensures more bytes are in-flight on the network, improving throughput, while TCP connection re-use eliminates much of the latency associated with establishing connections.   In this way, whether content is cached or not, it will be accelerated by delivery over optimized network paths. 
  • Finally, negotiating and offloading Transport Layer Security (TLS) at the CloudFront Edge further improves performance, reducing connection setup latency, and further supporting back-end connection re-use.

There are two ways we can associate CloudFront with the WordPress site 

Approach 1   

Upload all the wordpress static files (contents in  wp-content and wp-includes folders) onto a Amazon S3 bucket and set the S3 bucket as Cloudfront distribution origin. 

Advantage – if we ever change the WordPress host, the entire wp-content folder is stored off-site at Amazon S3, so you don’t need to even copy the wp-content folder to your new WordPress installation 

Approach 2 

“Origin Pull” – accessing files directly from the website. 

Advantage – This is easier and cheaper, we don’t need to upload the files to Amazon each time you make a new post. 

1) Create a user and user group in IAM 

  • Create a new group (give any name) and give administrative privilege. 
  • Create a user + click on add user + give username + grand programmatic access click on permissions and assign it to the group that we created and then click on create the user. 
  • Copy the access key and secret access key (or download them). 

2) Create an S3 bucket   

  • Create an S3 bucket.  
  • Set the domain name as the S3 bucket name. 
  • Make sure that the IAM user created in the earlier step has sufficient read permission to access the S3 bucket. 

3) Create a CloudFront distribution   

  • Create a Cloudfront web distribution. 
  • Select the S3 bucket as the origin name in the Cloudfront web distribution. 
  • Leave the rest of the options as default. 
  • Create the distribution. 

4) Setting up the WordPress w3-total-cache plugin

  • Log in to the WordPress dashboard and install w3-total cache plugin. 
  • Select “General settings” from W3 Total Cache Performance Menu 

  • Tick the box to enable CDN and select AWS CloudFront as CDN type.
  • Click the Save All Settings Button.
  • Select CDN from the W3 Total Cache Performance Menu

Set the following values
Access Key ID:
Secret Key ID:
Origin: <s3 bucket name(should be same as the domain name)>
Replace site’s Hostname with: <CNAME set in the CloudFront Distribution>

  • Click the “Test CloudFront Distribution” button at the bottom of the configuration section of W3 Total Cache.
  • If you see the message “Test passed”, and then click on the “Save Settings & Purge Caches” button
  • Click on the upload attachment button and click on start
  • This will upload all your WordPress contents to the s3 bucket
  • Same way you can click on the upload attachment buttons of other sections and upload the desired contents to the s3

5) Setting Up CloudFront With WP Super Cache  (Alternative plugin)

    • Log in to the WordPress dashboard and install WP Super Cache

Set the following values.
Off-site URL: <CloudFront Distribution>
Additional CNAMEs:  <CNAME of the CloudFront Distribution>

  • Now the content will be loaded from the AWS CloudFront instead of the old server.
  • (For confirmation, you can check the URL of the images that are been loaded by opening the image in a new tab).

Conclusion

By using CloudFront for the WordPress Websites which are been accessed globally makes the pages to load faster. For all the WordPress users it’s a great suggestion to use CloudFront CDN technology for their websites.   CloudFront can be implemented in various CMS such as Drupal, Magento, etc.