Tagging and Snapshotting in AWS with Lambda

01.16.2016 — Development, AWS, Lambda, Python, Automation — 1 min read — Mike Lapidakis

Update [February 10, 2021]: The Github community has actively contributed to the below code snippets over the past five years. I'd recommend reading the comments on the Gist page before proceeding with this post. Thank you, Github community!

One of the more challenging aspects to managing a large AWS infrastructure can be tag management for cost allocation and tracking. When you create an EC2 instance, several other assets are created with it, some of which generate charges that should be tracked over time. While keeping an instance's tags updated is fairly straightforward, ensuring that its EBS volumes, elastic IPs, elastic network interfaces and snapshots stay tagged appropriately can be a real headache.

In my quest to streamline the operations of my clients' AWS infrastructure using Lambda, I've created a Lambda Function that will write and update specific tags from an EC2 instance to that instance's attached volumes and network interfaces. I have this function's event set to trigger every hour to ensure the tags stay up to date.

Active Gist

1from __future__ import print_function
2
3import json
4import boto3
5import logging
6
7#setup simple logging for INFO
8logger = logging.getLogger()
9logger.setLevel(logging.ERROR)
10
11#define the connection region
12ec2 = boto3.resource('ec2', region_name="us-west-2")
13
14#Set this to True if you don't want the function to perform any actions
15debugMode = False
16
17def lambda_handler(event, context):
18    #List all EC2 instances
19    base = ec2.instances.all()
20
21    #loop through by running instances
22    for instance in base:
23        
24        #Tag the Volumes
25        for vol in instance.volumes.all():
26            #print(vol.attachments[0]['Device'])
27            if debugMode == True:
28                print("[DEBUG] " + str(vol))
29                tag_cleanup(instance, vol.attachments[0]['Device'])
30            else:
31                tag = vol.create_tags(Tags=tag_cleanup(instance, vol.attachments[0]['Device']))
32                print("[INFO]: " + str(tag))
33        
34        #Tag the Network Interfaces
35        for eni in instance.network_interfaces:
36            #print(eni.attachment['DeviceIndex'])
37            if debugMode == True:
38                print("[DEBUG] " + str(eni))
39                tag_cleanup(instance, "eth"+str(eni.attachment['DeviceIndex']))
40            else:
41                tag = eni.create_tags(Tags=tag_cleanup(instance, "eth"+str(eni.attachment['DeviceIndex'])))
42                print("[INFO]: " + str(tag))
43    
44#------------- Functions ------------------
45#returns the type of configuration that was performed    
46        
47def tag_cleanup(instance, detail):
48    tempTags=[]
49    v={}
50    
51    for t in instance.tags:
52        #pull the name tag
53        if t['Key'] == 'Name':
54            v['Value'] = t['Value'] + " - " + str(detail)
55            v['Key'] = 'Name'
56            tempTags.append(v)
57        #Set the important tags that should be written here  
58        elif t['Key'] == 'Application Owner':
59            print("[INFO]: Application Owner Tag " + str(t))
60            tempTags.append(t)    
61        elif t['Key'] == 'Cost Center':
62            print("[INFO]: Cost Center Tag " + str(t))
63            tempTags.append(t)  
64        elif t['Key'] == 'Date Created':
65            print("[INFO]: Date Created Tag " + str(t))
66            tempTags.append(t)  
67        elif t['Key'] == 'Requestor':
68            print("[INFO]: Requestor Tag " + str(t))
69            tempTags.append(t)  
70        elif t['Key'] == 'System Owner':
71            print("[INFO]: System Owner Tag " + str(t))
72            tempTags.append(t)
73        else:
74            print("[INFO]: Skip Tag - " + str(t))
75    
76    print("[INFO] " + str(tempTags))
77    return(tempTags)

Using the tagging Lambda function in conjunction with a snapshotting function that copies a volume's tags to all newly created snapshots will ensure your billing reports and charge backs capture all charges associated with running that instance in AWS, nearly automatically.

The following snapshot script also cleans up old snapshot (you can set the offset on line 15). I normally set this function's event to trigger once a day, during a low transactional point. I also recommend setting the timeout on this function to five minutes, as the cleanup process can to take a very long time depending on the number of snapshots you keep and the number of volumes you're snapshotting.

Active Gist

1import boto3
2import logging
3import datetime
4import re
5import time
6
7#setup simple logging for INFO
8logger = logging.getLogger()
9logger.setLevel(logging.ERROR)
10
11#define the connection
12ec2 = boto3.resource('ec2', region_name="us-west-2")
13
14#set the snapshot removal offset
15cleanDate = datetime.datetime.now()-datetime.timedelta(days=5)
16
17#Set this to True if you don't want the function to perform any actions
18debugMode = False
19
20def lambda_handler(event, context):
21    
22    if debugMode == True:
23        print("-------DEBUG MODE----------")
24 
25    #snapshop the instances
26    for vol in ec2.volumes.all():
27        tempTags=[]
28        
29        #Prepare Volume tags to be importated into the snapshot
30        if vol.tags != None:
31            for t in vol.tags:
32                
33                #pull the name tag
34                if t['Key'] == 'Name':
35                    instanceName =  t['Value']
36                    tempTags.append(t)
37                else:
38                    tempTags.append(t)
39        else:
40            print("Issue retriving tag")
41            instanceName = "NoName"
42            t['Key'] = 'Name'
43            t['Value'] = 'Missing'
44            tempTags.append(t)
45        
46        description = str(datetime.datetime.now()) + "-" + instanceName + "-" + vol.id + "-automated"
47        
48        if debugMode != True:
49            #snapshot that server
50            snapshot = ec2.create_snapshot(VolumeId=vol.id, Description=description)
51            
52            #write the tags to the snapshot
53            tags = snapshot.create_tags(
54                    Tags=tempTags
55                )
56            print("[LOG] " + str(snapshot))
57            
58        else:
59            print("[DEBUG] " + str(tempTags))
60    
61    print "[LOG] Cleaning out old entries starting on " + str(cleanDate)
62    
63    #clean up old snapshots
64    for snap in ec2.snapshots.all():
65
66        #veryify results have a value
67        if snap.description.endswith("-automated"): 
68            
69            #Pull the snapshot date
70            snapDate = snap.start_time.replace(tzinfo=None)
71            if debugMode == True:
72                print("[DEBUG] " + str(snapDate) +" vs " + str(cleanDate)) 
73            
74            #Compare the clean dates
75            if cleanDate > snapDate:
76                print("[INFO] Deleteing: " + snap.id + " - From: " + str(snapDate))
77                if debugMode != True:
78                    try:
79                        snapshot = snap.delete()
80                    except:
81                        
82                        #if we timeout because of a rate limit being exceeded, give it a rest of a few seconds
83                        print("[INFO]: Waiting 5 Seconds for the API to Chill")
84                        time.sleep(5)
85                        snapshot = snap.delete()
86                    print("[INFO] " + str(snapshot))

Using these two function in conjunction with one another will not only streamline your charge back and tagging models, but also ensure you have consistent snapshots of all of your instances over time. While I don't recommend snapshots as your sole backup method, I do recommend keeping at least one per day to speed up the recovery process if a disaster does occur.

If you're looking for a solid snapshotting and tagging solution, give these Lambda Functions a try. If you have any questions or suggestions, please leave a comment below or contact me on Twitter. I'm always looking for better ways to write and run these functions.