AWS ECR Cost Optimization: A Tech Insider's Guide to Saving Big in the Cloud

Hey fellow tech enthusiasts! If you’re navigating the intricate world of AWS like I am, you’ve probably realized that while the cloud offers unparalleled flexibility and scalability, it can also become a black hole for your budget if you’re not careful. One service that often slips under the radar when it comes to cost management is AWS Elastic Container Registry (ECR).

As someone who’s spent years in the trenches optimizing cloud infrastructures, I’ve learned that effective cost management isn’t just about cutting expenses—it’s about making smart choices that align with your operational needs. Today, I’m excited to share some insights on how you can optimize your AWS ECR costs without compromising on performance. We’ll explore lifecycle policies, delve into Terraform automation, and sprinkle in some best practices that have saved my team thousands of dollars.

So grab your favorite beverage, settle in, and let’s dive into the nitty-gritty of AWS ECR cost optimization!

Understanding AWS ECR Costs

Before we jump into solutions, it’s crucial to understand where your AWS ECR costs are coming from. AWS ECR uses a pay-as-you-go pricing model, which is fantastic for flexibility but can lead to unexpected charges if not monitored closely.

ECR Storage Costs: You’re billed based on the total size of the images stored in your repositories. Every Docker image, tagged or untagged, contributes to this cost.
Data Transfer Costs: Charges apply when data is transferred out of ECR to the internet or across AWS regions. This includes pulling images to different regions or making them publicly available.

Why It Matters

I recall a time when our team was so focused on deploying features that we overlooked the accumulating number of old Docker images. When the monthly AWS bill arrived, the storage costs for ECR had spiked unexpectedly. It was a wake-up call that even seemingly negligible costs can add up over time, especially with continuous integration and deployment pipelines pushing new images regularly.

Saving ECR Costs with Lifecycle Policies

One of the most effective ways to manage and reduce ECR costs is by implementing Lifecycle Policies. These policies allow you to automate the cleanup of images in your repositories, ensuring that only necessary images are retained.

What Are Lifecycle Policies?

Lifecycle policies in ECR let you define rules to automatically manage the lifecycle of images. You can specify conditions to:

Delete untagged images older than a certain number of days.
Retain only the most recent images based on tags.
Remove images that are no longer needed.

Use Case: Microservices Architecture with Compounding Storage Costs

Context: A company with a microservices architecture has 50 services, each pushing 300 images per month across development, staging, production, and testing environments. Each image averages 400 MB. Without cleanup, storage requirements compound monthly.

Monthly Storage Growth:

Per Service: 300 images * 400 MB = 120 GB
Total for 50 Services: 120 GB * 50 = 6,000 GB (6 TB)

Cost Comparison Over One Year

Without Optimization (Compounding Storage Costs):

Month 1: 6,000 GB = $600
Month 2: 12,000 GB = $1,200
Month 3: 18,000 GB = $1,800
Month 12: 6,000 GB * 12 months = 72,000 GB = $7,200

Total Cost Over 1 Year: $46,800

Costs With Optimization (Lifecycle Policy Cap)

Optimization Strategy: Retain only 10 recent images for development, staging, testing; 50 images for production. Expire older images after 30 days.

Total Storage Cap per Service: 32 GB
Total Monthly Storage (50 Services): 1,600 GB
Monthly Cost (Fixed): 1,600 GB * $0.10 = $160

Total Cost Over 1 Year with Lifecycle Policy: $160 * 12 months = $1,920

Summary of Savings

Without Lifecycle Policy: $46,800/year
With Lifecycle Policy: $1,920/year
Annual Savings: $44,880

By implementing lifecycle policies, the company saves $44,880 annually by preventing compounding storage costs.

Step-by-Step Guide to Setting Up a Lifecycle Policy

Let’s walk through setting up a lifecycle policy using the AWS Management Console:

Access the ECR Console: Log in to your AWS account and navigate to the ECR service.
Select Your Repository: Choose the repository you wish to manage.
Navigate to ECR Lifecycle Policies: Click on the “Lifecycle Policies” tab within your repository.
Create a New Policy: Click “Create lifecycle policy” and start defining your rules.
Define Policy Rules:
- Rule Priority: Assign a priority number (lower numbers have higher priority).
- Description: Provide a meaningful description for future reference.
- Select Images to Delete: Choose criteria such as untagged images or images older than a certain age.
- Retention Count: Specify how many images you want to retain.
Review and Save: Double-check your settings and save the policy.

Example Policy

Rule: Delete untagged images older than 30 days.
Retention: Keep the latest 10 tagged images.

Benefits

ECR Cost Reduction: By deleting unused images, you free up storage space, directly reducing costs.
ECR Repository Management: Keeps your repositories clean and easier to navigate.
Automation: Set it and forget it—policies run automatically without manual intervention.

Automating ECR Cost Savings with Terraform

Managing lifecycle policies manually is feasible for a few repositories, but what if you’re overseeing dozens or even hundreds? That’s where Terraform comes into play.

Why Terraform?

Terraform is an open-source Infrastructure as Code (IaC) tool that allows you to define and provision your infrastructure using code. It offers:

Scalability: Manage multiple repositories and policies effortlessly.
Consistency: Apply uniform policies across all repositories.
Version Control: Track changes and collaborate with your team effectively.

Terraform Configuration Example

Here’s how you can automate lifecycle policies for all your ECR repositories:

# Define a list of repository names to exclude from the lifecycle policy
locals {
  exclude_repos = ["repo-name-1", "repo-name-2"] # Add the repositories you want to exclude here
}

# Data source to fetch all ECR repositories
data "aws_ecr_repositories" "all" {}

# Apply lifecycle policy only to repositories not in the exclusion list
resource "aws_ecr_lifecycle_policy" "ecr_policies" {
  # Filter out excluded repositories by checking if the repository name is not in the exclude list
  for_each = {
    for repo in data.aws_ecr_repositories.all.repositories :
    repo.repository_name => repo
    if !(repo.repository_name in local.exclude_repos)
  }

  repository = each.key

  policy = jsonencode({
    rules = [
      {
        rulePriority = 1,
        description  = "Expire untagged images older than 30 days",
        selection    = {
          tagStatus     = "untagged",
          countType     = "sinceImagePushed",
          countUnit     = "days",
          countNumber   = 30
        },
        action       = {
          type          = "expire"
        }
      },
      {
        rulePriority = 2,
        description  = "Retain only the latest 10 tagged images",
        selection    = {
          tagStatus     = "tagged",
          tagPrefixList = [""],
          countType     = "imageCountMoreThan",
          countNumber   = 10
        },
        action       = {
          type          = "expire"
        }
      }
    ]
  })
}

Breaking Down the Code

Exclusion List: List of repositories by name to exclude from having standardized policies attached.
Data Source: Retrieves all existing ECR repositories in your AWS account.
for_each Loop: Iterates over each repository to apply the lifecycle policy.
Policy Rules:
- Rule 1: Expires untagged images older than 30 days.
- Rule 2: Keeps only the latest 10 tagged images.

Advantages

Standardization: Most if not all repositories have a standard scalable policy.
Efficiency: Automates repetitive tasks, saving time and reducing errors.
Scalability: Easily manage policies across multiple AWS accounts and regions.
Auditing: Keep track of changes through version control systems like Git.

Wrapping Things Up

Optimizing your AWS ECR costs is not a one-time task but an ongoing process that requires attention and proactive management. By leveraging lifecycle policies and automation tools like Terraform, you can significantly reduce unnecessary expenses and keep your repositories clean and efficient.

Remember, the goal isn’t just to cut costs but to optimize them in a way that supports your team’s productivity and your organization’s objectives. Implement the strategies and best practices discussed here, and you’ll be well on your way to mastering AWS ECR cost optimization.

Spread the savings