Cross-Account S3 Access for Shared Data: Save Storage Costs by Avoiding Replication

When managing multiple AWS accounts, one of the biggest pain points that often arises is how to handle shared data efficiently. The simple solution might seem like copying the data between accounts, but that can lead to massive storage costs, duplicated data management headaches, and a lot of inefficiencies. Fortunately, cross-account S3 bucket access policies can swoop in and save the day, eliminating the need for data replication and drastically reducing your storage costs. Here’s how to do it, why it’s a game changer, and some practical examples of how much you could save.

Why Duplicating Data is a Problem

Let’s start with the basics. Say you’ve got multiple AWS accounts under your organization. Maybe you’re segmenting based on different teams, projects, or clients. It’s easy to end up in a situation where everyone needs access to the same data—logs, media files, backups, whatever it might be.

The quick fix? Duplicate the data across the accounts. But here’s the issue with that:

1. Storage Costs Double (or Triple): If Account A, B, and C all store the same 10 TB dataset, that’s 30 TB of data you’re paying for, instead of 10 TB.
2. Management Overhead: Each time data is updated, it must be replicated to all other accounts, requiring automation, monitoring, and troubleshooting.
3. Inconsistency Risk: The more places the data exists, the greater the chance that one copy gets outdated, resulting in inconsistent data sets across accounts.

And let’s be real—data doesn’t stay small. It grows… and fast.

The Solution: Cross-Account S3 Bucket Access

AWS allows cross-account S3 access through bucket policies and IAM roles, enabling users in one AWS account to directly access data in another account’s S3 bucket. With proper permission controls, each account can read or write to a shared bucket, eliminating the need for duplicating that data across multiple buckets.

No more storing the same dataset across three accounts—now you store it in one S3 bucket and let the others access it securely.

How to Set Up Cross-Account S3 Access

Setting up cross-account access sounds intimidating, but AWS has made it pretty simple by using bucket policies and IAM roles. Here’s a basic step-by-step on how you can configure it safely and efficiently, including limiting access to specific folders (prefixes) for each account.

1. Create an S3 Bucket in Account A (the account that owns the data)

In Account A (the central data owner), create an S3 bucket to hold the shared data.

2. Update the Bucket Policy to Grant Access to a Specific IAM Role (Not Root)

You need to grant cross-account access by specifying the IAM roles from the other accounts in the bucket’s policy. For example, instead of giving access to the root user (a security risk), specify the IAM role of the other account.

Here’s an example bucket policy that grants read-only access to an IAM role from Account B:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::123456789012:role/S3AccessRole"  // Account B's IAM role
      },
      "Action": "s3:GetObject",
      "Resource": "arn:aws:s3:::your-bucket-name/*"
    }
  ]
}

• The Principal defines which IAM role (from Account B) is granted access.
• The Action defines what the role can do, in this case, it’s allowed to GetObject (read) files.
• The Resource specifies the S3 bucket and allows access to all objects in it.

3. Set Up IAM Roles in Account B

In Account B, you’ll need to create an IAM role that allows services or users to assume the role to access Account A’s S3 bucket.

Here’s an example policy for that role:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "s3:GetObject",
      "Resource": "arn:aws:s3:::your-bucket-name/*"
    }
  ]
}

This role should be assumed by trusted entities in Account B to access Account A’s bucket.

4. Restrict Access to Specific S3 Prefixes for Each Role

If you want to limit access to specific prefixes (like folders) for each role or account, you can further refine the bucket policy. For example, let’s say you want Account B to only access files in the /team-b/ prefix within the bucket:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::123456789012:role/S3AccessRole"  // Account B's IAM role
      },
      "Action": "s3:GetObject",
      "Resource": "arn:aws:s3:::your-bucket-name/team-b/*"  // Restrict to team-b prefix
    }
  ]
}

This way, Account B can only access the data stored under /team-b/, while other teams or accounts might have access to their own prefixes (like /team-a/ for Account A).

5. Assuming the Role in Account B

In Account B, you’ll need to configure the IAM role to assume access to the S3 bucket in Account A. The trust relationship in Account B would look like this:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "sts:AssumeRole",
      "Principal": {
        "AWS": "arn:aws:iam::111122223333:role/AccountA-S3Access"  // Account A's trusted role
      }
    }
  ]
}

This trust policy allows users or services in Account B to assume the IAM role from Account A and access the S3 bucket.

---

Additional relevant savings opportunity: Optimize your AWS networking costs for S3 and other services

---

Real-World Cost Savings Example

Let’s say you’re running a media production company that uses AWS for video rendering. You’ve got different teams across different AWS accounts—each needing access to the same 50 TB of raw footage stored in S3.

Here’s a quick breakdown of the costs if each account stores its own copy:

• 50 TB of storage in each account.
• S3 Standard Storage Cost: ~$0.023 per GB.
• Multiply by three accounts:50 TB/account x $0.023/GB = $1,150/month/account.
• For three accounts, that’s $1,150 x 3 = $3,450 per month in storage costs alone.

Now, with cross-account access:

• You only store the data in one account, reducing your storage costs to $1,150/month.
• Savings = $2,300/month or $27,600/year.

That’s a huge amount of savings for just setting up a few permissions.

Beyond Storage: Improving Efficiency and Security

It’s not just about cutting storage costs; you’re also improving data management and security. By centralizing data storage:

1. Data Consistency: Every account accesses the same source of truth. You don’t need to worry about outdated copies or discrepancies between data sets.
2. Simplified Data Management: Any updates to the data (e.g., new files, changes) only need to happen in one place, saving you the trouble of replicating changes across multiple buckets.
3. Tighter Security Controls: With cross-account S3 access, you can maintain stricter control over who can access what. Each account can be limited to specific permissions (e.g., read-only, write access) based on roles, preventing accidental data modifications.

Example Use Cases for Cross-Account S3 Access

1. Centralized Logging Across Multiple AWS Accounts

Imagine you’ve got a complex infrastructure spanning multiple AWS accounts, each running different microservices. Centralizing all logs in a single S3 bucket enables all accounts to push logs to one place, simplifying your logging pipeline and reducing duplication.

2. Cross-Team Collaboration in Research or Analytics

You’ve got teams working on different research projects or data analytics, each needing access to the same datasets. Instead of replicating those datasets in every account, teams can access shared S3 buckets, ensuring everyone works with the same data while cutting costs.

3. SaaS Providers Offering Shared Resources

If you’re a SaaS provider delivering services to multiple clients, each with their own AWS accounts, you can provide shared resources (like configuration files or templates) through S3. This setup minimizes data replication across client environments, which is particularly valuable for large-scale datasets.

Avoiding Pitfalls: Monitoring and Auditing Cross-Account Access

While cross-account access can reduce costs and improve efficiency, it’s important to monitor and audit access to your S3 buckets carefully. AWS offers services like AWS CloudTrail and AWS Config to help you track access to shared resources and ensure that your policies are applied correctly. You want to avoid accidental misconfigurations that expose sensitive data to accounts or users who shouldn’t have access.

Conclusion: Stop Wasting Storage, Start Using Cross-Account S3 Access

If your organization operates across multiple AWS accounts and you’re duplicating data, you’re likely burning through cash unnecessarily. By implementing cross-account S3 bucket access, you can reduce storage costs, streamline data management, and ensure better security and consistency across your infrastructure. It’s a no-brainer for anyone serious about optimizing their cloud architecture and minimizing operational costs.

The next time you’re tempted to replicate data across accounts, take a step back and ask: Isn’t there a more efficient way? Spoiler alert: yes, there is. And it’s cross-account S3 access.