Why AWS Lambda times out listing S3 buckets in a VPC, and how to fix with a NAT Gateway or S3 VPC endpoint

Troubleshoot AWS Lambda S3 30-second timeouts in a VPC. Learn why missing Internet access causes delays and how to fix with NAT Gateway or an S3 VPC endpoint.

When an AWS Lambda function that should simply list S3 buckets keeps hitting the 30-second timeout, the instinct is to hunt for missing permissions or syntax issues. In reality, this scenario is almost always a networking problem: the code runs, but it cannot reach the S3 API endpoint in time.

Minimal example that times out

The function below uses boto3 to enumerate S3 buckets. It is intentionally simple, yet it can still time out if there is no network path from Lambda to S3.

import boto3
def entrypoint(evt, ctx):
    s3_bucket_id = "<myS3_bucket>"
    object_key = "<path_to/file.csv>"
    s3res = boto3.resource('s3')
    for b in s3res.buckets.all():
        print(b.name)

The surrounding IaC grants Lambda permissions and opens a very permissive security group, which makes the timeout more confusing at first glance.

resource "aws_iam_role" "exec_role" {
  name = "${var.fn_name}_role"
  assume_role_policy = jsonencode({
    Statement = [
      {
        Action = "sts:AssumeRole"
        Effect = "Allow"
        Principal = { Service = "lambda.amazonaws.com" }
      }
    ]
  })
}
resource "aws_iam_role_policy_attachment" "s3_full_access" {
  role       = aws_iam_role.exec_role.name
  policy_arn = "arn:aws:iam::aws:policy/AmazonS3FullAccess"
}
resource "aws_iam_role_policy_attachment" "logs_basic" {
  role       = aws_iam_role.exec_role.name
  policy_arn = "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
}
resource "aws_iam_role_policy_attachment" "vpc_access" {
  role       = aws_iam_role.exec_role.name
  policy_arn = "arn:aws:iam::aws:policy/service-role/AWSLambdaVPCAccessExecutionRole"
}

data "aws_vpc" "default_net" {
  default = true
}
resource "aws_security_group" "fn_sg" {
  name        = "sg_${var.fn_name}"
  description = "Allow all the ports needed for lambda"
  vpc_id      = data.aws_vpc.default_net.id
  ingress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

resource "aws_s3_bucket_policy" "fn_bucket_perms" {
  bucket = "superset-dockerfiles"
  policy = jsonencode({
    Version = "2012-10-17",
    Statement = [
      {
        Effect = "Allow",
        Principal = { AWS = aws_iam_role.exec_role.arn },
        Action = ["s3:GetObject", "s3:ListBucket"],
        Resource = [
          "arn:aws:s3:::superset-dockerfiles",
          "arn:aws:s3:::superset-dockerfiles/*"
        ]
      }
    ]
  })
}

What actually causes the timeout

2025-05-16T14:37:13.093Z fdb6***4165 Task timed out after 30.03 seconds

The failure is not in boto3 or IAM. The function is running inside a VPC and, by default, Lambda functions in a VPC do not have Internet access. The S3 API is on the Internet. Without a network route from the VPC to the public S3 endpoint, requests never complete and the function times out. Adjusting security group rules alone does not fix this because the issue is routing, not port filtering. As was observed, a timeout is a classic symptom of missing connectivity.

How to resolve it

There are three valid paths forward, and the right one depends on whether the function needs to reach private resources inside your VPC. The simplest fix is to run the Lambda function outside of any VPC. When the function is not attached to a VPC, it has Internet access by default and can reach the S3 API directly. If the function must stay in a VPC, configure private subnets with a route to a NAT Gateway so that the function can reach the Internet from within the VPC. Alternatively, add an S3 endpoint to the VPC so the function can reach S3 without traversing the public Internet.

No changes to the Python code are required for this specific problem because the root cause is network reachability. Once a valid route to S3 exists, the same boto3 listing call will succeed.

Why this matters

It is easy to grant permissions and still be blocked by networking. Timeouts waste execution time and conceal the real cause, especially when logs do not show any explicit connection errors. Understanding that VPC-attached Lambda functions do not have Internet access unless explicitly provided helps avoid misdiagnosis and prevents chasing red herrings in IAM or application logic.

Takeaways

If your Lambda function does not need to talk to private resources, run it outside the VPC to keep things simple and reliable. If it does, ensure that you either provide a NAT Gateway with appropriate routing for outbound Internet access or create an S3 endpoint so that S3 calls stay inside the AWS network. With a proper network path in place, straightforward boto3 operations like listing buckets or fetching objects will work as expected.