Fix AWS AccessDenied Error: IAM & EKS Troubleshooting Guide

1. Introduction

An AccessDenied error from AWS means exactly what it says — an IAM principal (a user, role, or service account) attempted an API call it doesn't have permission to make. On EKS, these errors show up in pod logs, CI/CD pipelines, the AWS CLI, and CloudTrail. They can be frustrating to debug because the error message often omits critical context about which entity is being denied and why.

This guide covers every common AccessDenied scenario on EKS and AWS: missing IAM policy actions, IRSA misconfiguration, resource-based policy conflicts, cross-account access, and the often-overlooked Service Control Policy layer. Each section includes the exact commands to diagnose the issue and the minimum policy needed to fix it.

2. What AccessDenied Actually Means

AWS returns an AccessDenied (or UnauthorizedException for some services) when the IAM policy evaluation chain results in an implicit or explicit deny. The evaluation order matters:

• Explicit deny in any policy always wins — even if another policy allows the action
• Service Control Policies (SCPs) in AWS Organizations can silently block actions even when IAM allows them
• Resource-based policies (S3 bucket policies, ECR repository policies) can deny access independently of IAM identity policies
• Permission boundaries limit the maximum permissions a role can have, even if the role's policy grants more

3. Common Causes

• The IAM role or user is missing the required action in its attached policies
• IRSA is not configured correctly — the pod is using the node's instance profile instead of the intended role
• The action is resource-scoped but the policy uses an incorrect or overly restrictive resource ARN
• A Service Control Policy in the AWS Organization denies the action at the account or OU level
• A permission boundary on the IAM role limits the effective permissions
• Cross-account access: the resource's policy doesn't allow the caller's account
• The ecr:GetAuthorizationToken action is scoped to a resource ARN instead of "*"
• The trust policy on the IAM role has incorrect conditions (wrong namespace or service account name in the OIDC condition)

4. Step-by-Step Diagnosis and Fix

Step 1: Extract the exact error message and identity

# From pod logs:
kubectl logs <pod-name> -n <namespace> | grep -i "AccessDenied\|UnauthorizedException\|not authorized"

# The full error will look like:
# An error occurred (AccessDenied) when calling the GetObject operation:
# User: arn:aws:sts::123456789012:assumed-role/my-role/session-name
# is not authorized to perform: s3:GetObject on resource: arn:aws:s3:::my-bucket/key

# Key fields to extract:
# 1. The calling identity (User: arn:aws:sts::...)
# 2. The action being denied (s3:GetObject)
# 3. The resource ARN (arn:aws:s3:::my-bucket/key)

# Confirm the active identity from within the pod:
kubectl exec -it <pod-name> -n <namespace> --   aws sts get-caller-identity

Step 2: Check if IRSA credentials are being used correctly

# If the pod should be using IRSA but sts get-caller-identity
# returns the node's instance profile, IRSA is not configured correctly.

# Check service account annotation:
kubectl get sa <service-account-name> -n <namespace>   -o jsonpath='{.metadata.annotations}'
# Expected: {"eks.amazonaws.com/role-arn":"arn:aws:iam::123456789012:role/my-role"}

# Check the pod is using the annotated service account:
kubectl get pod <pod-name> -n <namespace>   -o jsonpath='{.spec.serviceAccountName}'

# Check IRSA env vars are injected:
kubectl exec -it <pod-name> -n <namespace> -- env | grep AWS_ROLE_ARN
# Expected: AWS_ROLE_ARN=arn:aws:iam::123456789012:role/my-role

# If missing, restart the pod (webhook re-injects on pod creation):
kubectl rollout restart deployment/<deployment-name> -n <namespace>

Step 3: Simulate the permission using IAM policy simulator

# Simulate whether the role can perform the action:
aws iam simulate-principal-policy   --policy-source-arn arn:aws:iam::123456789012:role/my-role   --action-names s3:GetObject   --resource-arns arn:aws:s3:::my-bucket/key

# Response will show:
# "EvalDecision": "allowed"  → the policy allows it (check resource policy)
# "EvalDecision": "implicitDeny" → no allow policy found
# "EvalDecision": "explicitDeny" → a deny policy exists

# Check for SCPs blocking the action:
aws iam simulate-principal-policy   --policy-source-arn arn:aws:iam::123456789012:role/my-role   --action-names s3:GetObject   --resource-arns arn:aws:s3:::my-bucket/key   --context-entries "ContextKeyName=aws:PrincipalAccount,ContextKeyValues=123456789012,ContextKeyType=string"

Step 4: Fix — Add the missing IAM action

# Minimum policy to fix S3 read access (example):
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:ListBucket"
      ],
      "Resource": [
        "arn:aws:s3:::my-bucket",
        "arn:aws:s3:::my-bucket/*"
      ]
    }
  ]
}

# Apply via AWS CLI:
aws iam put-role-policy   --role-name my-role   --policy-name s3-read-access   --policy-document file://policy.json

# Or attach an existing managed policy:
aws iam attach-role-policy   --role-name my-role   --policy-arn arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess

Step 5: Fix — IRSA trust policy has wrong conditions

# Get the current trust policy:
aws iam get-role --role-name my-role   --query 'Role.AssumeRolePolicyDocument' --output json

# Common mistake: namespace or service account name typo in the :sub condition
# Correct condition looks like:
# "oidc.eks.us-east-1.amazonaws.com/id/EXAMPLED539D:sub":
#   "system:serviceaccount:my-namespace:my-service-account"

# Update the trust policy (replace with your OIDC ID, region, namespace, SA):
aws iam update-assume-role-policy   --role-name my-role   --policy-document '{
    "Version": "2012-10-17",
    "Statement": [{
      "Effect": "Allow",
      "Principal": {
        "Federated": "arn:aws:iam::123456789012:oidc-provider/oidc.eks.us-east-1.amazonaws.com/id/EXAMPLED539D"
      },
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "oidc.eks.us-east-1.amazonaws.com/id/EXAMPLED539D:sub": "system:serviceaccount:my-namespace:my-service-account",
          "oidc.eks.us-east-1.amazonaws.com/id/EXAMPLED539D:aud": "sts.amazonaws.com"
        }
      }
    }]
  }'

Step 6: Fix — ECR GetAuthorizationToken scoped to resource

This is a very common mistake. ecr:GetAuthorizationToken must be granted with Resource: "*" — scoping it to a specific repository ARN causes AccessDenied even when the IAM policy looks correct.

# Wrong — this will always fail:
{
  "Effect": "Allow",
  "Action": "ecr:GetAuthorizationToken",
  "Resource": "arn:aws:ecr:us-east-1:123456789012:repository/my-app"
}

# Correct:
{
  "Effect": "Allow",
  "Action": "ecr:GetAuthorizationToken",
  "Resource": "*"
}

5. Verification Steps

# Verify from inside the pod after fixing:
kubectl exec -it <pod-name> -n <namespace> --   aws sts get-caller-identity
# Confirm the ARN is the intended IRSA role, not the node profile

# Test the specific action that was failing:
kubectl exec -it <pod-name> -n <namespace> --   aws s3 ls s3://my-bucket/ --region us-east-1

# Verify using the policy simulator:
aws iam simulate-principal-policy   --policy-source-arn arn:aws:iam::123456789012:role/my-role   --action-names s3:GetObject   --resource-arns arn:aws:s3:::my-bucket/key
# Expected: "EvalDecision": "allowed"

6. Common Mistakes

• Scoping ecr:GetAuthorizationToken to a repository ARN — it must use "Resource": "*"
• Adding permissions to the node's instance profile when the pod uses IRSA — the instance profile is irrelevant once IRSA is active
• Fixing the identity policy but ignoring a resource-based policy (S3 bucket policy, ECR repository policy) that has an explicit deny
• Not restarting the pod after fixing IRSA — the IRSA token is injected at pod creation time
• Typo in the trust policy :sub condition — namespace or service account name case mismatch causes the token to be rejected
• Ignoring SCPs — in multi-account AWS Organizations, an SCP at the management account level can deny actions regardless of IAM role policies

7. Prevention Tips

• Use IRSA for all pod-level AWS access — never use static IAM credentials in environment variables or Kubernetes secrets
• Use aws iam simulate-principal-policy to validate new policies before deploying — catches implicit denies before they reach production
• Tag IAM roles with the EKS cluster, namespace, and service account they serve — makes audit and debugging much faster
• Use IAM Access Analyzer to detect overly permissive policies and identify unused permissions
• For ECR access, always structure the policy with GetAuthorizationToken using Resource: "*" and the repository-level actions scoped to the specific repo ARN
• Enable CloudTrail in all accounts and configure CloudWatch alerts for AccessDenied events on critical resources

8. FAQ

9. Summary

AccessDenied errors on EKS almost always fall into one of five categories: missing IAM action, wrong IRSA configuration, resource-based policy conflict, SCPs, or the classic ecr:GetAuthorizationToken resource scoping mistake. Start by running aws sts get-caller-identity from inside the pod to confirm which identity is making the call, then use the policy simulator to understand exactly why it's being denied.