Fix AWS Load Balancer Not Routing Traffic

1. Introduction

An AWS load balancer that isn't routing traffic is one of the most urgent production issues you can face. Requests arrive at the load balancer DNS name, but nothing reaches your application. Users see 502 Bad Gateway or 503 Service Unavailable. The load balancer is running — but no traffic gets through.

This guide covers ALB (Application Load Balancer), NLB (Network Load Balancer), and the AWS Load Balancer Controller for EKS. The diagnostic approach varies by load balancer type, but the fundamental question is the same: are your targets healthy, and is the routing configuration correct?

2. What "Not Routing Traffic" Means

Load balancers route traffic to targets (EC2 instances, IPs, Lambda functions, or ECS tasks) registered in target groups. Before routing any traffic to a target, the load balancer runs health checks. If health checks fail, the target is marked unhealthy and receives no traffic. A load balancer with all unhealthy targets returns 503 to every request.

3. Common Causes

Target health checks are failing — the most common cause by far
Security group on the instances doesn't allow inbound from the load balancer SG
Wrong health check path (404 when 200 is expected), wrong port, or wrong protocol
Target group is empty — no instances are registered
EKS: AWS Load Balancer Controller is not installed or has wrong IAM permissions
EKS: Ingress annotation class mismatch (kubernetes.io/ingress.class vs alb)
EKS: Subnet tags missing (kubernetes.io/role/elb or kubernetes.io/role/internal-elb)
Listener rules not matching the request path or host header
For NLBs with IP target type in EKS: pod security groups blocking NLB traffic

4. Step-by-Step Diagnosis and Fix

Step 1: Check target group health status

# List target groups and their health
aws elbv2 describe-target-groups   --load-balancer-arn arn:aws:elasticloadbalancing:us-east-1:123456789012:loadbalancer/app/my-alb/abc123

# Check specific target group health
aws elbv2 describe-target-health   --target-group-arn arn:aws:elasticloadbalancing:us-east-1:123456789012:targetgroup/my-tg/abc123

# Look for TargetHealth.State: "healthy" | "unhealthy" | "draining"
# For unhealthy targets, check TargetHealth.Description for the reason

Step 2: Diagnose health check failures

# Get health check configuration
aws elbv2 describe-target-groups   --target-group-arns arn:aws:elasticloadbalancing:...   --query 'TargetGroups[0].{Protocol:HealthCheckProtocol,Path:HealthCheckPath,Port:HealthCheckPort,Codes:Matcher}'

# Common fix: wrong health check path
# Update health check path to one that returns 200
aws elbv2 modify-target-group   --target-group-arn arn:aws:elasticloadbalancing:...   --health-check-path /health   --matcher HttpCode=200

# Test the health check endpoint directly from inside the VPC
curl -sv http://10.0.1.50:8080/health
# Should return HTTP 200

Step 3: Fix security group rules

The target instances need an inbound rule from the ALB's security group. This is the second most common cause of health check failures. See Fix AWS Security Group Misconfiguration for the full diagnostic.

# Get the ALB's security group
aws elbv2 describe-load-balancers   --names my-alb   --query 'LoadBalancers[0].SecurityGroups'

# Add inbound rule: allow ALB SG to reach instances on app port
aws ec2 authorize-security-group-ingress   --group-id sg-instance-0abc12345   --protocol tcp   --port 8080   --source-group sg-alb-0abc12345

Step 4: Check listener rules for ALBs

# List listener ARNs
aws elbv2 describe-listeners   --load-balancer-arn arn:aws:elasticloadbalancing:...

# View rules for a listener
aws elbv2 describe-rules   --listener-arn arn:aws:elasticloadbalancing:...:listener/app/my-alb/abc123/def456

# If there's no rule matching your request, add one
aws elbv2 create-rule   --listener-arn arn:aws:elasticloadbalancing:...   --priority 10   --conditions Field=path-pattern,Values='/*'   --actions Type=forward,TargetGroupArn=arn:aws:elasticloadbalancing:...

Step 5: EKS — Fix Load Balancer Controller issues

# Check if the controller is running
kubectl get pods -n kube-system -l app.kubernetes.io/name=aws-load-balancer-controller

# Check controller logs for errors
kubectl logs -n kube-system   -l app.kubernetes.io/name=aws-load-balancer-controller   --tail=50

# Verify the controller has correct IAM permissions (via IRSA)
kubectl get serviceaccount aws-load-balancer-controller -n kube-system -o yaml
# Should have: eks.amazonaws.com/role-arn annotation

# Check that subnet tags exist (required for controller to discover subnets)
# For internet-facing ALBs:
aws ec2 create-tags   --resources subnet-public-0abc12345   --tags Key=kubernetes.io/role/elb,Value=1

# For internal ALBs:
aws ec2 create-tags   --resources subnet-private-0abc12345   --tags Key=kubernetes.io/role/internal-elb,Value=1

Step 6: EKS — Verify Ingress annotation and resource

# Check Ingress resource status
kubectl describe ingress my-ingress -n my-namespace

# Look for: Address field (should contain the ALB DNS name)
# Events field shows errors from the Load Balancer Controller

# Required annotations for ALB Ingress:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: my-ingress
  annotations:
    kubernetes.io/ingress.class: alb          # or use ingressClassName field
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip  # for EKS pod-IP targeting

# If using ingressClassName instead (Kubernetes 1.18+):
spec:
  ingressClassName: alb
  rules: [...]

5. Verification Steps

# Confirm targets are healthy
aws elbv2 describe-target-health   --target-group-arn arn:aws:elasticloadbalancing:...   --query 'TargetHealthDescriptions[].{Target:Target.Id,Health:TargetHealth.State}'

# Send a test request to the load balancer
curl -sv http://my-alb-1234567890.us-east-1.elb.amazonaws.com/health

# For EKS, confirm the Ingress has an address
kubectl get ingress my-ingress -n my-namespace
# ADDRESS column should show the ALB DNS name

# Check ALB access logs for error patterns (if enabled)
aws s3 ls s3://my-alb-logs-bucket/AWSLogs/123456789012/elasticloadbalancing/us-east-1/

6. Common Mistakes

Not checking the target health reason — the AWS console and CLI both show why a target is unhealthy; always read it
Forgetting to add the ALB's security group as an inbound rule on instance security groups
Using the wrong target type in EKS (instance vs ip) — this affects both routing and security group requirements
Missing subnet tags for the AWS Load Balancer Controller — it will fail silently without them
ALB Controller deployed without correct IAM policy — check it has elasticloadbalancing:*, ec2:Describe* and related permissions
Health check path returning a redirect (301/302) — configure the load balancer to follow redirects, or fix the health check path

7. Prevention Tips

Enable ALB access logs to S3 — they provide per-request visibility into 4xx/5xx errors and target selection
Use infrastructure-as-code (Terraform, CDK) to manage load balancer configurations so changes are tracked
Set up CloudWatch alarms on UnHealthyHostCount > 0 for each target group
Test health check endpoints independently before registering targets with the load balancer
For EKS, use a dedicated IAM role for the Load Balancer Controller via IRSA — never use node instance profile permissions
Document required security group rules for each load balancer as code comments or in a runbook

8. FAQ

My ALB shows "active" but returns 503. What's happening?

503 means the load balancer is working but all targets in the target group are unhealthy. Check describe-target-health — every target will show as "unhealthy" with a specific reason. The most common reasons: failed health check (wrong path/port), security group blocking the health check, or the application actually isn't running on the expected port.

The EKS Ingress was created but the ADDRESS field is empty. Why?

The AWS Load Balancer Controller couldn't provision the ALB. Check controller logs with kubectl logs. Common reasons: missing subnet tags (kubernetes.io/role/elb), the controller doesn't have IAM permissions, or the annotation class (kubernetes.io/ingress.class: alb) doesn't match what the controller is watching for.

Health checks pass but real traffic still fails. Why?

Health checks and real traffic may use different ports or paths. Check the listener configuration — health checks might be on port 8080/health while real traffic routes to 8080/api, which has different behavior. Also check if the health check uses a different protocol (HTTP vs HTTPS) than production traffic.

9. Summary

Error	Root Cause	Fix
503 on all requests	All targets unhealthy	Fix health check path/port or security groups
Health check: connection refused	App not listening on expected port	Align health check port with app listener port
Health check: timeout	Security group blocking ALB to instance	Add inbound rule from ALB SG to instance SG
EKS Ingress: no ADDRESS	Controller can't provision ALB	Check subnet tags, IAM permissions, controller logs
Intermittent 502 errors	Target connection errors	Check target app errors; tune connection draining

Explore More in This Category

Explore more in this category: AWS & Cloud guides. Browse all DevOps Compass articles or jump to: Kubernetes, AWS, CI/CD, Containers, Monitoring, Networking.