The traditional security model is a castle: hard perimeter, soft interior. Firewalls and VPNs guard the boundary, and once you are inside, you are trusted to access anything. This model was already fragile before the cloud; now it is indefensible. When your application runs across three AWS regions, your developers work from home on personal networks, your CI/CD pipeline has broader network access than any individual employee, and your SaaS product is accessible from any browser on earth, there is no perimeter left to defend.

Zero-trust architecture replaces the perimeter model with a single unifying principle: never trust, always verify. Every request, from every source, at every layer of the stack, must prove its identity and demonstrate its authorization to perform the requested action. There is no trusted network. There is no trusted device. There is no trusted service. Trust is earned per-request through cryptographic verification, not assumed by network location.

We adopted zero-trust principles at Harbor Software eighteen months ago, driven by a near-miss security incident where a compromised developer laptop had unrestricted access to our production database because it was “inside the VPN.” Here is how we implemented zero-trust across our SaaS product stack, layer by layer, and the practical tradeoffs we made along the way.

Identity: The Foundation of Everything

In a zero-trust architecture, identity is the new perimeter. Every entity in your system needs a verified, cryptographic identity: users, services, CI/CD pipelines, cron jobs, and infrastructure components. Anonymous or ambient access should not exist anywhere, not even for internal services communicating within the same Kubernetes cluster.

User Identity: Short-Lived Tokens

User authentication uses short-lived JWTs issued by our identity provider (Auth0). Tokens expire after 15 minutes and are refreshed silently via refresh tokens stored in httpOnly cookies. This limits the exposure window if a token is intercepted: an attacker has at most 15 minutes to use a stolen access token before it becomes worthless.

// JWT verification on every API request
const jwtConfig = {
  issuer: 'https://harbor.auth0.com/',
  audience: 'https://api.harborsoftware.com',
  algorithms: ['RS256'],  // Asymmetric signing - verify with public key only
  clockTolerance: 30,     // 30 seconds tolerance for clock skew between servers
};

async function authenticateUser(req: Request, res: Response, next: NextFunction) {
  const token = req.headers.authorization?.replace('Bearer ', '');
  if (!token) {
    return res.status(401).json({
      error: { code: 'UNAUTHENTICATED', message: 'Bearer token required' }
    });
  }

  try {
    const decoded = await verifyJWT(token, jwtConfig);

    // Extract claims into a typed auth context
    req.auth = {
      userId: decoded.sub,
      email: decoded.email,
      tenantId: decoded['https://harbor.com/tenant_id'],
      permissions: decoded.permissions || [],
      tokenIssuedAt: new Date(decoded.iat * 1000),
    };

    // Log successful authentication for audit trail
    logger.info({
      event: 'auth.success',
      userId: req.auth.userId,
      tenantId: req.auth.tenantId,
      ip: req.ip,
      userAgent: req.headers['user-agent'],
    });

    next();
  } catch (error: any) {
    if (error.name === 'TokenExpiredError') {
      return res.status(401).json({
        error: { code: 'TOKEN_EXPIRED', message: 'Access token has expired' }
      });
    }

    logger.warn({
      event: 'auth.failed',
      reason: error.message,
      ip: req.ip,
      userAgent: req.headers['user-agent'],
    });

    return res.status(401).json({
      error: { code: 'INVALID_TOKEN', message: 'Token verification failed' }
    });
  }
}

We use RS256 (RSA-SHA256) rather than HS256 for JWT signing. RS256 uses asymmetric keys: the identity provider signs with a private key, and our API verifies with a public key. This means our API never possesses the signing key. Even if our API server is fully compromised, the attacker cannot forge tokens because the private key exists only in Auth0’s infrastructure. With HS256, the same shared secret is used for both signing and verification, meaning a compromised API server can forge tokens for any user.

Service Identity: Mutual TLS

Internal services authenticate to each other using mTLS (mutual TLS). Unlike standard TLS where only the server proves its identity, mTLS requires both sides to present certificates. We run Istio as our service mesh, which handles certificate issuance, rotation, and verification automatically:

# Istio PeerAuthentication: require mTLS for all traffic in production
apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: strict-mtls
  namespace: production
spec:
  mtls:
    mode: STRICT   # Reject any non-mTLS traffic

---
# Authorization: only allow specific service-to-service communication
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: billing-service-access
  namespace: production
spec:
  selector:
    matchLabels:
      app: billing-service
  action: ALLOW
  rules:
    - from:
        - source:
            # Only the API service's service account can call billing
            principals:
              - "cluster.local/ns/production/sa/api-service"
      to:
        - operation:
            methods: ["POST", "GET"]
            paths:
              - "/internal/invoices/*"
              - "/internal/subscriptions/*"
    - from:
        - source:
            # Billing worker can also access billing service
            principals:
              - "cluster.local/ns/production/sa/billing-worker"
      to:
        - operation:
            methods: ["POST"]
            paths:
              - "/internal/process-payment"

With mTLS and Istio authorization policies, every service has a cryptographic identity (its SPIFFE identity derived from its Kubernetes service account). The mesh verifies identities automatically on every request. An attacker who compromises one service cannot impersonate another service because they do not possess its private key. And even if they could somehow intercept traffic, they cannot read or modify it because all communication is encrypted.

CI/CD Identity: OIDC Federation

CI/CD pipelines are one of the most dangerous attack vectors because they typically have broad access (deploy to production, read secrets, push container images, modify infrastructure) combined with weak identity controls. Many organizations still use long-lived AWS access keys stored as CI secrets, which never expire and are shared across all workflow runs.

We eliminated long-lived credentials entirely using OIDC federation:

# GitHub Actions: OIDC for AWS authentication
jobs:
  deploy:
    runs-on: ubuntu-latest
    permissions:
      id-token: write   # Required for OIDC token
      contents: read
    steps:
      - uses: aws-actions/configure-aws-credentials@v2
        with:
          role-to-assume: arn:aws:iam::123456789:role/github-actions-deploy
          aws-region: us-east-1
          # No AWS_ACCESS_KEY_ID. No AWS_SECRET_ACCESS_KEY.
          # GitHub's OIDC provider issues a short-lived token.
          # AWS STS exchanges it for temporary credentials scoped to this specific role.
          # Credentials expire when the workflow completes.

The IAM role’s trust policy restricts which GitHub repositories and branches can assume it:

{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Principal": {
      "Federated": "arn:aws:iam::123456789:oidc-provider/token.actions.githubusercontent.com"
    },
    "Action": "sts:AssumeRoleWithWebIdentity",
    "Condition": {
      "StringEquals": {
        "token.actions.githubusercontent.com:aud": "sts.amazonaws.com"
      },
      "StringLike": {
        "token.actions.githubusercontent.com:sub": "repo:harbor-software/platform:ref:refs/heads/main"
      }
    }
  }]
}

This means: only GitHub Actions workflows running from the harbor-software/platform repository on the main branch can assume this deployment role. A workflow from a fork, a different branch, or a different repository cannot. There are no static credentials to steal, no secrets to rotate, and no risk of credential leakage through log exposure.

Network: Defense in Depth, Not the Primary Boundary

Zero trust does not mean “no firewalls.” It means the network is one layer of defense among many, not the primary security boundary. We still implement network segmentation, but we do not rely on it as the sole control preventing unauthorized access.

Microsegmentation with Security Groups

Every service has its own security group with deny-all defaults. Communication paths are explicitly allowed based on actual service dependencies:

resource "aws_security_group" "api_service" {
  name        = "api-service-sg"
  description = "API service - accepts traffic from ALB only"
  vpc_id      = var.vpc_id

  # Ingress: ONLY from the application load balancer
  ingress {
    description     = "HTTP from ALB"
    from_port       = 8080
    to_port         = 8080
    protocol        = "tcp"
    security_groups = [aws_security_group.alb.id]
  }

  # Egress: explicitly listed destinations
  egress {
    description     = "PostgreSQL database"
    from_port       = 5432
    to_port         = 5432
    protocol        = "tcp"
    security_groups = [aws_security_group.database.id]
  }

  egress {
    description     = "Redis cache"
    from_port       = 6379
    to_port         = 6379
    protocol        = "tcp"
    security_groups = [aws_security_group.redis.id]
  }

  egress {
    description = "HTTPS outbound for external APIs"
    from_port   = 443
    to_port     = 443
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  egress {
    description = "DNS resolution"
    from_port   = 53
    to_port     = 53
    protocol    = "udp"
    cidr_blocks = [var.vpc_cidr]
  }

  # No other egress. The API service cannot reach services it does not need.
}

This means: if an attacker compromises the API service, they can reach the database and Redis (which are protected by their own authentication), make outbound HTTPS calls, and resolve DNS. They cannot reach other internal services directly, cannot access the metadata service for credential theft (we enforce IMDSv2 with hop limit 1), and cannot scan the internal network for additional targets.

VPC Endpoints: Keep Traffic Off the Public Internet

AWS services (S3, SQS, Secrets Manager, KMS, ECR) are accessed via VPC endpoints rather than over the public internet. This provides two security benefits: traffic stays on the AWS backbone network (not traversing the internet where it could be intercepted), and endpoint policies can restrict which resources are accessible:

resource "aws_vpc_endpoint" "s3" {
  vpc_id            = var.vpc_id
  service_name      = "com.amazonaws.us-east-1.s3"
  vpc_endpoint_type = "Gateway"
  route_table_ids   = var.private_route_table_ids

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Sid       = "RestrictToOurBuckets"
      Effect    = "Allow"
      Principal = "*"
      Action    = ["s3:GetObject", "s3:PutObject", "s3:ListBucket"]
      Resource  = [
        "arn:aws:s3:::harbor-*",
        "arn:aws:s3:::harbor-*/*"
      ]
    }]
  })
}

resource "aws_vpc_endpoint" "secrets_manager" {
  vpc_id              = var.vpc_id
  service_name        = "com.amazonaws.us-east-1.secretsmanager"
  vpc_endpoint_type   = "Interface"
  subnet_ids          = var.private_subnet_ids
  security_group_ids  = [aws_security_group.vpc_endpoints.id]
  private_dns_enabled = true
}

The S3 endpoint policy is particularly important. It restricts S3 access through this endpoint to only our buckets (matching the harbor-* prefix). Even if an attacker gains the ability to make S3 API calls from within our VPC, they cannot exfiltrate data to their own S3 bucket because the endpoint policy blocks access to non-Harbor buckets.

Data: Encryption Without Exceptions

Zero trust requires encryption at rest and in transit for all data, with no exceptions for “internal” traffic or “non-sensitive” data stores.

Encryption in Transit: TLS Everywhere

All communication channels use TLS 1.3, including channels that traditional architectures would consider “safe” because they are internal:

• Service-to-service: mTLS via Istio (covered above)
• Service-to-database: TLS required. Connection strings use sslmode=verify-full, which verifies both that the connection is encrypted and that the server’s certificate matches the expected hostname. This prevents MITM attacks even within the VPC.
• Service-to-cache: Redis 6+ with TLS enabled. In-transit encryption for cache data prevents credential harvesting from network traffic sniffing.
• Service-to-message-broker: RabbitMQ with TLS listener. Message payloads often contain sensitive data (user emails, order details) that must be encrypted in transit.

// Database connection: TLS with certificate verification
const dbConfig = {
  connectionString: process.env.DATABASE_URL,
  ssl: {
    rejectUnauthorized: true,  // Verify server certificate
    ca: fs.readFileSync('/etc/ssl/certs/rds-combined-ca-bundle.pem'),
  },
};

// Redis connection: TLS enabled
const redisConfig = {
  url: process.env.REDIS_URL,  // rediss:// scheme (note double s) = TLS
  tls: {
    rejectUnauthorized: true,
    ca: fs.readFileSync('/etc/ssl/certs/elasticache-ca.pem'),
  },
};

Encryption at Rest: Customer-Managed Keys

All data stores use encryption at rest with customer-managed KMS keys. This gives us the ability to audit key usage through CloudTrail, rotate keys automatically, and most critically, revoke access instantly by modifying the key policy:

resource "aws_kms_key" "database" {
  description             = "Encryption key for production database"
  deletion_window_in_days = 30
  enable_key_rotation     = true  # Automatic annual rotation

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Sid       = "KeyAdministration"
        Effect    = "Allow"
        Principal = { AWS = "arn:aws:iam::123456789:role/security-admin" }
        Action    = ["kms:Create*", "kms:Describe*", "kms:Enable*", "kms:List*",
                     "kms:Put*", "kms:Update*", "kms:Revoke*", "kms:Disable*",
                     "kms:Get*", "kms:Delete*", "kms:ScheduleKeyDeletion",
                     "kms:CancelKeyDeletion"]
        Resource  = "*"
      },
      {
        Sid       = "DatabaseEncryption"
        Effect    = "Allow"
        Principal = { Service = "rds.amazonaws.com" }
        Action    = ["kms:Encrypt", "kms:Decrypt", "kms:ReEncrypt*",
                     "kms:GenerateDataKey*", "kms:DescribeKey"]
        Resource  = "*"
        Condition = {
          StringEquals = {
            "kms:ViaService" = "rds.us-east-1.amazonaws.com"
          }
        }
      }
    ]
  })
}

resource "aws_rds_cluster" "production" {
  cluster_identifier  = "harbor-production"
  storage_encrypted   = true
  kms_key_id          = aws_kms_key.database.arn
  # ... other configuration
}

Least Privilege: Narrow Permissions Everywhere

The principle of least privilege is the operational backbone of zero trust. Every identity should have exactly the permissions it needs to perform its function and nothing more. This applies to user IAM roles, service IAM roles, CI/CD roles, database users, and even Kubernetes service accounts.

// Bad: overly broad permissions that grant far more access than needed
{
  "Effect": "Allow",
  "Action": "s3:*",
  "Resource": "*"
}

// Good: minimal permissions scoped to specific resources with conditions
{
  "Effect": "Allow",
  "Action": [
    "s3:GetObject",
    "s3:PutObject"
  ],
  "Resource": "arn:aws:s3:::harbor-uploads/tenants/${aws:PrincipalTag/tenant_id}/*",
  "Condition": {
    "StringEquals": {
      "s3:x-amz-server-side-encryption": "aws:kms"
    },
    "StringLike": {
      "s3:x-amz-server-side-encryption-aws-kms-key-id": "arn:aws:kms:us-east-1:123456789:key/tenant-uploads-key-*"
    }
  }
}

This policy allows the service to read and write objects only within the tenant-specific prefix of the uploads bucket (using the tenant_id tag from the calling principal for scoping), and only if the objects are encrypted with a specific KMS key. The service cannot list the bucket, delete objects, modify bucket policies, access objects in other tenant prefixes, or store unencrypted objects. Every unnecessary capability has been removed.

We use IAM Access Analyzer running continuously to detect permission policies that are broader than they need to be. When Access Analyzer flags a policy as overly permissive, we review and tighten it within the week. We also run quarterly access reviews where we audit all IAM roles and their actual usage (via CloudTrail) and remove permissions that have not been used in 90 days.

Secrets Management: No Hardcoding, Automatic Rotation

In a zero-trust architecture, secrets are never hardcoded in source code, never stored in environment variables that might be logged, and rotated automatically on a schedule:

resource "aws_secretsmanager_secret" "database_password" {
  name                    = "production/database/password"
  description             = "Production database master password"
  recovery_window_in_days = 7
}

resource "aws_secretsmanager_secret_rotation" "database_password" {
  secret_id           = aws_secretsmanager_secret.database_password.id
  rotation_lambda_arn = aws_lambda_function.rotate_db_password.arn

  rotation_rules {
    automatically_after_days = 30
  }
}

// Application: fetch secrets at startup, refresh periodically
class SecretManager {
  private cache: Map<string, { value: string; fetchedAt: number }> = new Map();
  private client: SecretsManagerClient;
  private maxAgeMs: number;

  constructor(maxAgeMs = 3600000) { // Refresh every hour
    this.client = new SecretsManagerClient({ region: 'us-east-1' });
    this.maxAgeMs = maxAgeMs;
  }

  async getSecret(name: string): Promise<string> {
    const cached = this.cache.get(name);
    if (cached && Date.now() - cached.fetchedAt < this.maxAgeMs) {
      return cached.value;
    }

    const result = await this.client.send(
      new GetSecretValueCommand({ SecretId: name })
    );

    const value = result.SecretString!;
    this.cache.set(name, { value, fetchedAt: Date.now() });
    return value;
  }
}

Automatic rotation is the most impactful secret management practice. A compromised secret that rotates every 30 days has a maximum exposure window of 30 days. A secret that never rotates has an infinite exposure window. The rotation Lambda handles the multi-step process (create new password, update database, test connectivity, finalize) atomically, with rollback if any step fails.

Comprehensive Logging and Anomaly Detection

Zero trust requires comprehensive logging because the verification happens at every layer, and you need to detect when verification is being abused or bypassed. Every authentication attempt, authorization decision, data access, and administrative action should produce an audit log entry:

// Security event types we log
type SecurityEvent =
  | { type: 'auth.success'; userId: string; method: string }
  | { type: 'auth.failed'; reason: string; ip: string }
  | { type: 'authz.granted'; userId: string; resource: string; action: string }
  | { type: 'authz.denied'; userId: string; resource: string; action: string }
  | { type: 'data.accessed'; userId: string; resourceType: string; resourceId: string }
  | { type: 'admin.role_changed'; actorId: string; targetId: string; changes: object }
  | { type: 'admin.secret_accessed'; actorId: string; secretName: string }
  | { type: 'anomaly.geo_change'; userId: string; previousCountry: string; newCountry: string }
  | { type: 'anomaly.impossible_travel'; userId: string; details: object };

function logSecurityEvent(event: SecurityEvent & { tenantId: string; requestId: string }) {
  securityLogger.info({
    ...event,
    timestamp: new Date().toISOString(),
    environment: process.env.NODE_ENV,
  });

  // High-severity events trigger real-time alerts
  if (['auth.failed', 'authz.denied', 'anomaly.impossible_travel'].includes(event.type)) {
    alertingService.evaluate(event);
  }
}

We alert on patterns that indicate compromise attempts: more than 5 failed authentication attempts from the same IP in 5 minutes, successful authentication from a new country within 1 hour of authentication from a different country (impossible travel), authorization denials for admin-level resources from non-admin users, and bulk data access patterns (downloading more than 1000 records in a single session).

Implementation Roadmap: Six Months to Zero Trust

Zero trust is not a weekend project, but it does not have to be a multi-year initiative either. Here is the order we recommend, prioritized by security impact per effort:

1. Month 1: Identity foundation. Short-lived JWTs for users. Service account tokens for internal services. OIDC federation for CI/CD. Remove all long-lived credentials. This alone eliminates the most common attack vector (compromised credentials).
2. Month 2: Authorization. RBAC with resource-level checks on every API endpoint. Audit logging for all authorization decisions. Frontend permission-based UI. This prevents horizontal and vertical privilege escalation.
3. Month 3: Network segmentation. Security groups with deny-all defaults. VPC endpoints for AWS services. mTLS for service-to-service communication. This limits lateral movement after compromise.
4. Month 4: Data encryption. Encryption at rest with KMS for all data stores. TLS for all connections including internal ones. This protects data even if network or storage is compromised.
5. Month 5: Secrets management. Migrate all secrets to Secrets Manager. Implement automatic rotation. Remove hardcoded credentials from code, environment variables, and CI configuration. This reduces the exposure window of any compromised secret.
6. Month 6: Detection and response. Comprehensive security logging. Anomaly detection rules. IAM Access Analyzer. Incident response runbooks. This enables you to detect and respond to breaches that bypass preventive controls.

Conclusion

Zero-trust security is not a product you install or a configuration you enable. It is an architectural principle applied consistently across every layer of your stack: identity, authorization, network, data, secrets, and detection. Each layer reinforces the others. Strong identity is insufficient without authorization checks. Network segmentation is insufficient without identity verification. Encryption is insufficient without key management. Detection is insufficient without comprehensive logging.

For a SaaS product, the investment in zero trust pays for itself the first time an attacker compromises a single component and finds that they cannot move laterally to other services, cannot escalate their privileges to admin, cannot access data belonging to other tenants, and cannot exfiltrate the data they can access because every action is logged and anomalies trigger alerts. The castle model assumes the perimeter will hold. Zero trust assumes it will not, and builds defenses accordingly at every layer. That assumption is not pessimism; in 2022, it is realism.