Enterprise Cloud Migration: Moving ClearPay from AWS to Google Kubernetes Engine Without Disrupting Production

In modern cloud architecture, migrating a running application between cloud providers while maintaining absolute continuity of service is one of the most formidable engineering challenges. Recently, engineering teams at fintech organizations have focused heavily on orchestrating zero-downtime migrations between Amazon Web Services (AWS) and Google Cloud Platform (GCP). A prominent example of this complex operational journey is the migration of ClearPay, a payments application featuring a React user interface and an Express API, from a single AWS Elastic Compute Cloud (EC2) instance to Google Kubernetes Engine (GKE).

The core objective of this architectural transition is to modernize the application infrastructure by moving the stateless user interface and API layers to Google Kubernetes Engine. Crucially, the project’s parameters dictate that the backend PostgreSQL database remains securely hosted on Amazon RDS (Relational Database Service) throughout the process. By keeping the database stationary, the engineering team effectively limits the scope of variables, avoiding the simultaneous migration of persistent data storage and application compute layers. This methodical approach significantly mitigates the risk of catastrophic data inconsistency and simplifies troubleshooting during cutover phases.

Preparing the Baseline and Establishing Infrastructure

Before introducing a second cloud provider into the operational ecosystem, the engineering team must establish a trusted baseline. This foundational phase involves running the application locally via Docker Compose, provisioning an identical single-server environment on AWS using Terraform, and rigorously measuring baseline performance metrics. Crucially, the application relies on idempotency keys to ensure that retried payment requests—which frequently occur during network timeouts or client-side application freezes—never result in duplicate financial charges. Proving this transaction safety mechanism locally and subsequently on AWS creates an immutable standard that the destination environment on Google Cloud must match.

Once the AWS baseline is established and verified via automated promotion gates and load tests—recording vital performance indicators such as p95 latency and transaction success rates—the architectural planning phase commences. The engineering design intentionally avoids relying on DNS-based traffic splitting. Because Domain Name System resolvers and internet service providers aggressively cache records, relying on DNS propagation for traffic shifts can leave legacy clients stranded on deprecated infrastructure for hours. Instead, the architecture deploys a global HTTP(S) load balancer on Google Cloud equipped with weighted backend services. This configuration allows engineers to precisely control traffic distribution down to the exact percentage, enabling real-time adjustments and instantaneous rollbacks if anomalies arise.

Building Google Cloud Infrastructure via Terraform

The construction of the destination environment on Google Cloud relies on a rigorous, multi-stage Terraform implementation divided into eight sequential modules. Rather than deploying the entire infrastructure in a monolithic operation, the engineering team provisions resources incrementally—starting with foundational Google API service activations, progressing through custom Virtual Private Cloud (VPC) network and subnet designs, and culminating in the deployment of a private GKE Standard cluster and a global external managed load balancer.

The network architecture leverages Cloud NAT to provide secure outbound internet access for private GKE nodes while preventing direct inbound exposure. To connect this newly minted Google Cloud environment to the existing AWS database without exposing the PostgreSQL database publicly, engineers configure strict IP allowlisting. Specifically, the static public IP address assigned to the Google Cloud NAT gateway is added exclusively to the AWS security group governing the RDS instance. Furthermore, connection strings enforce mandatory Transport Layer Security encryption, ensuring that cross-cloud database queries remain secure as they traverse the public internet between AWS us-east-1 and GCP us-east4.

Deploying to Staging and Executing Automated Pipelines

With the core infrastructure established, containerized application images are built, thoroughly scanned for security vulnerabilities using Trivy, and pushed to a private Artifact Registry on Google Cloud. To maintain strict security boundaries, the deployment pipeline avoids the traditional anti-pattern of storing long-lived service account JSON key files in GitHub repositories. Instead, the workflow leverages Workload Identity Federation, establishing a direct, token-based trust relationship between GitHub Actions and Google Cloud IAM.

The deployment of the application workloads into the GKE cluster relies on Kubernetes manifests managed through Kustomize overlays. Secret management is handled via the Secret Manager Secret Store CSI Driver, allowing pods to mount database credentials and JWT signing keys securely into memory without committing plain-text secrets to source control. Once the application pods achieve a healthy status within the private staging namespace, engineers execute an automated promotion gate script via port-forwarding tunnels. This validation script tests system health, database readiness, authentication flows, idempotency replay protection, and double-debit prevention. Only after these rigorous checks pass successfully is the application deemed ready for production exposure.

Managing the Gradual Traffic Shift and Mitigating Risks

Transitioning production traffic from AWS to Google Kubernetes Engine requires a methodical, step-by-step weight adjustment on the global load balancer. Rather than executing a high-risk "big bang" cutover, traffic is initially routed to GKE at a nominal one percent. Throughout this gradual escalation—progressing through ten, twenty-five, fifty, seventy-five, ninety, and finally one hundred percent—site reliability engineers continuously monitor error rates, latency percentiles, and synthetic transaction health checks.

A critical engineering consideration during this dual-run period involves connection pool math. Because both the legacy AWS server and the newly scaled GKE pods actively process requests concurrently while sharing the exact same PostgreSQL database, the cumulative maximum database connections across both environments must not exceed the database server’s max_connections limit. Rigorous calculation of potential pod scaling limits against database thresholds prevents resource exhaustion, connection timeouts, and cascading application failures. In the event that elevated error rates or degraded performance metrics breach predefined operational thresholds, engineers can execute an immediate, automated rollback script. This operation instantaneously restores 100 percent traffic routing to AWS and updates the user interface status banner to inform operators of the active rollback state.

Finalizing the Cutover and Retiring Legacy Infrastructure

Following the successful progression of traffic weights to 100 percent on Google Kubernetes Engine, the system undergoes a sustained observation period to verify operational stability under real-world production load. Performance comparisons between the initial AWS baseline and the final GKE architecture reveal valuable insights into cross-cloud latency dynamics, confirming that while the API successfully scaled and maintained zero customer-facing downtime, network transit times across cloud boundaries warrant future optimization, such as eventually migrating the database tier into the same cloud provider ecosystem.

With production fully operational on GKE and all verification scripts consistently passing, the temporary AWS compute infrastructure is officially decommissioned. By executing targeted Terraform destroy scripts specifically scoped to the legacy EC2 instance, Elastic IP, and associated security groups while preserving the primary RDS database instance, the organization successfully eliminates redundant cloud expenditure. The comprehensive migration record and postmortem analysis document these architectural decisions, providing a valuable operational blueprint for future infrastructure modernization initiatives.