AWS Compute Services Explained: EC2 vs ECS vs EKS

Introduction

Choosing the right AWS compute service isn't just a technical decision - it directly affects your team's velocity, your operational costs, and how quickly your business can respond to change. Pick the wrong one and you'll either pay for complexity you don't need, or find yourself locked into a setup that can't scale with you.

EC2, ECS, and EKS are not competing services. They solve fundamentally different problems at different levels of abstraction. EC2 gives you a raw virtual machine with full infrastructure control. ECS is a managed container platform built natively on AWS - no Kubernetes required. EKS brings the full power of Kubernetes to AWS for teams that need portability and advanced orchestration at scale.

Here is what we cover:

• What EC2, ECS, and EKS actually are - in plain terms
• Key differences across abstraction, scaling, cost, and operational overhead
• What each service costs, with real numbers from AWS's official pricing pages
• Real-world use cases and business implications for each
• A decision guide to help you pick the right one

Understanding the Basics

Amazon EC2 - Virtual Machines

EC2 is AWS's virtual machine service. You choose the operating system, CPU, memory, and storage - AWS provisions the server. Think of it as renting a physical server in the cloud. Your team owns everything from that point on: patching, scaling, monitoring, and security. Maximum control, maximum responsibility.

Business implication: EC2 requires engineering time to manage and maintain. That time has a cost. It is best suited for workloads where your team needs deep infrastructure control, or for migrating existing applications that weren't built for containers.

Amazon ECS - Managed Containers

ECS is AWS's native container orchestration service. You define your Docker image, CPU and memory requirements, and scaling rules. AWS handles the rest - scheduling, cluster management, health checks, and integrations with AWS services like Application Load Balancer, IAM, and CloudWatch. No Kubernetes knowledge required.

Business implication: ECS reduces the operational surface area significantly. Smaller teams can run production container workloads without dedicated DevOps headcount. It is AWS's recommended starting point for teams new to containers.

Amazon EKS - Managed Kubernetes

EKS runs Kubernetes on AWS. AWS manages the control plane - the brain of the cluster - while you manage the worker nodes, or offload them to AWS Fargate for a fully serverless setup. Kubernetes is the industry-standard container orchestration platform, and EKS makes it available as a managed service.

Business implication: EKS is the most powerful and flexible option, but it brings the highest operational complexity and cost. It pays off at scale, for teams running complex microservices architectures, or for organisations that want the option to run workloads across multiple cloud providers.

Key Differences at a Glance

Feature	EC2	ECS	EKS
Abstraction Level	Low (Infrastructure)	Medium (Platform)	High (Ecosystem)
Primary Unit	VM / Server	Container Task	Pod
Setup Complexity	High	Low	Very High
Scaling Speed	Slow (VM boot)	Fast	Fast
OS / Host Access	Full	None	Limited
Kubernetes Support	No	No	Yes
Multi-cloud Portability	Low	Low	High
Operational Overhead	High	Medium	Very High
Learning Curve	Low–Medium	Low	High
Relative Cost (entry)	Low–Medium	Medium	High
Best for	Legacy apps, full control	AWS-native container apps	Complex microservices, multi-cloud

Cost Breakdown - With Real Numbers

Cost is one of the most common decision factors, and it is also one of the most misunderstood. The sticker price of compute is only part of the picture. Operational overhead - the engineering time spent managing infrastructure - is a real cost that doesn't show up on your AWS bill but does show up on your payroll.

EC2

With EC2, you pay for instance uptime. AWS offers four main pricing models: On-Demand (pay by the second, no commitment), Reserved Instances (commit to 1 or 3 years for up to 72% off On-Demand rates, per official AWS pricing), Savings Plans (flexible commitment-based discounts, AWS's currently recommended approach over Reserved Instances), and Spot Instances (spare capacity at up to 90% off, but AWS can reclaim with two minutes notice). Most production workloads that run around the clock should be on Reserved Instances or a Savings Plan - running purely On-Demand for steady-state workloads leaves significant money on the table.

Business implication: EC2 can be the most cost-efficient option for stable, predictable workloads - but only if your team actively manages instance sizing and pricing commitments. Teams that over-provision and forget tend to pay more than they should.

ECS

ECS has no additional management fee. You pay for the underlying compute - either EC2 instances you manage yourself, or AWS Fargate (fully serverless, where AWS manages the infrastructure). With Fargate on ECS, billing is per second based on the vCPU and memory your containers actually use. As a reference, in the US East (N. Virginia) region, Fargate charges approximately $0.04048 per vCPU-hour and $0.004445 per GB-hour for memory, per AWS's official Fargate pricing page. A container running 2 vCPUs and 4 GB of RAM costs roughly $0.099 per hour, or around $72 per month running continuously.

Business implication: Fargate on ECS is often the most cost-effective choice for bursty, unpredictable, or variable traffic patterns - you pay only for what you use, and there is no idle compute cost. For high-volume, steady-state workloads, EC2-backed ECS can be cheaper.

EKS

EKS has a fixed control plane fee of $0.10 per cluster per hour - approximately $73 per month per cluster - regardless of cluster size or workload, per AWS's official EKS pricing page. This is just the management fee; worker node compute (EC2 or Fargate), storage, load balancers, and data transfer are all billed separately. Teams running dev, staging, and production environments on separate clusters pay this fee three times minimum. Additionally, if a Kubernetes version ages past its 14-month standard support window without being upgraded, the fee jumps to $0.60 per hour - a 6x increase that catches many teams by surprise.

Business implication: EKS is expensive at small scale and cost-efficient at large scale, where advanced resource scheduling and bin-packing justify the overhead. For small to medium workloads, ECS on Fargate will almost always be the cheaper option. The $73/month control plane fee is a fixed entry cost per cluster - for multi-cluster strategies, it adds up quickly.

Real-World Use Cases

Use EC2 when:

• You are migrating a legacy or monolithic application that was not built for containers
• Your application requires custom OS-level configuration, kernel parameters, or specific hardware access
• Your team needs full visibility and control over the underlying server environment
• You have predictable, steady-state workloads and want to maximise savings through Reserved Instances or Savings Plans

Business implication: EC2 is the right lift-and-shift vehicle. It minimises application changes during migration and gives your team time to modernise at their own pace. The trade-off is that it demands the most ongoing engineering attention.

Use ECS when:

• You want to run containers on AWS without learning Kubernetes
• You need fast, reliable deployments with minimal DevOps overhead
• Your workloads are bursty or variable and Fargate's pay-per-use model suits your traffic patterns
• Your team is AWS-first and wants tight, native integration with IAM, CloudWatch, and ALB

Business implication: ECS on Fargate is the fastest path to production for container workloads. It requires the least infrastructure expertise, no cluster maintenance, and scales automatically. It is a strong default choice for startups, product teams, and organisations without dedicated platform engineering.

Use EKS when:

• Your team already uses Kubernetes and has the expertise to operate it
• You need multi-cloud portability - the ability to run the same workloads on AWS, GCP, or Azure
• You are running complex microservices that benefit from Kubernetes-native features like Horizontal Pod Autoscaler, custom scheduling, or service meshes
• You are operating at a scale where advanced resource optimisation (bin-packing, Spot node groups, Karpenter) delivers meaningful cost savings

Business implication: EKS is a long-term platform investment. It requires Kubernetes expertise to operate well - either in-house or through a managed services partner. The payoff is flexibility, portability, and the ability to run sophisticated workloads that outgrow what ECS can offer.

Quick Decision Guide

Two questions cut through most of the noise:

1. Do you need Kubernetes?
   • Yes - your team uses it already, or you need multi-cloud portability → EKS
   • No → Move to question 2
2. Do you want containers?
   • Yes → ECS
   • No, or you have a legacy app to migrate → EC2

Situation	Recommended Service	Why
Small team, new to containers	ECS (Fargate)	Lowest ops overhead, no cluster to manage
Legacy app migration	EC2	Minimal app changes, full control
Startup scaling fast on AWS	ECS (Fargate)	Fast deployments, pay-as-you-go, AWS-native
Enterprise microservices	EKS	Advanced orchestration, multi-team platform
Existing Kubernetes users	EKS	Familiar tooling, avoid re-platforming cost
Multi-cloud strategy	EKS	Kubernetes runs anywhere - avoids AWS lock-in
Predictable, high-volume compute	EC2 (Reserved / Savings Plan)	Up to 72% savings over On-Demand with commitment

Conclusion

EC2, ECS, and EKS are tools for different jobs at different stages of growth. The right choice depends on your team's skills, your application's architecture, your traffic patterns, and your budget - both the AWS bill and the engineering time to manage it.

• Want simplicity and speed? → ECS on Fargate
• Have a legacy app to migrate? → EC2
• Need Kubernetes power and portability? → EKS

ECS on Fargate handles a huge range of production workloads reliably and cost-effectively - and it is far easier to migrate to EKS later than to unwind unnecessary Kubernetes complexity from day one.

If you have any questions, you can reach out to our AWS Cloud Consulting team here.

How to Update and Retrieve Secrets from Azure Key Vault Using the REST API

Introduction

Azure Key Vault is a cloud service that provides a secure and centralized way to store and manage secrets, keys, and certificates used by applications and services. It helps teams avoid hardcoding sensitive values like API keys, connection strings, or passwords directly into code or configuration files.

In this guide, you will learn how to update and retrieve secrets from Azure Key Vault using the REST API - a useful approach for automation scripts, CI/CD pipelines, and external integrations where using an SDK is not preferred or available.

Prerequisites

• An Azure Key Vault - if you don't already have one, create it from the Azure portal.
• At least one secret inside the Key Vault - click Generate/Import inside the vault to create your first secret.

Enable Azure RBAC on the Key Vault (Required)

• Azure Key Vault supports two permission models: Vault Access Policy (legacy) and Azure RBAC. To use IAM role assignments (like Key Vault Secrets Officer), your Key Vault must have Azure RBAC enabled. Without this, role assignments won't grant access to secrets.
• For a new Key Vault: During creation, go to the Access configuration tab and under Permission model, select Azure role-based access control (RBAC).

For an existing Key Vault:

1. Open your Key Vault in the Azure Portal.
2. Go to Settings → Access configuration.
3. Under the Permission model, select Azure role-based access control.
4. Click Save.

Important: If you switch an existing Key Vault from Vault Access Policy to Azure RBAC, all previously configured access policies will stop working. Make sure you reassign equivalent Azure roles before or immediately after switching.

Create an Azure AD App Registration (Required)

• To access Key Vault through the REST API, you must authenticate with an Azure AD application.

Assign API Permissions

• Go to: API Permissions → Add Permission → Azure Key Vault → Delegated Permissions.
• Select: user_impersonation
• Then click Grant Admin consent.

Create a Client Secret

• In the App Registration:
• Go to Certificates & Secrets
• Click New client secret
• Copy the generated secret value (you will need it in API calls)

Copy the Client ID and Tenant ID

• From the Overview page of your App Registration, copy:
• Client ID (Application ID)
• Tenant ID (Directory ID)

Assign IAM Role on the Key Vault

• To allow the App Registration to get or update secrets, assign it one of the following roles:
• Key Vault Secrets Officer OR Key Vault Administrator
• Path: Key Vault → Access control (IAM) → Add Role Assignment
• Select the role and assign it to your App Registration.

Generate an Access Token

• Before calling the Key Vault REST API, you must generate an OAuth 2.0 access token.
• Method: POST
• URL: https://login.microsoftonline.com/{TenantId}/oauth2/v2.0/token
• Headers: Content-Type: application/x-www-form-urlencoded
• Body: client_id={ClientId}&scope=https://vault.azure.net/.default&client_secret={ClientSecret}&grant_type=client_credentials
• This returns an access_token used in all Key Vault requests.

Get Secret Value from Azure Key Vault

• Use this API to retrieve a secret.
• Method: GET
• URL: https://{Key_Vault_Name}.vault.azure.net/secrets/{Secret_Name}?api-version=2026-02-01
• Headers: Authorization: Bearer {Access_Token}
• Content-Type: application/json

Set or Update a Secret in Azure Key Vault

• Use the PUT Request to create or update a secret.
• Method: PUT
• URL: https://{Key_Vault_Name}.vault.azure.net/secrets/{Secret_Name}?api-version=2026-02-01
• Headers: Authorization: Bearer {Access_Token}
• Content-Type: application/json
• Body:
  • { "value": "{Value}", "tags": { "source": "Postman" }, "contentType": "text/plain" }

Conclusion

With these steps, you can easily authenticate through Azure AD, retrieve secrets, and update values in Azure Key Vault using REST API calls. This approach is beneficial for automation, CI/CD pipelines, and external integrations where SDKs are not preferred.

If you have any questions, you can reach out to our Azure Cloud Consulting team here.

Event-Driven Autoscaling on EKS with KEDA and Karpenter

Introduction

It's 2 a.m. Your e-commerce platform just hit the front page of a major news site. Your SQS order queue skyrockets from 200 to 200,000 messages in minutes. Pods are overwhelmed. Customers see errors. Your on-call engineer is manually scaling deployments.

This is exactly the failure mode that event-driven scaling prevents. By combining KEDA (Kubernetes Event-Driven Autoscaler) and Karpenter, you can build a platform that reacts to demand automatically - scaling pods and nodes in seconds, then returning to zero when load disappears - all without human intervention.

Why Kubernetes HPA Falls Short for Event-Driven Workloads

Kubernetes' built-in Horizontal Pod Autoscaler (HPA) is often configured with CPU and memory metrics. While HPA does technically support custom and external metrics through the Kubernetes metrics API, wiring it up to event sources like SQS queue depth requires additional metrics adapters and careful configuration. Even then, a deeper problem remains: HPA reacts to metrics after the fact. For event-driven workloads, this creates a dangerous lag:

• An SQS queue floods with messages; pods start struggling
• CPU climbs - HPA detects the spike after a 15s scrape and sync cycle
• New pods are scheduled, but nodes are full - they sit Pending
• Cluster Autoscaler requests EC2 nodes - taking 3–5 minutes to arrive
• By then, the queue has grown by tens of thousands of messages

KEDA: Scale Pods on Real Events

KEDA is a CNCF Graduated project that extends Kubernetes to scale workloads based on external event sources - SQS, Kafka, Prometheus, DynamoDB Streams, and 70+ built-in scalers. It installs as a lightweight operator and works alongside your existing setup, connecting workloads directly to event sources without requiring custom metrics adapters.

KEDA introduces two core resources:

• ScaledObject - scales long-running Deployments/StatefulSets (APIs, background workers)
• ScaledJob - spawns individual Kubernetes Jobs per event (ETL, ML inference, video transcoding)

Installing KEDA via Helm

helm repo add kedacore https://kedacore.github.io/charts
helm repo update
helm install keda kedacore/keda \
  --namespace keda --create-namespace

Configuring AWS Authentication via TriggerAuthentication

The recommended approach for AWS authentication is a TriggerAuthentication resource using EKS Pod Identity or IRSA. The older identityOwner field on the scaler itself was deprecated in KEDA v2.13 and will be removed in v3 - avoid teaching or using it in new deployments.

First, create a TriggerAuthentication that references your KEDA operator's IAM role via pod identity:

apiVersion: keda.sh/v1alpha1
kind: TriggerAuthentication
metadata:
  name: keda-aws-auth
  namespace: order-processing
spec:
  podIdentity:
    provider: aws               # Uses EKS Pod Identity (recommended)
    # provider: aws-eks         # Use this if still on IRSA

ScaledObject Example: SQS Queue Depth

This scales an order-processing Deployment to maintain 10 messages per pod. With 500 messages, KEDA targets 50 pods - capped at maxReplicaCount.

Production note on in-flight messages: By default, KEDA's SQS scaler counts both ApproximateNumberOfMessages (queued) and ApproximateNumberOfMessagesNotVisible (in-flight / being processed). This means pods processing messages are included in the scaling calculation, which is usually the right behaviour. However, if your workers have long processing times or you see unexpected scale-down events mid-processing, tune scaleOnInFlight, your SQS visibility timeout, and your worker shutdown handling carefully - and ensure a Dead Letter Queue is configured to catch messages that fail repeatedly.

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: order-processor-scaledobject
spec:
  scaleTargetRef:
    name: order-processor
  pollingInterval: 15       # Check queue every 15s
  cooldownPeriod: 60        # Wait 60s before scaling down
  minReplicaCount: 0        # Scale to zero when idle
  maxReplicaCount: 100
  triggers:
  - type: aws-sqs-queue
    authenticationRef:
      name: keda-aws-auth   # References TriggerAuthentication above
    metadata:
      queueURL: https://sqs.us-east-1.amazonaws.com/123456/orders
      queueLength: '10'     # Target messages-per-pod ratio
      awsRegion: us-east-1
      scaleOnInFlight: 'true'   # Default: true. Set false to exclude in-flight messages

Karpenter: Right-Sized Nodes, Right Now

When KEDA scales your pods, those pods need nodes to land on. Karpenter watches for Pending pods, then automatically provisions the optimal EC2 instance type to satisfy them - typically in under 60 seconds. It also continuously bin-packs workloads and terminates underutilized nodes.

Karpenter vs. Cluster Autoscaler

Feature	Cluster Autoscaler	Karpenter
Provisioning Speed	3–5+ minutes	Typically 30–60 seconds
Instance Selection	Pre-configured ASG groups	Dynamic - picks optimal type per workload
Spot Support	Manual node group setup	Native, single NodePool
Node Consolidation	Limited	Automatic bin-packing

NodePool Configuration

The NodePool resource defines what Karpenter is allowed to provision. The example below targets the stable karpenter.sh/v1 API (available from Karpenter v1.0+) and configures a mixed Spot/On-Demand pool for batch workloads. Note that in v1, the consolidation policy is named WhenEmptyOrUnderutilized (renamed from WhenUnderutilized in v1beta1), and consolidateAfter is now supported alongside it and is required.

apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: batch-workers
spec:
  template:
    spec:
      nodeClassRef:
        group: karpenter.k8s.aws
        kind: EC2NodeClass
        name: batch-workers
      requirements:
      - key: karpenter.sh/capacity-type
        operator: In
        values: ['spot', 'on-demand']    # Spot-first, On-Demand fallback
      - key: node.kubernetes.io/instance-category
        operator: In
        values: ['c', 'm', 'r']          # General, compute, memory families
      - key: kubernetes.io/arch
        operator: In
        values: ['amd64', 'arm64']       # Support Graviton for savings
  limits:
    cpu: 1000                            # Safety cap on total cluster cost
  disruption:
    consolidationPolicy: WhenEmptyOrUnderutilized   # v1 name (was WhenUnderutilized in v1beta1)
    consolidateAfter: 30s                            # Required in v1; set to 0s for immediate

End-to-End Architecture Flow

When an SQS burst hits, the full scale-up sequence - from event arrival to active pod processing - completes in roughly one to two minutes in a well-tuned cluster. Actual time depends on image pull speed, node bootstrap, daemonset startup, and workload readiness. Here is the sequence:

01	Amazon SQS queue depth spikes (e.g., 200,000 messages)
02	KEDA polls queue every 15s, calculates required pod count, updates Deployment replica target
03	New pods are created - many land in Pending state (no capacity yet)
04	Karpenter detects Pending pods, selects optimal EC2 instance types, launches Spot nodes - typically in under 60s
05	Nodes join the cluster; pods are scheduled and begin processing messages
06	Queue drains → KEDA scales pods to 0 → Karpenter terminates idle nodes → worker compute cost drops to zero

Production Best Practices

KEDA

• Always set maxReplicaCount to guard against runaway scaling from a misconfigured scaler
• Use cooldownPeriod: 60–120s to prevent scale-down thrashing near zero
• Authenticate via TriggerAuthentication with podIdentity.provider: aws - the identityOwner field on the scaler is deprecated since v2.13 and will be removed in KEDA v3
• Set scaleDown.stabilizationWindowSeconds to smooth out spiky workloads
• For SQS workers, configure visibility timeout, scaleOnInFlight, and graceful shutdown carefully - and always attach a Dead Letter Queue to catch failed messages
• Test scale-to-zero in staging - some apps have cold-start latency that affects first-message SLA

Karpenter

• Use the stable karpenter.sh/v1 API - v1beta1 is supported but planned for deprecation
• Use consolidationPolicy: WhenEmptyOrUnderutilized (the v1 name; WhenUnderutilized from v1beta1 is renamed)
• Specify multiple instance families (c, m, r) so Karpenter can find available Spot capacity
• Set consolidateAfter explicitly - it is required in v1 when using WhenEmptyOrUnderutilized; use 0s for the same behaviour as v1beta1
• Include arm64 (Graviton) in your NodePool - AWS Graviton instances cost up to 20% less per hour than comparable x86 instances, with equal or better performance for most cloud-native workloads
• Set cpu and memory limits on the NodePool as a hard cost cap
• Tag all EC2NodeClass nodes with environment, team, and cost-center for AWS Cost Explorer analysis

Observing the Stack in Production

With two autoscalers operating in tandem, visibility across KEDA, Karpenter, SQS, and EC2 simultaneously is what separates a smooth on-call experience from a painful one. When something goes wrong - pods not scaling, nodes not terminating, queue backing up - you need correlated signals from all layers at once.

• Expose KEDA's /metrics endpoint to Prometheus - scaler values, replica counts, and error rates are all there
• Use CloudWatch Container Insights for correlated node + pod metrics
• Alert on SQS ApproximateAgeOfOldestMessage to catch backlogs before they compound
• Dashboard pod count (KEDA) and node count (Karpenter) together - a node spike without pods often means a misconfigured NodePool
• Monitor SQS NumberOfMessagesMoved on your Dead Letter Queue - a rising DLQ count signals worker failures that scaling alone cannot fix

Conclusion

KEDA and Karpenter together eliminate the manual scaling work that falls on your on-call engineer at the worst possible moment - scaling pods from real event signals, provisioning the right nodes in seconds, and returning to zero when load clears. Getting the details right (authentication, API versions, SQS in-flight behaviour, consolidation policy) is what makes this stack hold up under pressure in production.

If you have any questions or need help implementing this on your platform, you can reach out to our DevOps & Cloud Engineering team here.

SonarCloud “Supported Node.js Version Required” Error in GitHub Actions CI [Issue Resolved]

Introduction

CI/CD pipelines are designed to provide confidence. When a pipeline fails, the assumption is that something is genuinely wrong - a test broke, a dependency failed, or a configuration is missing.

Occasionally, however, pipelines fail in ways that don’t immediately make sense.

In our case, SonarCloud began failing in a GitHub Actions CI pipeline even though Node.js was installed, all tests were passing, and the pipeline configuration appeared correct. At first glance, the failure looked like a tooling issue. In reality, it turned out to be a subtle runtime-ordering problem - the kind that’s easy to overlook and difficult to diagnose.

This article walks through what happened, why the issue was misleading, and how we fixed it cleanly without introducing hacks or brittle workarounds.

What is SonarCloud and Why We Use It

SonarCloud is a static code analysis service that continuously inspects code for bugs, security vulnerabilities, and maintainability issues. In our organisation, SonarCloud acts as a quality gate in the CI pipeline, helping ensure that new changes meet agreed-upon code quality standards before they are merged.

For JavaScript and TypeScript projects, SonarCloud performs deep source-code analysis using a JavaScript engine that depends on Node.js. This means that, even though SonarCloud is not building or running the application itself, it still requires a compatible Node.js runtime to analyse the code correctly.

In our CI setup, SonarCloud runs automatically as part of GitHub Actions on pull requests and feature branches. Any failure at this stage blocks the pipeline, making SonarCloud a critical part of our development workflow.

This dependency on Node.js - and when that runtime is active - is the key to understanding the issue we encountered.

CI/CD Setup Context

The issue occurred in a GitHub Actions–based CI pipeline used for a JavaScript/TypeScript application. The pipeline was designed to be straightforward and intentionally modular, with clear separation between development, testing, and quality checks.

At a high level, the CI workflow included:

• Code checkout
• Dependency installation using pnpm
• Linting
• Multiple test suites
• SonarCloud analysis as a quality gate

The project used Node.js 18 as the primary runtime for application development and testing. All test commands were explicitly run using this version, and they completed successfully on every execution. From an application perspective, the pipeline was healthy.

SonarCloud was configured to run after tests, analysing the same source code that had just passed validation. The expectation was simple: if the code compiled and tests passed under Node.js 18, static analysis should also succeed.

Importantly, this pipeline existed in the application repository itself. Infrastructure and deployment were handled in a separate repository, where this application was included as a Git submodule. SonarCloud analysis was intentionally scoped only to the application repository to keep CI responsibilities clearly separated.

From a configuration standpoint, everything appeared correct:

• Node.js was installed
• The correct version was specified
• SonarCloud was integrated using the official GitHub Action

Yet despite this, the SonarCloud step failed consistently. That contradiction - a clean pipeline up until the quality gate - is what made this issue both confusing and time-consuming to debug.

The Problem and Error Symptoms

The failure occurred only during the SonarCloud analysis step. Every stage before it - dependency installation, linting, and all test suites - completed successfully.

SonarCloud failed with errors related to its JavaScript analysis engine, including messages suggesting:

• Unsupported JavaScript features
• Failure to start the internal analysis process
• Requirement for a supported Node.js version

This was confusing because Node.js was clearly installed and working. The pipeline explicitly set up Node.js, node -v returned a valid version, and the same runtime had just executed all tests without issue.

From a debugging perspective, this created a contradiction:

• If Node.js were missing or broken, tests should have failed.
• If the Node version were incompatible, the pipeline should have failed earlier.

Instead, the failure surfaced only at the SonarCloud stage, with error messages pointing to Node.js but not clearly explaining the underlying problem. This ambiguity made it difficult to immediately determine whether the issue was related to SonarCloud, GitHub Actions, or the CI configuration.

Initial Assumptions and Why They Were Wrong

Given the error messages, the initial assumption was that the Node.js setup was incorrect. This was a reasonable conclusion - SonarCloud explicitly reported problems related to the Node runtime.

The first checks focused on confirming that Node.js was installed correctly:

• Verifying the Node version in the pipeline
• Ensuring Node was available in the PATH
• Confirming that tests were running with the expected runtime

All of these checks passed.

The next assumption was that SonarCloud itself might be misconfigured. Various common fixes were considered, including reinstalling Node.js, forcing a specific executable path, or adding additional environment variables to guide the scanner.

While these approaches seemed promising, they were ultimately addressing symptoms rather than the root cause. Node.js was not missing, and SonarCloud was not misconfigured in an obvious way.

The real problem was not whether Node.js was installed, but which version of Node.js was active at the exact moment SonarCloud started its analysis. This distinction was easy to overlook and was not immediately obvious from the error messages.

Root Cause Summary

The issue was caused by the order in which the CI pipeline steps were executed. Although Node.js 18 was installed and used successfully for running tests, the SonarCloud GitHub Action was triggered before the intended Node.js runtime was fully applied to the runner environment.

As a result, SonarCloud defaulted to a different Node.js version that lacked support for certain modern JavaScript features.

This mismatch led to misleading errors during the analysis phase, even though the application itself was functioning correctly under the expected runtime.

How We Fixed the Issue

To resolve the problem cleanly and ensure consistent runtime behaviour across the pipeline, we made some adjustments:

Instead of adding overrides, hardcoding paths, or introducing custom scanner settings, we only needed to ensure that the required Node.js version was active immediately before SonarCloud started.

In practice, this meant keeping Node.js 18 for dependency installation and test execution, and then switching to Node.js 20 just before running the SonarCloud scan.

By doing this, we allowed each stage of the pipeline to use the runtime it expected, without interfering with other steps.

# Run tests with Node 18

- name: Setup Node.js 18

  uses: actions/setup-node@v4

  with:

    node-version: 18

# ... tests run here ...

# Switch runtime right before SonarCloud

- name: Setup Node.js 20

  uses: actions/setup-node@v4

  with:

    node-version: 20

- name: SonarCloud Scan

  uses: SonarSource/sonarcloud-github-action@v2

Once these changes were applied, SonarCloud analysis ran successfully, and the CI pipeline stabilised.

Key Lessons Learned

• CI tools depend on the active runtime environment, not only on what is installed in the pipeline.
• The order of execution can significantly change how tools behave.
• Verifying the runtime in use at each stage is just as important as validating the configuration itself.
• When failures appear to be code-related, the root cause may actually be environmental.

If you have any questions or need help with CI/CD pipelines, DevOps automation, or cloud solutions, feel free to reach out to our team here.

PostgreSQL Major Version Upgrades on Azure: A Terraform-based Approach

Introduction

PostgreSQL 11 has reached its end of life, and Azure recommends upgrading to PostgreSQL 13 or later for enhanced security, improved performance, and long-term support. Unlike minor upgrades, Azure Database for PostgreSQL (Flexible Server) does not support in-place major version upgrades. This makes the upgrade process slightly non-trivial—especially when the server is provisioned using Terraform, and some environments use VNet integration.

In this blog, we’ll walk through:

• How Azure PostgreSQL upgrades work
• Why does Terraform recreate the server
• Multiple migration strategies
• The exact steps I followed to upgrade PostgreSQL 11 → 13 safely

Existing Setup

My environment had the following characteristics:

• Azure Database for PostgreSQL – Flexible Server
• PostgreSQL version: 11
• SKU: Burstable B1ms (1 vCore, 2 GiB RAM)
• Storage: 32 GiB
• Region: Central US
• Provisioned using Terraform
• Mixed environments: Some with public access, some with VNet integration
• Firewall rules restricted to specific IPs

Terraform snippet (simplified):

Important Reality: No In-Place Major Version Upgrade

This is the most critical thing to understand: Azure PostgreSQL Flexible Server does NOT support in-place major version upgrades.

That means:

• You cannot upgrade PostgreSQL 11 → 13 on the same server
• Changing version = "13" in Terraform:
• Deletes the existing PostgreSQL 11 server
• Creates a brand-new PostgreSQL 13 server
• All data is lost unless you migrate or restore it manually

 

Terraform makes this very clear: forces replacement. This is not really an upgrade — it’s a rebuild and a migration.

Why This Upgrade Looks Simple — and Why It Isn’t

At first glance, the upgrade appears trivial: version = "13" 

But behind this single line:

• Azure treats PostgreSQL major versions as immutable
• Terraform maps this to a ForceNew operation
• Automated backups are tied to the old server lifecycle
• Configuration and data do not carry over

What Actually Happens (Timeline)

Understanding the timeline helps avoid surprises:

T-0: PostgreSQL 11 running

• Applications connected
• Data live
• Automated backups available

T-1: Terraform version updated

• version = "11" → version = "13"
• Plan shows forces replacement

T-2: Terraform apply

• PostgreSQL 11 server is deleted
• Databases and backups disappear

T-3: PostgreSQL 13 server created

• Empty server
• Default parameters
• No firewall rules
• No databases

T-4: Manual restore

• Data restored
• Configuration reapplied
• Applications reconnect

Available Upgrade Approaches

1. Azure Database Migration Service (DMS)

2. Backup & Restore (pg_dump / pgAdmin)

3. Temporary Public Access

Here we focus on Option 3, which was simple, cost-effective, and acceptable for my downtime window.

Step 1: Take a Backup

I used pgAdmin 4 with a custom format backup.

Why Custom format?

• Includes schema + data
• Best compatibility across versions
• Works cleanly with pg_restore

pg_dump `

  -h myserver.postgres.database.azure.com `

  -U pgsqladmin@myserver `

  -d master_data_service `

  -Fc `

  --sslmode=require `

  -f master_data_service_v11.dump

Step 2: Upgrade PostgreSQL Version via Terraform

In Terraform, change the code: version = "13"
Important: This destroys the PostgreSQL 11 server and creates a new PostgreSQL 13 server with the same name.
Run:
terraform plan
terraform apply

This immediately destroys the PostgreSQL 11 server and creates a new PostgreSQL 13 server with the same name.

Step 3: Restore the Database to PostgreSQL 13

pg_restore `

  -h myserver.postgres.database.azure.com `

  -U pgsqladmin@myserver `

  -d postgres `

  --create `

  -Fc `

  --sslmode=require `

  master_data_service_v11.dump

This:

• Recreated the database
• Restored schema and data
• Worked cleanly from v11 → v13

Step 4: Server Parameters & Configuration
Azure applies default server parameters when a new PostgreSQL server is created.

Key learning:

• Server parameters are NOT automatically migrated
• If you changed parameters manually in the portal, you must reapply them

Step 5: VNet-Integrated Environments

For servers with VNet integration:

• No public endpoint exists
• Local pgAdmin / pg_dump won’t connect

Available options:

• Use Azure DMS inside the VNet
• Use a VM or jumpbox
• Temporarily enable public access

We temporarily enabled public access with strict /32 firewall rules and disabled it immediately after migration.

Step 6: Validate & Cutover

After restoring:

• Verified tables, row counts, and extensions
• Tested application connectivity
• Updated connection strings where required
• Disabled public access again for private environments

Cost Considerations

• PostgreSQL B1ms server: ~$25/month
• Temporary overlap or migration time: a few dollars
• Azure DMS (Standard): Often free for migration scenarios
• Overall upgrade cost: minimal

Key Takeaways

• Azure PostgreSQL major upgrades are not in-place
• Terraform recreates the server when version changes
• Always backup before upgrading
• Server parameters must be reapplied
• For VNet setups, plan connectivity carefully
• PostgreSQL supports direct jump from 11 → 13

Final Thoughts

Upgrading PostgreSQL on Azure requires careful planning, but with the right approach, it can be a predictable and safe process.

If you’re using Terraform:

• Treat major version upgrades as rebuild + restore
• Automate as much as possible
• Test in lower environments first