Showing posts with label Azure. Show all posts
Showing posts with label Azure. Show all posts

May 7, 2026

How to Update and Retrieve Secrets from Azure Key Vault Using the REST API

Introduction

Azure Key Vault is a cloud service that provides a secure and centralized way to store and manage secrets, keys, and certificates used by applications and services. It helps teams avoid hardcoding sensitive values like API keys, connection strings, or passwords directly into code or configuration files.

In this guide, you will learn how to update and retrieve secrets from Azure Key Vault using the REST API - a useful approach for automation scripts, CI/CD pipelines, and external integrations where using an SDK is not preferred or available.

Prerequisites

  • An Azure Key Vault - if you don't already have one, create it from the Azure portal.
  • At least one secret inside the Key Vault - click Generate/Import inside the vault to create your first secret.

Enable Azure RBAC on the Key Vault (Required)

  • Azure Key Vault supports two permission models: Vault Access Policy (legacy) and Azure RBAC. To use IAM role assignments (like Key Vault Secrets Officer), your Key Vault must have Azure RBAC enabled. Without this, role assignments won't grant access to secrets.
  • For a new Key Vault: During creation, go to the Access configuration tab and under Permission model, select Azure role-based access control (RBAC).

For an existing Key Vault:

  1. Open your Key Vault in the Azure Portal.
  2. Go to Settings → Access configuration.
  3. Under the Permission model, select Azure role-based access control.
  4. Click Save.

Important: If you switch an existing Key Vault from Vault Access Policy to Azure RBAC, all previously configured access policies will stop working. Make sure you reassign equivalent Azure roles before or immediately after switching.

Create an Azure AD App Registration (Required)

  • To access Key Vault through the REST API, you must authenticate with an Azure AD application.

Assign API Permissions

  • Go to: API Permissions → Add Permission → Azure Key Vault → Delegated Permissions.
  • Select: user_impersonation
  • Then click Grant Admin consent.

Create a Client Secret

  • In the App Registration:
  • Go to Certificates & Secrets
  • Click New client secret
  • Copy the generated secret value (you will need it in API calls)

Copy the Client ID and Tenant ID

  • From the Overview page of your App Registration, copy:
  • Client ID (Application ID)
  • Tenant ID (Directory ID)

Assign IAM Role on the Key Vault

  • To allow the App Registration to get or update secrets, assign it one of the following roles:
  • Key Vault Secrets Officer OR Key Vault Administrator
  • Path: Key Vault → Access control (IAM) → Add Role Assignment
  • Select the role and assign it to your App Registration.

Generate an Access Token

  • Before calling the Key Vault REST API, you must generate an OAuth 2.0 access token.
  • Method: POST
  • URL: https://login.microsoftonline.com/{TenantId}/oauth2/v2.0/token
  • Headers: Content-Type: application/x-www-form-urlencoded
  • Body: client_id={ClientId}&scope=https://vault.azure.net/.default&client_secret={ClientSecret}&grant_type=client_credentials
  • This returns an access_token used in all Key Vault requests.

Get Secret Value from Azure Key Vault

Set or Update a Secret in Azure Key Vault

Conclusion

With these steps, you can easily authenticate through Azure AD, retrieve secrets, and update values in Azure Key Vault using REST API calls. This approach is beneficial for automation, CI/CD pipelines, and external integrations where SDKs are not preferred.

If you have any questions, you can reach out to our Azure Cloud Consulting team here.

January 9, 2026

PostgreSQL Major Version Upgrades on Azure: A Terraform-based Approach

Introduction

PostgreSQL 11 has reached its end of life, and Azure recommends upgrading to PostgreSQL 13 or later for enhanced security, improved performance, and long-term support. Unlike minor upgrades, Azure Database for PostgreSQL (Flexible Server) does not support in-place major version upgrades. This makes the upgrade process slightly non-trivial—especially when the server is provisioned using Terraform, and some environments use VNet integration.

In this blog, we’ll walk through:
  • How Azure PostgreSQL upgrades work
  • Why does Terraform recreate the server
  • Multiple migration strategies
  • The exact steps I followed to upgrade PostgreSQL 11 → 13 safely


Existing Setup

My environment had the following characteristics:

  • Azure Database for PostgreSQL – Flexible Server
  • PostgreSQL version: 11
  • SKU: Burstable B1ms (1 vCore, 2 GiB RAM)
  • Storage: 32 GiB
  • Region: Central US
  • Provisioned using Terraform
  • Mixed environments: Some with public access, some with VNet integration
  • Firewall rules restricted to specific IPs

Terraform snippet (simplified):



Important Reality: No In-Place Major Version Upgrade

This is the most critical thing to understand: Azure PostgreSQL Flexible Server does NOT support in-place major version upgrades.

That means:
  • You cannot upgrade PostgreSQL 11 → 13 on the same server
  • Changing version = "13" in Terraform:
  • Deletes the existing PostgreSQL 11 server
  • Creates a brand-new PostgreSQL 13 server
  • All data is lost unless you migrate or restore it manually

 

Terraform makes this very clear: forces replacement. This is not really an upgrade — it’s a rebuild and a migration.

Why This Upgrade Looks Simple — and Why It Isn’t

At first glance, the upgrade appears trivial: version = "13" 

But behind this single line:
  • Azure treats PostgreSQL major versions as immutable
  • Terraform maps this to a ForceNew operation
  • Automated backups are tied to the old server lifecycle
  • Configuration and data do not carry over


What Actually Happens (Timeline)

Understanding the timeline helps avoid surprises:

T-0: PostgreSQL 11 running

  • Applications connected
  • Data live
  • Automated backups available


T-1: Terraform version updated

  • version = "11" → version = "13"
  • Plan shows forces replacement


T-2: Terraform apply

  • PostgreSQL 11 server is deleted
  • Databases and backups disappear


T-3: PostgreSQL 13 server created

  • Empty server
  • Default parameters
  • No firewall rules
  • No databases


T-4: Manual restore

  • Data restored
  • Configuration reapplied
  • Applications reconnect


Available Upgrade Approaches

1. Azure Database Migration Service (DMS)
2. Backup & Restore (pg_dump / pgAdmin)
3. Temporary Public Access

Here we focus on Option 3, which was simple, cost-effective, and acceptable for my downtime window.

Step 1: Take a Backup

I used pgAdmin 4 with a custom format backup.

Why Custom format?
  • Includes schema + data
  • Best compatibility across versions
  • Works cleanly with pg_restore
pg_dump `
  -h myserver.postgres.database.azure.com `
  -U pgsqladmin@myserver `
  -d master_data_service `
  -Fc `
  --sslmode=require `
  -f master_data_service_v11.dump

Step 2: Upgrade PostgreSQL Version via Terraform

In Terraform, change the code: version = "13"
Important: This destroys the PostgreSQL 11 server and creates a new PostgreSQL 13 server with the same name.
Run:
terraform plan
terraform apply

This immediately destroys the PostgreSQL 11 server and creates a new PostgreSQL 13 server with the same name.

Step 3: Restore the Database to PostgreSQL 13

pg_restore `
  -h myserver.postgres.database.azure.com `
  -U pgsqladmin@myserver `
  -d postgres `
  --create `
  -Fc `
  --sslmode=require `
  master_data_service_v11.dump

This:
  • Recreated the database
  • Restored schema and data
  • Worked cleanly from v11 → v13


Step 4: Server Parameters & Configuration
Azure applies default server parameters when a new PostgreSQL server is created.

Key learning:

  • Server parameters are NOT automatically migrated
  • If you changed parameters manually in the portal, you must reapply them


Step 5: VNet-Integrated Environments

For servers with VNet integration:
  • No public endpoint exists
  • Local pgAdmin / pg_dump won’t connect

Available options:
  • Use Azure DMS inside the VNet
  • Use a VM or jumpbox
  • Temporarily enable public access

We temporarily enabled public access with strict /32 firewall rules and disabled it immediately after migration.

Step 6: Validate & Cutover

After restoring:
  • Verified tables, row counts, and extensions
  • Tested application connectivity
  • Updated connection strings where required
  • Disabled public access again for private environments

Cost Considerations
  • PostgreSQL B1ms server: ~$25/month
  • Temporary overlap or migration time: a few dollars
  • Azure DMS (Standard): Often free for migration scenarios
  • Overall upgrade cost: minimal


Key Takeaways

  • Azure PostgreSQL major upgrades are not in-place
  • Terraform recreates the server when version changes
  • Always backup before upgrading
  • Server parameters must be reapplied
  • For VNet setups, plan connectivity carefully
  • PostgreSQL supports direct jump from 11 → 13


Final Thoughts

Upgrading PostgreSQL on Azure requires careful planning, but with the right approach, it can be a predictable and safe process.

If you’re using Terraform:

  • Treat major version upgrades as rebuild + restore
  • Automate as much as possible
  • Test in lower environments first

April 17, 2025

How to Integrate Azure OpenAI with SharePoint Library Data

Azure Open AI with SharePoint

We can index documents from a SharePoint library with Azure Cognitive Search, and use an Azure OpenAI model to query the data. Using a custom connector, we can bring this power into Power Automate.

Prerequisites

  • Have documents (*.docx, *.pdf) in a SharePoint library
  • Azure Cognitive Search service (Basic or Standard tier)
  • Azure OpenAI

Preparations

  • Note down the URL of the Site your library.
    • Example: https://***nexus.sharepoint.com/sites/Manish/AI_Documents
  • Note down the URL and the Admin Key of your Azure Cognitive Search Service











  • Turn on System-Assigned Managed Identity in your Azure Cognitive Search Service
  • Create an Entra Id app registration with the following parameters:

Connect your SharePoint library with Azure Cognitive Search

  • Create a Data Source with the Azure Cognitive Search Preview REST API
    • We have use the Automate flow to create a new data source for Azure Cognitive search using the REST API.
      • POST to https://(name-of-your Azure Cognitive Search service).search.windows.net/datasources?api-version=2024-06-01-Preview
      • Params:
        • api-version: 2024-06-01-Preview
      • Headers:
        • Content-Type: application/json
        • api-key: (the Admin key of your Azure Cognitive Search Service)
      • Body:
        					  
          {
              "name": "sharepoint-datasource",
              "type": "sharepoint",
              "credentials": {
                  "connectionString": "SharePointOnlineEndpoint=
                  (your SharePoint Site URL0);ApplicationId=(your_App_Id)"
              },
              "container": {
              "name": "defaultSiteLibrary",
              "query": null
              }
          }   
                         
        				





















































      • Below is the HTTP Action we have used for create a source.









































      • This will return a 201 response, indicating that your data source was created. You can check this in the Azure portal.

Create your Index

  • Let’s now leverage metadata of your document to enhance your search experience. This as well is done by using the REST API. We will again do this in automate flow.
  • POST to https://(name-of-your Azure Cognitive Search service).search.windows.net/indexes?api-version=2024-06-01-Preview.
  • Params:
    • api-version: 2024-06-01-Preview
  • Headers:
    • Content-Type: application/json
    • api-key: (the Admin key of your Azure Cognitive Search Service)






























































  • Body:
  • 					  
      {
      "name": "sharepoint-index",
      "fields": [
        {
          "name": "id",
          "type": "Edm.String",
          "key": true,
          "searchable": false
        },
        {
          "name": "metadata_spo_item_name",
          "type": "Edm.String",
          "key": false,
          "searchable": true,
          "filterable": false,
          "sortable": false,
          "facetable": false
        },
        {
          "name": "metadata_spo_item_path",
          "type": "Edm.String",
          "key": false,
          "searchable": false,
          "filterable": false,
          "sortable": false,
          "facetable": false
        },
        {
          "name": "metadata_spo_item_content_type",
          "type": "Edm.String",
          "key": false,
          "searchable": false,
          "filterable": true,
          "sortable": false,
          "facetable": true
        },
        {
          "name": "metadata_spo_item_last_modified",
          "type": "Edm.DateTimeOffset",
          "key": false,
          "searchable": false,
          "filterable": false,
          "sortable": true,
          "facetable": false
        },
        {
          "name": "metadata_spo_item_size",
          "type": "Edm.Int64",
          "key": false,
          "searchable": false,
          "filterable": false,
          "sortable": false,
          "facetable": false
        },
        {
          "name": "content",
          "type": "Edm.String",
          "searchable": true,
          "filterable": false,
          "sortable": false,
          "facetable": false
        }
      ]
    }
                     
    				
  • This will return a 201 response, indicating that your data source was created. You can check this in the Azure portal.

  • Create your indexer

    • We want to create an indexer. It will later automate the indexing process from your SharePoint library to the Azure Cognitive Search service.
    • Once again, we do this in automate flow. This is a two-step process as we first need to POST a Create an indexer request - which will run and run and run as it is waiting for us to log in. So we will run a second call, which to GET the indexer status. This will return a device code with which we can sign in - Once we did that we can see that the call returns a 200. After that, the POST will succeed and return a 201 as well.

    Create an indexer request

    • POST to https://(name-of-your Azure Cognitive Search service).search.windows.net/indexers?api-version=2024-06-01-Preview
    • Params:
      • api-version: 2024-06-01-Preview
    • Headers:
      • Content-Type: application/json
      • api-key: (the Admin key of your Azure Cognitive Search Service)
    • * Image *

    • Body:
    • 					  
      {
        "name": "sharepoint-indexer",
        "dataSourceName": "aidocumentssource",
        "targetIndexName": "sharepoint-index",
        "parameters": {
      	"batchSize": null,
      	"maxFailedItems": null,
      	"maxFailedItemsPerBatch": null,
      	"base64EncodeKeys": null,
      	"configuration": {
      	"indexedFileNameExtensions": ".pdf, .docx",
      	"excludedFileNameExtensions": ".png, .jpg",
      	"dataToExtract": "contentAndMetadata"
      	}
      },
      "schedule": {},
      "fieldMappings": [{
      	"sourceFieldName": "metadata_spo_site_library_item_id",
      	"targetFieldName": "id",
      	"mappingFunction": {
      	"name": "base64Encode"}
      }]
      }
                       
      				

    Get indexer status

    • Now, we have create a new flow for get a device login code.
      • GET to https://(name-of-your Azure Cognitive Search service).search.windows.net/indexers/sharepoint-indexer/status?api-version=2024-06-01-Preview
      • Params:
        • api-version: 2024-06-01-Preview
      • Headers:
        • Content-Type: application/json
        • api-key: (the Admin key of your Azure Cognitive Search Service)
      • This will return a response that contains an errormessage:
        • "errorMessage": "To sign in, use a web browser to open the page https://microsoft.com/devicelogin and enter the code LFXXXXXP to authenticate.\r\nTo sign in, use a web browser to open the page https://microsoft.com/devicelogin and enter the code LFXXXXXP to authenticate."
      • Copy the code and open the link, then paste the code into the device login popup. Once you are logged in, you can close that browser tab again.












































      • Check now in the Azure portal that you do not only have an index, but also an indexer and documents indexed.


    Test your app in the Playground

    The Azure Open AI playground is a fabulous way to test and try out - so let’s do this.

    • In the Playground, create a new deployment
    • Select Add your data and then Add a data source
    • Select the Azure Cognitive Search service, your Subscription its running in, and the index we just created. All of these will automagically appear in the respective dropdown fields.
    • Now proceed with the index data field mapping - where you select all fields to be content
    • Save and close

    You can now chat against your documents and ask the bot questions about it. By check/uncheck of the Limit responses to your data content you can determine whether you want the bot only to consider content from your documents or not. You can now deploy this as a web app - Or you can walk with me some more steps and have that power in Power Apps or Power Automate

    If you have any questions you can reach out our SharePoint Consulting team here.

    April 3, 2025

    Managing Multiple Azure Environments with Terraform

    Introduction

    Managing cloud infrastructure across multiple environments can be complex. Terraform simplifies this process using modules and workspaces, Allows us more efficient and scalable infrastructure management in any cloud. This guide explores leveraging Terraform modules in a multi-workspace setup for Microsoft Azure. 

    Benefits of Terraform Modules and Workspaces

    Terraform Modules: Enhancing Reusability

    Modules allow infrastructure components to be defined once and reused across different environments. This reduces redundancy and enhances maintainability.


    Terraform Workspaces: Isolating Environments

    Workspaces create separate states for different environments, ensuring isolation and preventing conflicts between deployments. Utilizing Terraform variables further refines environment-specific configurations.


    Structuring Terraform for Multi-Environment Deployment

    A well-structured Terraform directory simplifies management across environments. Below is a recommended directory structure:


    Directory Layout

    $ tree complete-module/
    .
    ├── README.md
    ├── main.tf
    ├── variables.tf
    ├── outputs.tf
    ├── ...
    ├── modules/
    │   ├── nestedA/
    │   │   ├── README.md
    │   │   ├── variables.tf
    │   │   ├── main.tf
    │   │   ├── outputs.tf
    │   ├── nestedB/
    │   ├── .../
    ├── examples/
    │   ├── exampleA/
    │   │   ├── main.tf
    │   ├── exampleB/
    │   ├── .../

    Creating a Reusable Terraform Module

    Defining a Virtual Network Module:

     - modules/network/main.tf
    resource "azurerm_virtual_network" "network" {
      name                = var.network_name
      location            = var.location
      resource_group_name = var.resource_group_name
      address_space       = var.address_space
    }

     - modules/network/variables.tf

    variable "network_name" {
      type = string
    }

    variable "location" {
      type = string
    }

    variable "resource_group_name" {
      type = string
    }

    variable "address_space" {
      type = list(string)
    }

     - modules/network/outputs.tf

    output "network_id" {
      value = azurerm_virtual_network.network.id
    }

    Utilizing the Module in the Main Configuration

    - main.tf

    terraform {
      required_providers {
        azurerm = {
          source  = "hashicorp/azurerm"
          version = "4.16.0"
        }
      }

      backend "azurerm" {
        resource_group_name  = "terraform-state-rg"
        storage_account_name = "terraformstate"
        container_name       = "tfstate"
        key                  = "terraform.tfstate"
      }
    }

    provider "azurerm" {
      features {}
    }

    module "network" {
      source              = "./modules/network"
      network_name        = "my-network-${terraform.workspace}"
      location            = "East US"
      resource_group_name = "my-rg"
      address_space       = ["10.0.0.0/16"]
    }

    Managing Workspaces for Different Environments

    Initializing and Creating Workspaces

    Run the following commands to initialize Terraform and create new workspaces:

    terraform init
    terraform workspace new development
    terraform workspace new staging
    terraform workspace new production

    Switch between workspaces:

    terraform workspace select development

    Applying Configuration to a Specific Workspace

    terraform apply -var-file=environments/development.tfvars

    Terraform plan output:

    Terraform used the selected providers to generate the following execution plan.
    Resource actions are indicated with the following symbols:
      + create

    Terraform will perform the following actions:

      # module.network.azurerm_virtual_network.network will be created
      + resource "azurerm_virtual_network" "network" {
          + address_space       = ["10.0.0.0/16"]
          + id                  = (known after apply)
          + location            = "East US"
          + name                = "my-network-default"
          + resource_group_name = "my-rg"
        }

    Plan: 1 to add, 0 to change, 0 to destroy.


    Terraform apply output:

    module.network.azurerm_virtual_network.network: Creating...
    module.network.azurerm_virtual_network.network: Creation complete after 30s ...

    Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

    Outputs:

    network_id = "/.../my-rg/.../Microsoft.Network/virtualNetworks/my-network-default"

    Advantages of This Approach

    • Code Efficiency: Reusable modules minimize code duplication.
    • Environment Segregation: Workspaces ensure different state for different environment.
    • Scalability: With this approach we can easily add multiple environments as needed.

    Reference links:


    Conclusion

    Using Terraform modules and workspaces in Azure streamlines environment management, improves reusability, and enhances scalability. This structured approach keeps infrastructure organized and adaptable to change.

    Happy Terraforming!

    March 21, 2025

    Setting Up a Local Kubernetes Cluster with Minikube.

    Kubernetes has become the go-to container orchestration platform for deploying, scaling, and managing applications. However, setting up a full-scale Kubernetes cluster can be complex, especially for local development. That’s where Minikube comes in! Minikube allows you to run a lightweight Kubernetes cluster locally, perfect for development and testing purposes.

    In this Blog, I’ll walk through the steps to set up a local Kubernetes cluster using Minikube, ensuring that you can start experimenting with Kubernetes in no time.


    What is Minikube?

    Minikube is a tool that sets up a single-node Kubernetes cluster on your local machine. It supports multiple container runtimes like Docker, containerd, and CRI-O, and it’s an excellent option for developers who want to test Kubernetes deployments before pushing them to production.


    Prerequisites

    Before we dive into the setup process, you’ll need:

    • A machine with at least 2 CPUs and 2GB of RAM
    • A hypervisor like VirtualBox or Hyper-V (if using Windows)
    • kubectl (Kubernetes CLI tool)
    • Minikube

    Step 1: Install Minikube

    First, you need to install Minikube on your machine. The installation process varies depending on your operating system. Follow these instructions based on your platform:

    For macOS (via Homebrew):

    brew install minikube

    For Linux (via curl):

    curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64
    sudo install minikube-linux-amd64 /usr/local/bin/minikube

    For Windows (via Chocolatey):

    choco install minikube





    Step 2: Install kubectl

    The Kubernetes command-line tool, kubectl, is essential for interacting with the cluster.

    For macOS (via Homebrew):

    brew install kubectl

    For Linux:

    sudo apt-get install -y kubectl

    For Windows (via Chocolatey):

    choco install kubernetes-cli









    Step 3: Start Minikube

    Once Minikube is installed, start it with the following command:

    minikube start

    This command automatically sets up a Kubernetes cluster using the hypervisor installed on your machine (VirtualBox, Hyper-V, Docker, etc.). You can also specify the driver using the --driver flag, like so:

    minikube start --driver="ANY REQUIRED"

    Note: By default, Minikube will use Docker as the container runtime. If you prefer containerd or CRI-O, you can specify it with the required flag





    Step 4: Verify the Setup

    After Minikube has started, you can verify that your cluster is running by checking the nodes in the cluster:

    kubectl get nodes






    Step 5: Deploy an Application on Minikube

    A deployment in Kubernetes is a higher-level abstraction that manages the rollout and scaling of applications. It defines how to create and update instances of the application (called pods) consistently across a cluster. Deployments ensure that the desired number of pod replicas are running, and they automatically handle updates, rollbacks, and scaling based on user-defined conditions.

    The main purpose of Deployments is because they are essential for handling production workloads and managing containerized apps in a reliable, automated way. This ensures high availability by running multiple instances of an application and scale the application dynamically in response to traffic or resource usage.

    Now that Minikube is running, let’s deploy a simple application. We’ll use a sample NGINX deployment to demonstrate.

    First, create a Kubernetes deployment:

    kubectl create deployment nginx --image=nginx

    Verify that the deployment has been created:

    kubectl get deployments





    Step 6: Expose the Application

    By default, the NGINX deployment is not accessible from outside the cluster. To expose it, we’ll create a service:

    kubectl expose deployment my-nginx --port=80






    Step 7: Access the Application

    Now that the service is exposed, you can access the NGINX web server using Minikube’s IP. To get the Minikube IP, run:

    minikube ip: 19X.XXX.XXX.XXX:PORT

    Combine this IP with the NodePort value from the previous step to access the application in your browser:







    Step 8: Stop the Cluster

    Once you’re done experimenting, you can stop the Minikube cluster using the following command:

    minikube stop

    If you want to delete the cluster entirely, run:

    minikube delete


    Conclusion

    Minikube is a fantastic tool for local Kubernetes development, offering a quick and easy way to spin up a local cluster. In this Blog, we went through the setup process, deployed a simple application, and exposed it for external access. Now you can start experimenting with Kubernetes features and workflows in a local environment before deploying them to a production environment.

    Start your Kubernetes journey with Minikube today, and happy developing!


    Reference Links: