· 5 min read
Migrating Azure AI Search
Every now and then you hit one of those engineering chores we all politely call “straightforward". REcently I had to migrate an Azure AI Search instance from one tenant to another. It sounds simple enough, only to encounter the fact that it is not supported out of the box, so with the help of some AI-first, or vibecoding I ended up scripting it myself.
Turns out there is four steps, assuming you have an AI Search Instance already set up in the target tenant:
- Export the index schema from the source
- Export the documents from the source
- Import the index schema
- Import the documents into the target
I opted for not doing anything with the schema as those came from a BICEP template so creating those was no issue. The documents however needed to be exported and imported. For this I used PowerShell and the Azure AI Search REST API.
Make sure to run az login first to authenticate, and then execute the following script. The script exports documents in batches of 1000 (configurable) and saves them as JSON files in an output directory, so depending on the amount of documents you get a bunch of json files. The JSON contains all necessary metadata to import them back into another Azure AI Search instance. If you have a large index with many columns you might want’t to change the page size to a lower number to avoid hitting huge files.
Don’t forget to update the variables at the top of the script to match your environment.
$service = "some-azure-ai-search-service"
$index = "my-index-name"
$apiVersion = "2024-07-01"
$outputDir = "./export"
$apiKey = "my-admin-api-key"
$pageSize = 1000
$searchQuery = "*"
# Create output directory
if (Test-Path $outputDir) {
Remove-Item $outputDir -Recurse -Force
}
New-Item -ItemType Directory -Path $outputDir | Out-Null
$initialUrl = "https://$service.search.windows.net/indexes/$index/docs/search?api-version=$apiVersion"
function Invoke-SearchRequest {
param (
[string]$Url,
[int]$Skip,
[int]$Top,
[string]$Query
)
$headers = @{
"api-key" = $apiKey
"Content-Type" = "application/json"
}
$body = @{
search = $Query
top = $Top
}
if ($Skip -gt 0) {
$body.skip = $Skip
}
$bodyJson = $body | ConvertTo-Json -Compress
Invoke-RestMethod -Method Post -Uri $Url -Headers $headers -Body $bodyJson
}
# Function to save batch
function Save-Batch($docs, $batchNr, $outputDir) {
$outFile = Join-Path $outputDir ("batch-{0:D3}.json" -f $batchNr)
$wrapped = @{
value = @()
}
foreach ($doc in $docs) {
# Add @search.action for import
$doc | Add-Member -NotePropertyName "@search.action" -NotePropertyValue "upload"
$wrapped.value += $doc
}
$wrapped | ConvertTo-Json -Depth 100 | Out-File $outFile -Encoding utf8
Write-Host "Saved $outFile"
}
# ===========================================
# PAGINATION LOOP
# ===========================================
$batchNr = 0
$skip = 0
while ($true) {
$batchNr++
Write-Host "Fetching batch $batchNr..."
$json = Invoke-SearchRequest -Url $initialUrl -Skip $skip -Top $pageSize -Query $searchQuery
if (-not $json.value -or $json.value.Count -eq 0) {
Write-Host "No more documents returned."
break
}
Save-Batch $json.value $batchNr $outputDir
if ($json.value.Count -lt $pageSize) {
break
}
$skip += $pageSize
}
Write-Host "Export complete. Batches saved in folder: $outputDir"
Once exported you can use a similar approach to import the documents into the target Azure AI Search instance. Just make sure to adjust the target service name, index name, and API key accordingly, and if the Search Index lives in another tenant make sure to rerun az login to authenticate against the correct tenant.
This time the script will pick up all JSON files in the specified input directory and upload them one by one to the target Azure AI Search instance.
$service = "some-azure-ai-search-service"
$index = "my-index-name"
$apiVersion = "2024-07-01"
$inputDir = "./export"
$apiKey = "my-admin-api-key"
if (-not (Test-Path $inputDir)) {
throw "Input directory '$inputDir' not found. Run export first or adjust the path."
}
$files = Get-ChildItem -Path $inputDir -Filter "*.json" | Sort-Object Name
if (-not $files -or $files.Count -eq 0) {
throw "No JSON batch files found in '$inputDir'."
}
$endpointUrl = "https://$service.search.windows.net/indexes/$index/docs/index?api-version=$apiVersion"
function Invoke-ImportRequest {
param (
[string]$Payload
)
$headers = @{
"api-key" = $apiKey
"Content-Type" = "application/json"
}
Invoke-RestMethod -Method Post -Uri $endpointUrl -Headers $headers -Body $Payload
}
foreach ($file in $files) {
Write-Host "Uploading $($file.Name)..."
$payloadObject = Get-Content -Raw $file.FullName | ConvertFrom-Json
$payload = $payloadObject | ConvertTo-Json -Depth 100
Invoke-ImportRequest -Payload $payload
Write-Host "$($file.Name) uploaded."
}
Write-Host "All batches imported successfully."
After importing it might take a few minutes to reflect all changes, depending on the size of your index and the number of documents. Once done, your Azure AI Search instance in the target tenant should be fully populated with the documents from the source tenant. In our case when migrating around 16k documents it took 2 minutes for everything to show up correctly.
Final thoughts
Two final tips when building or testing the scripts;
- Validate your index schema before importing, if you are not using BICEP or ARM templates to create the index.
- Consider batch sizes. Too large and you might hit file size limits or timeouts, too small and the process takes longer.
- Rotate your keys after the migration to ensure security.
This workflow is a perfect example of where strategy and hands-on pragmatism meet, I speced out the process, asked some AI assistants for help, and ended up with a almost working solution 😉. The biggest issue AI had was that it called some unsupported REST endpoints, something you can quickly fix by adding the Microsoft Learn MCP server. By doing that it will figure out the latest documentation and the script results was way better. A few more manual edits and I was off happily migrating my items!
Albert-Jan Schot
CTO, Microsoft MVP & FastTrack Recognized Solution Architect
I am Albert-Jan Schot, CTO at Blis Digital, Microsoft MVP, and FastTrack Recognized Solution Architect focused on Microsoft 365, Azure, and AI agents. I help teams turn complex Microsoft Cloud challenges into practical architecture decisions and shipped outcomes.
Zuid Holland, Netherlands
