At Portiva we like to use ShareGate for migrating SharePoint data. ShareGate offers a clear and user-friendly application to kick off a migration. But what if you have so many sites to migrate that manually starting migrations takes a long time? In this blog, you will learn how to easily automate your migration jobs with the ShareGate Powershell module and Windows Task Scheduler.
ShareGate and the PowerShell CLI
Before we can get started creating the necessary code, we need a few things:
ShareGate PS Module
To create a PowerShell script that can use ShareGate, the ShareGate PowerShell module must first be imported. Then three connections are created; 2 connections are needed to actually migrate data from the old environment (connection 1) to the new environment (connection 2), a third connection is needed to keep the SharePoint list up-to-date with the migrated items statuses. To keep the script as clean as possible, the credentials of these connections have been created in the file "variables.ps1".
We then retrieve the ongoing tasks that ShareGate is currently still working on. This allows us to start a maximum number of tasks and minimizes the risk of throttling (https://support-desktop.sharegate.com/hc/en-us/articles/360000437006-Office-365-Throttling). The maximum number of migrations we can start is therefore "maximum running tasks minus current tasks".
Maximum running tasks
If fewer than the maximum number of tasks are running, we retrieve the maximum number of migration items from the migration list using the "Connect-PnPOnline" and "Get-PnPMigrationItem". Then we iterate through the list of migration items and set some variables that we assign to the desired migration actions. In our example, this is mass migration of a site, starting a delta migration, or copying a Team.
The migration script connects to the old and new environment and starts the ShareGate task. Using the Set-PnPListItem, we save the new status of the migration item.
Automated ShareGate migrations
When we run the script in PowerShell, the maximum allowed tasks will start running. However, this will not initiate the next migrations by itself. To start the script automatically, in timed intervals, we use the Windows Task Scheduler application. Create a new task and set the desired interval for checking running tasks.
Create a new action that runs the Start-Migration PowerShell script.
Use the path to PowerShell.exe, for example:
Under ‘Add Arguments’ fill in the following:
-noprofile -executionpolicy unrestricted -noninteractive -File "Path_To_PSFile"
This ensures that PowerShell runs in the background and does not request permission to run certain commands.
Save the job and wait for the next timed trigger. And voila; ShareGate will now be controlled every x minutes using the Task Scheduler and the PowerShell scripts.
To make sure the PowerShell code isn’t terminated until all jobs are done, the following code is added in the Start-Migration script. Running the migration script without this can make PowerShell close itself before all background jobs are finished.
With this demo script it is easy to start your automated migration script. It just lacks one important feature: proper logging. The Start-Transcript does allow us to see some of the logging, but it is advised to create a logging system to log all possible errors, enable try-catching, and store the logging in a convenient place.