In this blog article you can read about my recent experience using the Office 365 unencrypted PST file upload feature. The procedure that I followed is described in the following Microsoft TechNet Library article:
- Article Name: Import unencrypted PST files to Office 365
- URL: https://technet.microsoft.com/en-us/library/mt644809.aspx
- Topic Last Modified: 2016-02-25
We have a hybrid Exchange 2013 server environment. Until now almost all of our mailboxes have been on-premises. I had been trying to migrate my mailbox to Exchange Online over the last week using various methods. Before I began the PST upload/import procedure, I had a working mailbox in Office 365 containing about half of the content that I wanted to migrate.
Overall, I was satisfied with the whole upload and import process.
In my case, it took approximately 2 hours to upload a single 3.5 GB PST file to Microsoft Azure.
After the upload, the import process took roughly 4.5 hours to import 3.5 GB of data into a single Office 365 mailbox (approx. 0.77 GB/hour). Approximately 26,000 items were imported and 18,000 items were skipped because they were already in the mailbox. I don’t know what effect the skipped items had on the import performance.
Following is a walk-through of the procedure with comments and an example. I’ve also shown how to use the updated Microsoft Azure Storage Explorer tool instead of the tool listed in the TechNet article that is no longer actively maintained. The various steps that are shown below correspond to the steps listed in the TechNet article.
Step 0: Before you begin
This step contains instructions about how to assign the correct permissions to import PST files to Office 365 mailboxes. Since I was performing the procedure as an Office 365 global admin, I decided to assign the “Mailbox Import Export” role to the “Organization Management” role group in Exchange Online. Try not to skip this bit; in my case I “tested” the process by not performing this step until after I submitted the import job. The effect was that I received an error message saying that the import process would be delayed for an hour until after I had assigned the correct permissions. I was quite impressed that the process was robust enough to do this.
I recommend that you create a new folder in which you will store the PST file/s that are going to be uploaded. The reason for this is that the AzCopy program will upload all files in the source location (and optionally subfolders).
I created a new folder in which to store the PST file/s (e.g. “C:\Temp\O365PSTUpload“) and then shared it as follows:
- Share name: \\FS1\O365PSTUpload
- Share permission: I gave the Everyone group Read permissions to this share.
Step 1: Download the PST Import tools
The following figure shows the “Upload files over the network” window. I downloaded and installed the PST Import tools (i.e. AzCopy) by clicking on “Download tool“.
Step 2: Obtain the storage key and upload URL for PST Import
In this step you obtain the storage key and upload URL, then save them somewhere for future reference. In my case, I created a text file called “Office365PSTImportInfo-20160309.txt” and copied the key and URL into the file.
Tip: There is a section titled “More information” right at the end of the TechNet article that contains a good example of what the file should contain.
The following figure shows what happens when you click on “Show key“. The key should appear in the box within a few minutes.
Note: In case the key doesn’t appear in a few minutes, simply close the “Upload files over the network” window, go back to the “Import files to Office 365” window, click on “New job +“, and then click “Upload files over the network“. Then go to the start of Step 2. The storage key should only take a few seconds to display.
Step 3: Upload your PST files to Office 365
I recommend using the text file that you created in Step 2 to work out what the command will be before pasting it into the command prompt. This will also allow you to save a record of the command that you used for future reference.
Here is an example of what my file (Office365PSTImportInfo-20160309.txt) looked like:
Storage account key: yaNIIs9Uy5g25Yoak+LlSHfqVBGOeNwjqtBEBGqRMoidq6/e5k/VPkjOXdDIXJHxHvPoGoFH5NcVUJXHwu9ZxQ== Network upload URL: https://3c3e5952a2764023ad14984.blob.core.windows.net/ingestiondata/ AzCopy.exe /Source:<Location of PST files> /Dest:<Network upload URL + PST file path> /Destkey:<Storage account key> /V:<Log file location> EXAMPLE AzCopy.exe /Source:"\\FS1\O365PSTUpload" /Dest:"https://3c3e5952a2764023ad14984.blob.core.windows.net/ingestiondata/FS1/O365PSTUpload" /Destkey:"yaNIIs9Uy5g25Yoak+LlSHfqVBGOeNwjqtBEBGqRMoidq6/e5k/VPkjOXdDIXJHxHvPoGoFH5NcVUJXHwu9ZxQ==" /V:"C:\Temp\O365PSTUpload.log"
After you enter the command in a command prompt, the PST files that you specified will be uploaded to the Azure storage account. In my case the initial upload timed out. I seamlessly resumed the upload by simply running the same command again. The contents of my log file (O365PSTUpload.log) showing both upload runs was as follows:
[2016/03/09 15:18:26.818+02:00] >>>>>>>>>>>>>>>> [2016/03/09 15:18:26.880+02:00][VERBOSE] 5.0.0 : AzCopy /Source:\\FS1\O365PSTUpload /Dest:https://3c3e5952a2764023ad14984.blob.core.windows.net/ingestiondata/FS1/O365PSTUpload /Destkey:****** /V:C:\Temp\Office365PSTUpload.log [2016/03/09 15:18:30.354+02:00][VERBOSE] Start transfer: PLapornik-20160309.pst => PLapornik-20160309.pst [2016/03/09 17:14:14.400+02:00][VERBOSE] Transfer FAILED: PLapornik-20160309.pst => PLapornik-20160309.pst. [2016/03/09 17:14:14.400+02:00][ERROR] PLapornik-20160309.pst: The client could not finish the operation within specified timeout. The client could not finish the operation within specified timeout. [2016/03/09 17:14:14.447+02:00] Transfer summary: ----------------- Total files transferred: 1 Transfer successfully: 0 Transfer skipped: 0 Transfer failed: 1 Elapsed time: 00.01:55:47 [2016/03/09 17:49:22.211+02:00] >>>>>>>>>>>>>>>> [2016/03/09 17:49:22.227+02:00][VERBOSE] 5.0.0 : AzCopy /Source:\\FS1\O365PSTUpload /Dest:https://3c3e5952a2764023ad14984.blob.core.windows.net/ingestiondata/FS1/O365PSTUpload /Destkey:****** /V:C:\Temp\Office365PSTUpload.log [2016/03/09 17:49:36.137+02:00][VERBOSE] [PROMPT] Incomplete operation with same command line detected at the journal directory "C:\Users\Administrator\AppData\Local\Microsoft\Azure\AzCopy", do you want to resume the operation? Choose Yes to resume, choose No to overwrite the journal to start a new operation. (Yes/No) : Y [2016/03/09 17:49:37.053+02:00][VERBOSE] Start transfer: PLapornik-20160309.pst => PLapornik-20160309.pst [2016/03/09 18:06:38.929+02:00][VERBOSE] Finished transfer: PLapornik-20160309.pst => PLapornik-20160309.pst [2016/03/09 18:06:38.976+02:00] Transfer summary: ----------------- Total files transferred: 1 Transfer successfully: 1 Transfer skipped: 0 Transfer failed: 0 Elapsed time: 00.00:17:16
On my Internet connection, the transfer rate to upload the PST file using AzCopy was approximately 1.6 GB/hr.
Step 4: View a list of the PST files uploaded to Office 365
The TechNet Library article shows how to verify that the PST file got uploaded using the Azure Storage Explorer tool. This tool is not being actively maintained. Instead, I have shown how to do the same thing using the Microsoft Azure Storage Explorer tool (a different tool) that is being actively maintained by Microsoft. Microsoft Azure Storage Explorer is supported by Microsoft and can be downloaded from http://storageexplorer.com A direct link to download the tool is http://go.microsoft.com/fwlink/?LinkId=708343
After installing and launching the application, I closed the initial prompt that asks you for an account.
I right-clicked on “Storage Accounts” then clicked on “Attach External Storage…”
In the “Attach External Storage” window, I entered the following information:
- Account name: Use the numeric portion of the network upload URL that you obtained in Step 2. For example, if the network upload key is https://3c3e5952a2764023ad14984.blob.core.windows.net/ingestiondata/, you would use 3c3e5952a2764023ad14984 for the account name.
- Account key: Use the storage account key that you obtained in Step 2. For example, yaNIIs9Uy5g25Yoak+LlSHfqVBGOeNwjqtBEBGqRMoidq6/e5k/VPkjOXdDIXJHxHvPoGoFH5NcVUJXHwu9ZxQ==.
Finally, I browsed through the storage accounts hierarchy until I was able to confirm that the PST file had been uploaded.
For example, 3c3e5952a2764023ad14984 (external) –> Blob Containers –> ingestiondata –> FS1 –> O365PSTUpload
Here is a screenshot from my storage account (sensitive information has been obscured).
Step 5: Create the PST Import mapping file
The next step is to create a mapping file. The following is an example of the mapping file that I used for a single mailbox:
In my case I decided to import the PST file into the root folder of my mailbox. I did this because I wanted mail items to be imported back into their original places.
Step 6: Create a PST import job in Office 365
To create the import job you need to go back to the “Upload files over the network” screen, tick the options to indicate that you have uploaded the PST file/s and created a mapping file, then click on “Next“.
As shown in the example, you will now be prompted to specify a name for your import job. The name must consist of 2-64 lowercase letters, numbers, hyphens and underscores. It must start with a letter, No spaces are permitted. I used “importplapornikpst”.
The next step is to upload and verify the mapping file. As shown below, I selected my mapping file, clicked on “Validate“, agreed to the terms and conditions, then clicked on “Finish“.
Now that you have successfully submitted the import job, you can close the “Upload files over the network” window.
Viewing the status of the import job
If you go to the “Import data to Office 365” window, you can view the status of the import job. You can also click on “View details” for more detailed information.
The following screenshot shows an example of the amount of detail provided in the detailed view.
Post import report
The following figure provides an example of what you might see after the PST import has completed. In my case, the “Missing skipped item” column referred to emails that were skipped because they were already imported in my mailbox.
I was able to see more details about skipped items by clicking on “Download report“. This allowed me to cross-reference emails in my mailbox with the list of emails that were downloaded as a CSV file. A small sample from the report is shown below.
Subject,Created on,Folder,Skipped item reason,, SSO for ISA Server (DITS),03/10/2016 18:28,Sent Items,"Item is missing in a folder. Folder name: Sent Items, folder ID: 000000000D60CE1C58423E458AF905D442289DC962840000, item ID: 000000000D60CE1C58423E458AF905D442289DC9E44A2E00.",, Office 365 for Meice,03/10/2016 18:28,Sent Items,"Item is missing in a folder. Folder name: Sent Items, folder ID: 000000000D60CE1C58423E458AF905D442289DC962840000, item ID: 000000000D60CE1C58423E458AF905D442289DC9044B2E00.",, FW: Cytram,03/10/2016 18:28,Sent Items,"Item is missing in a folder. Folder name: Sent Items, folder ID: 000000000D60CE1C58423E458AF905D442289DC962840000, item ID: 000000000D60CE1C58423E458AF905D442289DC9244B2E00.",,
After you import information into a mailbox using the Office 365 unencrypted PST file upload feature, the mailbox is automatically placed on retention hold. This is not the same as as a litigation hold or in-place hold. A retention hold suspends the processing of any retention policies. Just remember to take the mailbox off retention hold to resume normal processing of any retention policies.