Introduction:
- In this blog, we will demonstrate how to upload large files to a SharePoint document library from the SPFx web part, even when the file size exceeds 100MB or even reaches the GB range.
- SharePoint provides us with the REST API for uploading files to the document library. The API Is {site_url}/_api/web/getfolderbyserverrelativeurl('/sites/{site_name}/{library_name}')/file s/add(overwrite=true,url='{file_name}'). However, the issue is that this API only allows us to upload files up to 2MB in size. Any files larger than 2MB cannot be uploaded using this API.
- Here I came up with a solution that allows us to upload files larger than 10 MB, going up to GBs, in SharePoint Document Library from the SPFx Web Part. To achieve this, we can use the chunk upload process. The SharePoint REST API provides methods Or query parameters for the chunk upload process, including “Start Upload”, “Continue Upload”, and “Finish Upload”. Using chunk upload, we can handle uploading any size of the file.
Function that handles the large upload in the SharePoint document library:
- Create a custom function to handle large file uploads in the library. This function requires parameters such as file data, filename, SharePoint site URL, document library name, user's digest value, and desired chunk size.
- At the beginning of the function, we need to declare some variables, such as the headers to be passed in our REST API, the starting point for uploading, and the endpoint, and so on.int, and so on.
- After that, we need to call another function to started the upload session for file uploading. In this function, we simply add a blank file to our document library to initialize our uploading session. From the API response, we receive the unique ID of the blank file, which helps us identify the file whose content we need to overwrite.
- After that, we need to generate a unique GUID that is used in our method for starting upload, continuing upload, and finishing upload.
- After that, we have to check the starting position of the file. According to that, we divide the file into chunks. If the condition is true, then we have to call the "Start Upload" method with a unique GUID, which we have generated, to begin uploading the first chunk of the file to the document library.
- After uploading the first chunk, we loop through every subsequent chunk of the file and call the "Continue Upload" REST API using the same GUID that we used in the "Start Upload" method to upload the chunks. We continue uploading the chunks until we reach the 2nd to last chunk.
- For the last step, we upload the last chunk of the file using the SharePoint REST API method called "Finish upload". to signal to SharePoint that this is the last chunk of the file, thus completing the uploading process.
private async UploadLargeFile(
file: Blob,
siteUrl: string,
libraryName: string,
fileName: string,
chunkSize: number,
digest: any
) {
const headers = {
"Accept": "application/json;odata=verbose",
"X-RequestDigest": digest
};
const fileSize = file.size;
const uploadId = this.GenrateUploadId();
let start = 0;
let end = chunkSize;
let chunkNumber = 0;
let fileId = "";
const uploadSessionResponse = await this.StartUploadSession(siteUrl, libraryName, fileName, headers);
fileId = uploadSessionResponse.d.UniqueId;
while (start < fileSize) {
const chunk = file.slice(start, end);
const isLastChunk = end >= fileSize;
if (chunkNumber === 0) {
await this.UploadFirstChunk(siteUrl, libraryName, fileName, chunk, uploadId, headers, fileId);
} else if (isLastChunk) {
await this.UploadLastChunk(siteUrl, libraryName, fileName, chunk, uploadId, headers, start, fileId);
} else {
await this.UploadIntermediateChunk(siteUrl, libraryName, fileName, chunk, uploadId, headers, start, fileId);
}
start = end;
end = start + chunkSize;
chunkNumber++;
}
}
// Function for the generate unique GUI ID
private GenrateUploadId(): string {
return 'xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx'.replace(/[xy]/g, c => {
const r = Math.random() * 16 | 0;
const v = c === 'x' ? r : (r & 0x3 | 0x8);
return v.toString(16);
});
}
// Starting Upload Session Method
private async StartUploadSession(siteUrl: string, libraryName: string, fileName: string, headers: any) {
try {
return await this.Retry(async () => {
const response = await fetch(
`${siteUrl}/_api/Web/Lists/getByTitle('${libraryName}')/RootFolder/Files/Add(url='${fileName}',overwrite=true)`,
{
method: 'POST',
headers: headers
}
);
if (!response.ok) {
const errorText = await response.text();
console.error('Failed to start upload session:', errorText);
throw new Error(`Failed to start upload session: ${errorText}`);
}
return response.json();
});
} catch (error) {
console.error('Failed to start upload session after retries:', error);
throw error;
}
}Start Upload Method:
- This method is called when attempting to upload the first chunk of the file to our SharePoint document library.
- In this method, we called the SharePoint Post REST API with the parameter of the start upload method along with a unique GUID.
- The endpoint for the API is "`${siteUrl}/_api/web/GetFileById('${fileId}')/StartUpload(uploadId=guid'${uploadId}') `".
private async UploadFirstChunk(
siteUrl: string,
libraryName: string,
fileName: string,
chunk: any,
uploadId: string,
headers: any,
fileId: string
) {
try {
return await this.Retry(async () => {
const response = await fetch(
`${siteUrl}/_api/web/GetFileById('${fileId}')/StartUpload(uploadId=guid'${uploadId}')`,
{
method: 'POST',
headers: headers,
body: chunk
}
);
if (!response.ok) {
const errorText = await response.text();
console.error('Failed to upload first chunk:', errorText);
throw new Error(`Failed to upload first chunk: ${errorText}`);
}
return response.json();
});
} catch (error) {
console.error('Failed to upload first chunk after retries:', error);
await this.CancelUpload(siteUrl, fileId, uploadId, headers);
await this.DeleteFile(siteUrl, fileId, headers);
throw error;
}
}Continue Upload Method:
- The "Continue Upload" method in SharePoint's REST API allows for the upload of intermediate chunks of a file during a large file upload session.
- The API endpoint for continuing the upload is:
"/_api/web/GetFileById('
')/ContinueUpload(uploadId=guid' ',fil eOffset= )".
This endpoint specifies the file being uploaded (fileId), the unique upload session ID (uploadId), and the starting byte position of the chunk (fileOffset).
The "file Offset" parameter specifies the starting byte position of the chunk being uploaded in the overall file. It helps SharePoint understand where this chunk fits within the entire file.
it indicates the position in the file where the current chunk starts.
For example, if the first chunk is 1MB (1048576 bytes) in size, the file Offset for the second chunk would be 1048576, the third chunk would be 2097152, and so on.
private async UploadIntermediateChunk(siteUrl: string, libraryName: string, fileName: string, chunk: any, uploadId: string, headers: any, start: any, fileId: string) {
try {
return await this.Retry(async () => {
const response = await fetch(`${siteUrl}/_api/web/GetFileById('${fileId}')/ContinueUpload(uploadId=guid'${uploadId}',fileOffset=${start})`, {
method: 'POST',
headers: headers,
body: chunk
});
if (!response.ok) {
const errorText = await response.text();
console.error('Failed to upload chunk:', errorText);
throw new Error(`Failed to upload chunk: ${errorText}`);
}
return response.json();
});
} catch (error) {
console.error('Failed to upload intermediate chunk after retries:', error);
await this.CancelUpload(siteUrl, fileId, uploadId, headers);
await this.DeleteFile(siteUrl, fileId, headers);
throw error;
}
}
Finish Upload Method:
- The "Finish Upload" method is used to upload the final chunk of a large file to a SharePoint library, signaling the end of the upload process.
- The method sends a POST request to the SharePoint API endpoint to finish the upload.
- API endpoint is: "/_api/web/GetFileById('<fileId>')/FinishUpload(uploadId=guid'<uploadId>',fileOffset=<start>)".
private async UploadLastChunk(siteUrl: string, libraryName: string, fileName: string, chunk: any, uploadId: string, headers: any, start: any, fileId: string) {
try {
return await this.Retry(async () => {
const response = await fetch(`${siteUrl}/_api/web/GetFileById('${fileId}')/FinishUpload(uploadId=guid'${uploadId}',fileOffset=${start})`, {
method: 'POST',
headers: headers,
body: chunk
});
if (!response.ok) {
const errorText = await response.text();
console.error('Failed to upload chunk:', errorText);
throw new Error(`Failed to upload chunk: ${errorText}`);
}
return response.json();
});
} catch (error) {
console.error('Failed to upload last chunk after retries:', error);
await this.CancelUpload(siteUrl, fileId, uploadId, headers);
await this.DeleteFile(siteUrl, fileId, headers);
throw error;
}
}Cancel Upload And Delete File:
- The "Cancel Upload" method is used to cancel an ongoing large file upload session in SharePoint. This is typically done when an error occurs during the upload process, and you want to terminate the session to prevent incomplete or corrupted files from being saved.
- Sends a request to the SharePoint API to cancel the current upload session identified by uploadId.Utilizes the unique fileId and uploadId to specify which upload session to cancel.
- Helps ensure that partially uploaded files are not left in an inconsistent state.
- The "Delete File" method is used to delete a file from a SharePoint library. This is usually called after canceling an upload session to remove any partially uploaded files and clean up the SharePoint library.
- Sends a request to the SharePoint API to delete the file identified by file.
- Ensures that any incomplete or unwanted file uploads are removed, maintaining the integrity of the document library.
private async CancelUpload(siteUrl: string, fileId: string, uploadId: string, headers: any) {
try {
const response = await fetch(`${siteUrl}/_api/web/GetFileById('${fileId}')/CancelUpload(uploadId=guid'${uploadId}')`, {
method: 'POST',
headers: headers
});
if (!response.ok) {
const errorText = await response.text();
console.error('Failed to cancel upload session:', errorText);
throw new Error(`Failed to cancel upload session: ${errorText}`);
}
} catch (error) {
console.error('Error occurred while canceling upload session:', error);
}
};
private async DeleteFile(siteUrl: string, fileId: string, headers: any) {
try {
const response = await fetch(`${siteUrl}/_api/web/GetFileById('${fileId}')`, {
method: 'DELETE',
headers: headers
});
if (!response.ok) {
const errorText = await response.text();
console.error('Failed to delete file:', errorText);
throw new Error(`Failed to delete file: ${errorText}`);
}
} catch (error) {
console.error('Error occurred while deleting file:', error);
}
}Summary:
This blog explains how to upload large files to SharePoint using a segmented approach for efficiency and reliability. It starts with the Start Upload Method, which initializes the upload session and prepares the file. Next, the Continue Upload Method handles middle segments, ensuring sequential upload using fileOffset. Finally, the Finish Upload Method completes the upload by sending the last segment, ensuring all parts are integrated into SharePoint. These methods include error handling and retries to ensure successful uploads, overcome file size limits, and enhance system performance.
If you have any questions you can reach out our SharePoint Consulting team here.
No comments:
Post a Comment