How to overcome URL length limitation while fetching data using REST API in SharePoint?

Problem Statement:

Sometimes we might need to fetch data from long URL containing more filters while using REST API. REST query allows max –length of URL is 2,048 characters. If you exceed this limit, will throw 404 error in the code. let's check out how we should overcome this issue.

Analysis:

When we execute the query with URL having less filters & columns and thus having URL length less than the limit, we can get the response correctly:

But if we keep on increasing the filters & columns in REST API, once URL exceeds the limit of 2048 characters, we should start getting 404 error as shown below.

So, I thought of checking the URL with same filters in SharePoint directly, would it give same error? Surprisingly,  I found that SharePoint shows the results correctly with same numbers of filter and even more.

Technical Solution:

My analysis got me on the conclusion that the request is getting 404 caused by browser execution limitation of REST API and is not the issue from SharePoint end. So, we need to find a way through which we can send query to SharePoint and execute without browser itself.  So, I thought of using Batch API option for the same. I followed below steps to execute long URL successfully:

    Step 1: Load BatchUtils script into browser.

      Step 2: Verify BatchUtils loaded and Run in browser.

     Step 3: Prepare data for BatchUtils.

·         You need to specify Root Site URL of SharePoint tenant.

·         Push Long URL in array.

     Step 4: Request for fetch data using BatchUtils.

·         Here, you go! get the results without any error.

Note: We can use BatchUtils in Custom SPFX Web Part.

Below are useful reference links which helps us in calling Batch API in SharePoint Online and Run Code snippets direct in chrome DevTools.

If you have any questions you can reach out our SharePoint Consulting team here.

Manage more than 100 API calls with Batch Request in SharePoint Framework (SPFx) using React

Requirement:
We have IDs of 1000 list items and want to fetch list items for only those particular IDs using Rest API with OR condition in the filter.

Problem Statement:

1. If we append all IDs in the filter parameter with OR condition, it will give REST API length limitation (255 characters).
2. If we use API with OR condition for all IDs and use this API in a batch request, batch API will return a blank array.
3. If we create API for each ID and create an array of APIs & use this array in a batch request, it will work for maximum up to 100 APIs only. As we have 1000 list items to be retrieved, this will also not work due to the 100 APIs limit with a single batch request.

Solution:

We can create API for 100 Item IDs with OR condition in the filter API and create a bunch of 100 APIs in an array and then use this array in a batch request.

You can find batch utility code here on GitHub. Here is the code snippet for this solution:

 //Here we have imported batchutility.ts  
 import { BatchUtils } from "../../BatchUtils";  
 public componentDidMount() {  
   var listItemIds = [1, 2, 3, 4, 5, 6, ..............................., 1000];  
   var url = this.props.SPUrl + "/_api/web/lists/getbytitle('" + this.props.projectPhaseListName + "')/items?$filter=";  
   this.getDashboardBatchData(listItemIds, 0, listItemIds.length, url);  
 }  
 //Below function will create bunch of filters with max of 100 id in filter with or condition  
 public getDashboardBatchData(listItemIds, Index, totalCount, url) {  
   var loopLen = Index + 100;  
   if (Index <= totalCount) {  
     var filterString = "";  
     var tempApi = '';  
     var callNext = true;  
     if (totalCount > Index && totalCount < loopLen) {  
       loopLen = totalCount;  
       callNext = false;  
     }  
     for (var i = Index; i < loopLen; i++) {  
       if (filterString == '') {  
         filterString = "ID eq " + listItemIds[i];  
       }  
       else {  
         filterString += " or ID eq " + listItemIds[i];  
       }  
     }  
     tempApi = url + filterString;  
     dashboardBatchArray.push(tempApi);  
     if (callNext) {  
       this.getDashboardBatchData(listItemIds, loopLen, totalCount, url);  
     }  
     else {  
       //dashboardBatchArray will have all APIs with max of 100 filters for ID  
       this.processBatch(dashboardBatchArray);  
     }  
   }  
   else {  
     //dashboardBatchArray will have all APIs with max of 100 filters for ID  
     this.processBatch(dashboardBatchArray);  
   }  
 }  
 //Below funcion will create bunch of 100 APIs and will be used in batch request  
 public processBatch(dashboardBatchArray){  
   var index = 0;  
   var arrayLength = dashboardBatchArray.length;  
   var tempArray = [];  
   var chunk_size = 100;  
   //Below code will create array of 100 APIs in single bunch  
   for (index = 0; index < arrayLength; index += chunk_size) {  
     var myChunk = dashboardBatchArray.slice(index, index + chunk_size);  
     tempArray.push(myChunk);  
   }  
   //Below code will execute all apis of tempArray  
   let listItemsArray = [];  
   for (var i = 0; i < tempArray.length; i++) {  
     await BatchUtils.GetBatchAll({ rootUrl: this.props.SPUrl, FormDigestValue: '', batchUrls: tempArray[i] }).then((batchResult) => {  
       if (batchResult.length > 0) {  
         for (var i = 0; i < batchResult.length; i++) {  
           for (var j = 0; j < batchResult[i].d.results.length; j++) {  
             listItemsArray.push(batchResult[i].d.results[j]);  
           }  
         }  
       }  
     }  
   }  
   //You will see all 1000 items in listItemsArray variable  
   console.log(listItemsArray);  
 }  

If you have any questions you can reach out our SharePoint Consulting team here.