Batch requests

What you’ll learn

In this guide you’ll learn:

• What is batching
• How batch requests can be made with GraphQL
• Considerations when attempting batching

What is batching?

Batching is taking multiple requests (e.g. create 100 products) and rolling them into a single request, then either:

• Waiting synchronously for the request to complete, or
• Obtaining a synchronous reply that the job had been submitted, and await asynchronously for the job to complete

Batching with GraphQL

There are 2 primary approaches to batching request with GraphQL:

• Using aliases
• Supplying operations in an array

We’ll cover each of these in the next section.

Using Aliases

GraphQL natively supports batching through the use of aliases, in the example below we assign aliases to each of the operations in the request (the externalIdUpsert mutation) to attach 2 external IDs to a variant.

mutation {
  a: externalIdUpsert(
    input: { owner: "VmFyaWFudC0xMDA2Nzk=", key: "SYS1", value: "ABCDE" }
  ) {
    errors {
      field
      messages
    }
    externalId {
      ...ExternalIdResponse
    }
  }
  b: externalIdUpsert(
    input: { owner: "VmFyaWFudC0xMDA2Nzk=", key: "SYS2", value: "12345" }
  ) {
    errors {
      field
      messages
    }
    externalId {
      ...ExternalIdResponse
    }
  }
}
fragment ExternalIdResponse on ExternalID {
  key
  value
  owner {
    ... on Variant {
      id
      label
      countOnHand
    }
  }
}

This would return:

{
  "data": {
    "a": {
      "errors": null,
      "externalId": {
        "key": "SYS1",
        "value": "ABCDE",
        "owner": {
          "id": "VmFyaWFudC0xMDA2Nzk=",
          "label": "White",
          "countOnHand": 109
        }
      }
    },
    "b": {
      "errors": null,
      "externalId": {
        "key": "SYS2",
        "value": "12345",
        "owner": {
          "id": "VmFyaWFudC0xMDA2Nzk=",
          "label": "White",
          "countOnHand": 109
        }
      }
    }
  }
}

You can see that the aliases a and b are represented in the single response payload.

One of the disadvantages of this approach is the fact that the request is treated as 1 request, making tracing and measurement of the separate alias components much more difficult.

Supplying operations in an array

The second approach is to supply each operation in an array, in the example below we are:

• Using the sellersWhere query to perform a lookup on multiple seller IDs
• Showing the request in its “raw” JSON format to illustrate the array of operations

[
  {"query":"query query_array_item_1{\n\ta: sellersWhere(first: 1, retailerIds: [\"U2VsbGVyLTQy\"]) {\n\t\tnodes {\n\t\t\t__typename\n\t\t\tid\n\t\t\tlegacyId\n\t\t\tbusinessName\n\t\t}\n\t}\n\n}\n","operationName":"query_array_item_1"}
    ,
  {"query":"query query_array_item_2{\n\tb: sellersWhere(first: 1, retailerIds: [\"U2VsbGVyLTM=\"]) {\n\t\tnodes {\n\t\t\t__typename\n\t\t\tid\n\t\t\tlegacyId\n\t\t\tbusinessName\n\t\t}\n\t}\n\n}\n","operationName":"query_array_item_2"}
]

This would return the following:

[
  {
    "data": {
      "a": {
        "nodes": [
          {
            "__typename": "Seller",
            "id": "U2VsbGVyLTQy",
            "legacyId": 42,
            "businessName": "ACME Corp."
          }
        ]
      }
    }
  },
  {
    "data": {
      "b": {
        "nodes": [
          {
            "__typename": "Seller",
            "id": "U2VsbGVyLTM=",
            "legacyId": 3,
            "businessName": "Devlin MacGreggor"
          }
        ]
      }
    }
  }
]

You will observe that we are still using aliases (although we don’t need to) but we are getting 2 separate data responses, (contrasted with the 1st example where there was only 1 data response).

Some points to note about using this approach:

• Not all GraphQL servers necessarily support this approach
• With this approach the requests are treated discretely from tracing & measurement perspective (unlike with the alias approach in example 1)
• GraphQL clients (e.g. Apollo) will construct batch requests in this way

Considerations

Irrespective of whether you use the alias or array approach, making batch requests in a synchronous way should be treated with caution. While it may appear simpler on the surface (you only need to make 1 call), the following are also true:

• You are waiting for a response for all the work to be done, therefore this approach doesn’t really scale. Supplying 10 operations in the 1 request may be responsive - but what about 100, or a 1000 or more? You may well hit a timeout scenario, or trigger rate limiting of some degree, e.g. GraphQL complexity scores. Just because it’s 1 request doesn’t necessarily equate to it being dramatically quicker than individual requests.
• Error handling is more complex, partial failures of 1 operation require more sophisticated handling. Debugging can also be harder.
• 1 operation in the batch may take significantly longer than the others, slowing the entire batch down from a client response perspective.

Conclusion

Batching is baked into GraphQL, but it should be used with caution. You need to assess whether adopting it for your given use-case will actually deliver the benefits you expect. Most of all, when using it in a synchronous way, it’s not a magic bullet in necessarily getting things done faster.