Errors and Faults

In this guide we cover:

• How errors can be returned via webhook payloads
• How to fault find when webhooks appear not to work

Webhook Error Payloads

An error payload will be returned in a webhook response when such an event arises, however as we validate the GraphQL payload definitions against our current schema at the point of creation, such occurrences should be rare. Nonetheless it’s still good practice to provide such a mechanism.

Example of GraphQL Query Validation

When errors do occur

In the event that an error does occur, an appropriate error response will be sent as the webhook payload:

{
  "event_name": "update",
  "payload": {
    "errors": [
      {
        "path": [
          "query",
          "node",
          "... on Advert",
          "badQuery"
        ],
        "message": "Field 'badQuery' doesn't exist on type 'Advert'",
        "locations": [
          {
            "line": 1,
            "column": 77
          }
        ],
        "extensions": {
          "code": "undefinedField",
          "typeName": "Advert",
          "fieldName": "badQuery"
        }
      }
    ]
  }
}

In this case we placed an invalid expression: badQuery in the GraphQL webhook definition, (we had to temporarily disabled validation to force this error to occur).

Anatomy of the Error Payload

The make-up of an error payload is shown below:

Each error object will also have:

• path: the path to the offending item in the query
• message: a summary of the error that occurred
• locations: the line and column that the offending item resides at
• extensions: further detail on the error type

Legacy Webhook Payloads

Prior to September 2021 we did not send error payloads with webhook responses, (you would have received a null payload in the event of an error). If you have pre-existing webhooks that you would like to enable error responses on, you can edit the webhook to enable them, we detail this below.

Differences in payload shape

The differences between the shape of generic webhook messages (current vs legacy) are shown below:

How to enable errors on pre-existing webhooks

To enable to the delivery of error payloads on webhooks created prior to September 2021, login to the Admin Portal and select:

• Configuration -> Webhooks and click the edit icon next to the webhook you wish to edit:

For existing webhooks the Legacy Format check box should be checked, to enable the new format, uncheck the checkbox, (the reverse is of course true if you wish to re-enable the legacy format).

Don’t forget to click Save

Other Fault Finding Tips

When working with Webhooks it may appear that you are not getting those webhooks delivered to you, the following section outlines some fault finding tips along with the most common errors that we see.

1. Checking if Marketplacer Generated a Webhook Event

The first thing to check on the Marketplacer side, is whether a webhook has been:

1. Sent
2. Cancelled
3. Rejected
4. Failed
5. Skipped

We define these states further below.

To check this, login to the Admin portal and select:

• Configuration -> Webhooks and click the “+” icon next to the webhook you want to investigate:

If webhooks are being triggered by Marketplacer, you should see the events listed here, along with the:

• The triggering Object (the “target”, e.g. a specific Invoice)
• Event Name (e.g. Create, Update etc.)
• Idempotency Key
• The Payload we sent
• The current status of the webhook
• When the webhook was created (i.e. “triggered”)

A list of webhook events in differing states are shown below:

In this example you can see we have number of use-cases in play, let’s look at each 1 in turn:

1. Sent: This webhook event has been successfully sent to the endpoint (Marketplacer received a HTTP 2xx response).
2. Skipped This means that we have additional deduplication enabled and we have not sent this event due to the fact it has an identical payload to the last event for this object. See Webhook Deduplication for more info.
3. Cancelled: This webhook has been cancelled by the operator mid-retry cycle. The cancel operation can be activated by navigating to the Response page. You can Requeue a Cancelled event if you want to attempt to re-send.
4. Failed: We have completed all retry attempts and the webhook event has failed. You can navigate to the Response page to look deeper into the responses that were returned. You can Requeue a Failed event if you want to attempt to re-send.
5. Rejected: We are currently in the Webhook Retry Cycle and so far we have only received error responses (“Rejections”) from the webhook endpoint. You can elect to Cancel or Requeue the event when it is in this state.
6. Unsent: [Not shown] The webhook event has been triggered but we have not attempted to send yet. This state only occurs prior to the 1st send attempt (not subsequent retries - see Rejected state) and should only exist for short period of time.

In all scenarios, you can check to see the responses Marketplacer has been receiving from the endpoint - we cover that in the Checking Webhook Responses section below.

No Events?

If you do not see any events listed here, then that would suggest that no webhook events have been triggered - yet, (or that the webhook event occurred over 7 days ago and has been purged). If a suitable scenario has occurred, (refer to the Webhook Scenario Event Matrix), then we would typically expect a webhook event to have been created in well under 1 minute.

If this is the case:

• Check that you are running a scenario that would invoke an event
• Check that the environment where you are triggering event scenarios is the same environment where you have configure webhooks
• Check that the webhook event was not fired over 3 days ago (we purge webhook events over 3 days old)

2. Checking Webhook Responses

The next thing to check, is to look at the response received from Marketplacer when we attempted to send a Webhook payload. It is also from this page that you can decide to:

• Cancel a currently retrying webhook event
• Requeue a Cancelled, Failed or currently retrying webhook event

To do this, ensure you are looking at the Webhook event page (discussed above), then select the “+” icon next to the event you want to investigate further:

Examples of the response page for all 4 event states are shown below.

1. Successfully Sent Webhook

In this case you will see that we received a http 200 OK response from the endpoint along with headers and body. This webhook has a sent time stamp.

2. Cancelled Webhook

Here we can see:

• Marketplacer was receiving a HTTP 410 (Unauthorized) response
• An Operator had decided to cancel any further retry attempts (we have a cancelled timestamp)
  1. You can elect to “requeue” this Cancelled request, (this re-starts the retry cycle from the beginning again)

3. Rejected Webhook

Here we can see:

• Marketplacer is currently receiving a HTTP 500 (Internal Server Error) response
• We are still in the retry cycle, (we don’t have either a sent, cancelled or failed time stamp), and have 2 options:
  1. Cancel all further retries
  2. Requeue the current retry cycle, (this re-starts the retry cycle from the beginning again)

4. Failed Webhook

Here we can see:

• Marketplacer was receiving a HTTP 500 (Internal Server Error) response
• Marketplacer had exhausted its retry attempts and has marked the event has failed (we have a failed timestamp)
  1. You can elect to “requeue” this Failed request, (this re-starts the retry cycle from the beginning again)

Common Fault Scenarios

While the list below is in no means exhaustive, we have found that issues with webhooks not being sent usually relate to the following:

• The webhook destination endpoint is not available
• The webhook destination endpoint is “behind” some other gateway
• The webhook destination endpoint has some kind of authentication enabled
• The webhook destination endpoint is throwing an exception
• The webhook destination endpoint cannot deal with the traffic volume
• The webhook destination endpoint certificate cannot be validated

In any case the HTTP response code along with the associated headers and body should give great indication of the nature of the error.

How we retry

Please refer to this article on the webhook retry process.

Raising a support ticket

If after following the fault finding tips above you still believe there to be an issue with webhooks, please raise a support ticket containing the following information here.

• Vertical / Instance Name
• Environment (e.g. Production / Staging etc.)
• Webhook trigger scenario (e.g. Creating an Advert, deleting a Variant etc.)
• Was a Webhook event generated (y/n)
• Was a webhook sent (what is the HTTP Response code)
• What you expected to happen
• What is actually happening