Handling Zendesk events via EventBridge

· 5 min · medium

Note: This article was written before the announcement on Zendesk deprecating their partnered EventBridge integration in December which can be replaced via a webhook integrating into EventBridge. However, this article is relevant to any event-based broker integration and the architecture discussions it entails are valuable.

Introduction

In this post, we will cover how we solved some inefficiencies in our business by using Zendesk’s EventBridge partnered connector in conjunction with an event-driven architecture. If you’re unfamiliar with Zendesk it’s a powerful customer service platform that helps organisations manage and resolve customer inquiries efficiently. We use this here at Moonpig for customer service operations.

AWS EventBridge is a serverless bus that enables services to be loosely coupled and event-driven. It can consume data from multiple sources and publish these to different destinations.

Problem Space

Within Moonpig when a customer interacts with us through our Chatbot via Zendesk we’re aware due to processes such as looking up an order. We are also aware of when a resolution has been reached. When this happens, we add meta information to each ticket, saving our support team time. One of the challenges we face is that Zendesk has a variable time window for ticket creation, so we don’t know when tickets will be available for our enrichment process.

Initially, this was solved via long polling, a common but inefficient pattern that can lead to increased infrastructure costs and higher API rate limits. Polling regularly consumes unnecessary resources and creates latency in data processing. However, thanks to the introduction of Zendesk’s partner resource, we can now move to a more efficient, event-driven process, which eliminates these challenges.

Architecture

For our architecture, we’ll be using AWS serverless offerings to create a redundant, event-driven architecture. Here’s our starting point:

Figure 1) Proposed architecture

At a high level, we have:

Our architecture for this example looks good, however, we’re only dealing with two CRUD events on a single resource which is Ticket. But, what if we want more? There are over 100 different events across all resources, would our above architecture cope? The answer is yes, but that doesn’t mean it’s necessarily the best solution.

Being pragmatic we want to focus on value delivery and get the smallest unit out quickly to gain knowledge, however, we need to understand the tradeoffs if we’re to stick with the architecture in the context of an enterprise which follows the AWS Well-Architected Framework.

Considerations

Figure 2) Architecture illustration between accounts
  1. Although decoupled, the proposed architecture still presents a level of rigidity because we need to manually update SNS subscriptions for each consumer. This kind of tight coupling can become an anti-pattern in event-driven architectures, as it reduces agility and increases maintenance overhead.
  2. The second drawback is that this doesn’t empower our downstream consumers with granularity. We could use Amazon EventBridge’s cross-account event delivery feature. This approach would allow you to decouple your event producers and consumers even further, reducing the need to manage subscriptions directly on SNS topics.

EB to EB

Figure 3) These secondary EBs could be in separate accounts too

Forwarding broad entity events into another EventBridge would allow for the following:

A simple example of the power of this architecture would be a service only cares about an entity that has specific attributes, e.g. in the context of a ticket perhaps our service only wishes to ingest if the tags on a ticket are either foo or bar. With this architecture, we can use our own rules to make this possible and eliminate our consumers from receiving irrelevant data.

However, the architecture is crucial here, if we applied the above to our architecture in Figure 1 via highly specific event patterns, this could become an anti-pattern, making the system difficult to maintain and debug. It’s important to balance granularity with simplicity, ensuring that patterns are easy to manage over time.

Implementation

To create this setup we’d use OpenTofu, our preferred IaC tool. To keep this agnostic we’ll go through the AWS console instead. This section assumes the following:

Steps

  1. EventBridge > Rules > Create a rule, and call it ticket-created
  2. Event Source > Select ‘Other’
  3. We can ignore the Sample event section. However, this is useful for finding pre-existing patterns and testing our event pattern against specific data shapes.
  4. In the Event Pattern section, we’ll need to specify what event we wish to receive, however, an important consideration is depending on your configuration of Zendesk workflows, the data you require may not be immediately available, which, because this is event-driven, would mean we’re missing data. For us, the event of a status change is appropriate so we’ll use this.
Support Ticket: Custom Status Changed

This means our example will look like the following in Event Pattern:

{
  "detail-type": ["Support Ticket: Custom Status Changed"],
  "resources": ["Support Ticket"]
}

  1. You’ll also want to make sure Prefix matching is set in the dropdown.

  2. Next, we’ll want to set a target for the data which matches the above rule, we’re going to choose our SNS, but this could easily be Kinesis, Lambda or another EventBridge

  3. The SNS will now publish events based upon our resource and detail ruling, it’s important we test these rules to ensure our event contains the data we require, as it will be a snapshot of the entity at the time of changing

🤔 Conclusion

We’ve gone through setting up an event-driven system, from consuming SaaS data into our estate for internal consumption and speaking about the benefits of this being async and event-driven. We’ve also considered our EventBridge broker in the context of our AWS strategy and how it may be built upon, speaking about different architecture decisions we could take and how they would set us up for future success. Finally, we briefly touched on the high-level implementation using the AWS Console.