Most organizations are adopting an event-driven architecture (EDA) to compete in a world where customer satisfaction requires real-time outcomes.

In this code lab we'll build and expand your toolbox by learning how an Event Portal, paired with industry standard specifications and frameworks, enable a smooth journey to bring your EDA from initial architecture and design to code running in production while also setting your team up for success as the business needs, architecture and applications themselves are enhanced over time.

Throughout this workshop we will get hands on and talk about:

PubSub+ Event Portal

AsyncAPI

🛠 This page covers the setup needed to perform this codelab. 🛠

AsyncAPI Generator Requirements

✅ Install instructions available here

We'll install the generator itself later 👍

Spring Cloud Stream Requirements

✅ Spring Cloud Stream just requires Java and Maven to use 🚀

PubSub+ Event Broker Connection Info

✅ The credentials below are for a public event feed found on the Solace feed Marketplace that we'll use during this codelab.

✅ Note that this client-username has permissions to subscribe to taxinyc/> and test/taxinyc/> and permissions to publish to test/taxinyc/>

Prepare PubSub+ Event Portal

Sign-up for Solace Cloud

✅ If you already have a Solace Cloud account just login, otherwise please sign-up for a free Solace Cloud Account using this link. Note that no credit card is required. You will receive an email to activate the account and will then be prompted to start the free trail.

sc_trial

Import Existing Designed EDA

✅ Download the Application Domain export file: EventPortal_Export_NYCModernTaxiCo.json

You can download the file via curl or by cloning the git repo

curl -k -XGET https://raw.githubusercontent.com/Mrc0113/design-to-code-workshop/master/EventPortal_Export_NYCModernTaxiCo.json -o EventPortal_Export_NYCModernTaxiCo.json

OR

git clone https://github.com/Mrc0113/design-to-code-workshop.git

✅ Inside of your logged into Solace Cloud Account navigate to the Event Portal Designer by clicking "Designer" in the menu on the left.

ep_select_designer

✅ Then import the previously downloaded Application Domain file by clicking the Import button at the top right of the Designer and importing the file.

ep_click_import

🚀 Setup complete! Let's get going! 🚀

Before we dive deeper, let ensure we are all aligned with terminology of the objects and concepts we will use in PubSub+ Event Portal.

Application Domain & Workspace

An application domain represents a namespace where applications, events, and schemas can live. Within this namespace, you can create a suite of applications, events and schemas that are independent of other application domains. In our NYC Taxi use case we introduced earlier, we may group application into different domains, for ex. we may have a domain for our rideshare apps and services, one for our back-office apps where we invoicing and background checks are being processed, and maybe another domains for analytics where we group apps that are responsible for analyzing the successful operation of our rideshare services.

In the Event Portal you will associate all objects like Consumer Groups, Topics, Schema, etc, to one or more Application Domains.

You can further group multiple domains into a Workspace, which will make it easier to review our Discovery scan. So our Analytics, Operations, and Back-Office Application Domain in the NYC taxi example could be part of a single Workspace.

Workspace Example

Events/Topics

Events are an important part of the Event Portal. Think of a event as a concept of the publish-subscribe (pub/sub) architectural pattern. Topics are used to route data or events (in the form of messages) between distributed applications, often using a message broker or an event broker.

A Solace topic and an Apache Kafka topic might seem fundamentally the same but there are quite a few differences between them. Later in this CodeLab, when you run a discovery scan against a Kafka cluster the Topic Scheme for events discovered will be Kafka format.

Here are some examples from our use case:

Schemas

In simple terms, a schema represents the contract to describe the payload of an event. Producers and consumers of an event can trust that the event's payload matches the schema definition assigned to that event. Schemas define a type of payload through JSON, AVRO, XML, Binary, or Text. JSON, AVRO, and XML schemas have content that describes each property of the schema.

In our use case all events are in AVRO Schema format.

Schema Example

Applications

An application represents a piece of software that produces and consumes events. Applications connect to the event broker in an event-driven architecture and communicate with other applications via events. A single application represents a class of applications that are running the same code base; therefore, a Kafka consumer group can be associated with an Application object in the Event Portal.

Kafka Specific Objects and Terminology

Consumer Groups

Event Portal supports the concept of Kafka's consumer groups. A consumer group is used by Kafka to group consumers into a logical subscriber for a topic. In Event Portal, you can model consumer groups in Designer. This enables the Event Portal's runtime discovery to associate a discovered consumer group to an existing application.

Kafka consumers that belong to the same consumer group share a group ID. The consumers in a group divide the topic partitions, as fairly as possible, so that each consumer consumes only a single partition from the group.

A few examples of Consumers Groups from our NYC Taxi Analytics use case would be:

Connector

A connector is used in Kafka for connecting Kafka brokers with external systems to stream data into or out of Apache Kafka. In the Event Portal, a Kafka Connector is an application class you select to configure associated published and/or subscribed events and a set of Kafka-native attributes like Connector Type, Class, Cluster ID, and Maximum Task.

Decomposing the Enterprise

Whether you perform discovery manually or using our agent, it is important to consider how your enterprise is organized so that it can be decomposed using the Application Domain construct. An Application Domain provides the ability to organize and decompose an enterprise into logical groupings. These groupings could be based on-line of business, related functional capabilities or based on team dynamics. The benefits of doing this include:

  1. Event sharing rules – decide which events should be shared with other application domains and those which are for internal application domain usage only. This has implications both from a security perspective, but also which events need to be managed more tightly as they affect others outside of the application domain
  2. Provide uniform event topic prefixes – ensures that the prefix is unique and that topic best practices are followed

Topic Naming Best Practices

The topic of which an event is addressed seems like a pretty simple decision, but in reality, it can result in some negative consequences if not planned in advance. A topic is more than an address, it is metadata that describes the event and can be used for several purposes such as routing, access control and versioning. Thus, it is important to properly govern and manage the topic structure. Regardless of your broker type, it is a good practice to make topics structured and hierarchical the same way a RESTful Resource uses hierarchical addressing. In other words we want to produce hierarchical topics that rank from least specific to most specific.

Parts of the Event Topic

The event topic structure has two parts:

The event topic root contains enough information to describe the type of event that has occurred. Each event topic root is a static field that describes the type of event. The list of event topic roots forms a catalog of events that can be produced and consumed. This catalog could be brought into the PubSub+ Event Portal's event catalog, listing each event type along with details about the event. Each event topic root describes the event in as much detail as necessary to map it to a single data schema.
The event topic properties are optional fields that further describe a particular event. This part of the topic has fields that are dynamically filled when the producer publishes the event. These fields are used to describe the specific or unique attributes of this event instance that would be used for routing and filtering.

For more information about topic best practices, review the Topic Architecture Best Practices Guide

Event Information Exchange Patterns

There are multiple Event Exchange Patterns (EEP) that should be considered when using EDA:

You are a member of the engineering team at the NYC Modern Taxi Co, a fictional taxi cab company based in New York City. Your team is playing from behind and racing to catch up with technology innovation introduced to the industry by Rideshare competitors such as Uber and Lyft. In order for the company to survive and eventually thrive your team has convinced the board that transforming the companies' IT systems is of utmost importance. Your team has done it's research and determined that moving to an Event-Driven Architecture is essential to future rapid innovation and has already kicked this initiative off by deploying a Solace Event Mesh and updating the taxi fleet to stream real-time events that include ride and location information. We know what the fleet is up to! Now it's time to start to continually improve and provide a world class customer experience.

In order to react in a real-time manner the team has decided that we want to process the updates as they stream in from the fleet of taxis instead of putting them directly into a datastore and then having to retrieve them to do processing later. To prototype this work, you'll see a high level design in the diagram below. Since we already have the taxi fleet streaming their updates into our PubSub+ Event Mesh we need to do three things:

  1. 🚖 Create and capture this design in the PubSub+ Event Portal where we can define our Event-Driven Architecture, including its' components: Applications, Events and Schemas. This will allow us to define the details needed to implement, visualize and extend the architecture as it evolves, and share/collaborate with our entire engineering team as we continue to innovate.
  2. 🚕 Next up we're going to document some of the designed applications and events so that they can be understood and reused by others.
  3. 🚕 We will run a "discovery" scan of a Kafka Cluster to reverse engineer what another team at NYC Taxi already has implemented
  4. 🚕 Learn, Understand and Reuse some of our events in a new use case
  5. 🚖 Lastly we'll implement the ProcessPayment microservice that that receives the stream of RideUpdated events, charges the customer's credit card and generate a PaymentCharged Event.

Architecture

By designing a new event-driven application or extending your extending event-driven architecture, your able to deliver new real-time business capabilities in a decoupled and reusable fashion. There are however several key elements which should be considered when designing events, schemas and applications including topic best practices, options for exchanging event data and sharing/visibility rules. Considering these things early will put you on the road to success and enable better reusability down the road.

Now that you're familiar with the use case 🚕 🚖 🚕 and you've imported the application domain into the Event Portal, let's update our Event-Driven Architecture (EDA).

Lets say that your tasked with working within the Back Office team (where the cool kids all work) and are asked to architect the way in which we will charge our passengers for their rides and if the passenger is part of a commerical account, send to our Invoicing System.

Step 1: Determine What Can Trigger Payment

So essentially we need to consider, is there a business event that would help us trigger on the moment when the ride has been completed?

  1. Navigate to the Catalog component of the Event Portal
  2. Click on the Schemas tab and search for "completed"
  3. In the Search Results click on the matched fields in order to understand the matching text context.
  4. We now know that the RideUpdated Schema has a field called _ridestatus that can have a value of completed. So how do we get acccess to that data? Click on the RideUpdated schema and we will find out!
  5. We now see the metadata about the RideUpdated schema and at the bottom we can see there is an Event that references this schema called RideUpdated. The topic being used leverages the _ridestatus attribute which is pretty sweet! So we can filter on completed as a client.
  6. Lets navigate to the RideUpdated Event and look at its documentation to ensure its what we would want to trigger our ProcessPayment Application.

Step 2: Design the PaymentCharged Schema

Next we should decide what we want the data to look like once we have processed a payment.

  1. First we must decide what Event Exchange Pattern (EEP) we will use. For Maximum flexibility, and because time is not of the essence, we will leverage "Event-Carried State Transfer".
  2. Click into the Designer component of the Event Portal
  3. Double Click on the NYC Modern Taxi Co - Back Office Application Domain and its time to get creating!
  4. On the Upper Right Corner, Click the Create button and select Create Schema
    1. Name: PaymentCharged
    2. Content Type: JSON
    3. Shared: YES
    4. Owner: Assign Yourself
    5. Tags: NONE
    6. Description: NONE
    7. Versions: Leave unchecked
    8. Content:
      {
      "$schema": "http://json-schema.org/draft-07/schema",
      "$id": "http://example.com/example.json",
      "type": "object",
      "title": "The root schema",
      "description": "The root schema comprises the entire JSON document.",
      "default": {},
      "examples": [
      {
      "payment_charged_id": "23232323",
      "timestamp": "2020-06-03T16:51:47.29612-04:00",
      "information_source": "ProcessPayment",
      "payment_status": "accepted",
      "invoice_system_id": "PSG-32923",
      "amount_charged": 12.32,
      "ride_id": 2345234,
      "entity_type": "Driver",
      "driver": {
      "driver_id": 1234132,
      "first_name": "Frank",
      "last_name": "Smith",
      "rating": 4,
      "car_class": "SUV"
      },
      "passenger": {
      "passenger_id": 2345243,
      "first_name": "Jesse",
      "last_name": "Menning",
      "rating": 2
      }
      }
      ],
      "required": [
      "payment_charged_id",
      "timestamp",
      "information_source",
      "payment_status",
      "invoice_system_id",
      "amount_charged",
      "ride_id",
      "entity_type",
      "driver",
      "passenger"
      ],
      "properties": {
      "payment_charged_id": {
      "$id": "#/properties/payment_charged_id",
      "type": "string",
      "title": "The payment_charged_id schema",
      "description": "An explanation about the purpose of this instance.",
      "default": "",
      "examples": [
      "23232323"
      ]
      },
      "timestamp": {
      "$id": "#/properties/timestamp",
      "type": "string",
      "title": "The timestamp schema",
      "description": "An explanation about the purpose of this instance.",
      "default": "",
      "examples": [
      "2020-06-03T16:51:47.29612-04:00"
      ]
      },
      "information_source": {
      "$id": "#/properties/information_source",
      "type": "string",
      "title": "The information_source schema",
      "description": "An explanation about the purpose of this instance.",
      "default": "",
      "examples": [
      "ProcessPayment"
      ]
      },
      "payment_status": {
      "$id": "#/properties/payment_status",
      "type": "string",
      "title": "The payment_status schema",
      "description": "An explanation about the purpose of this instance.",
      "default": "",
      "examples": [
      "accepted"
      ]
      },
      "invoice_system_id": {
      "$id": "#/properties/invoice_system_id",
      "type": "string",
      "title": "The invoice_system_id schema",
      "description": "An explanation about the purpose of this instance.",
      "default": "",
      "examples": [
      "PSG-32923"
      ]
      },
      "amount_charged": {
      "$id": "#/properties/amount_charged",
      "type": "number",
      "title": "The amount_charged schema",
      "description": "An explanation about the purpose of this instance.",
      "default": 0,
      "examples": [
      12.32
      ]
      },
      "ride_id": {
      "$id": "#/properties/ride_id",
      "type": "integer",
      "title": "The ride_id schema",
      "description": "An explanation about the purpose of this instance.",
      "default": 0,
      "examples": [
      2345234
      ]
      },
      "entity_type": {
      "$id": "#/properties/entity_type",
      "type": "string",
      "title": "The entity_type schema",
      "description": "An explanation about the purpose of this instance.",
      "default": "",
      "examples": [
      "Driver"
      ]
      },
      "driver": {
      "$id": "#/properties/driver",
      "type": "object",
      "title": "The driver schema",
      "description": "An explanation about the purpose of this instance.",
      "default": {},
      "examples": [
      {
        "driver_id": 1234132,
        "first_name": "Frank",
        "last_name": "Smith",
        "rating": 4,
        "car_class": "SUV"
      }
      ],
      "required": [
      "driver_id",
      "first_name",
      "last_name",
      "rating",
      "car_class"
      ],
      "properties": {
      "driver_id": {
        "$id": "#/properties/driver/properties/driver_id",
        "type": "integer",
        "title": "The driver_id schema",
        "description": "An explanation about the purpose of this instance.",
        "default": 0,
        "examples": [
          1234132
        ]
      },
      "first_name": {
        "$id": "#/properties/driver/properties/first_name",
        "type": "string",
        "title": "The first_name schema",
        "description": "An explanation about the purpose of this instance.",
        "default": "",
        "examples": [
          "Frank"
        ]
      },
      "last_name": {
        "$id": "#/properties/driver/properties/last_name",
        "type": "string",
        "title": "The last_name schema",
        "description": "An explanation about the purpose of this instance.",
        "default": "",
        "examples": [
          "Smith"
        ]
      },
      "rating": {
        "$id": "#/properties/driver/properties/rating",
        "type": "integer",
        "title": "The rating schema",
        "description": "An explanation about the purpose of this instance.",
        "default": 0,
        "examples": [
          4
        ]
      },
      "car_class": {
        "$id": "#/properties/driver/properties/car_class",
        "type": "string",
        "title": "The car_class schema",
        "description": "An explanation about the purpose of this instance.",
        "default": "",
        "examples": [
          "SUV"
        ]
      }
      },
      "additionalProperties": true
      },
      "passenger": {
      "$id": "#/properties/passenger",
      "type": "object",
      "title": "The passenger schema",
      "description": "An explanation about the purpose of this instance.",
      "default": {},
      "examples": [
      {
        "passenger_id": 2345243,
        "first_name": "Jesse",
        "last_name": "Menning",
        "rating": 2
      }
      ],
      "required": [
      "passenger_id",
      "first_name",
      "last_name",
      "rating"
      ],
      "properties": {
      "passenger_id": {
        "$id": "#/properties/passenger/properties/passenger_id",
        "type": "integer",
        "title": "The passenger_id schema",
        "description": "An explanation about the purpose of this instance.",
        "default": 0,
        "examples": [
          2345243
        ]
      },
      "first_name": {
        "$id": "#/properties/passenger/properties/first_name",
        "type": "string",
        "title": "The first_name schema",
        "description": "An explanation about the purpose of this instance.",
        "default": "",
        "examples": [
          "Jesse"
        ]
      },
      "last_name": {
        "$id": "#/properties/passenger/properties/last_name",
        "type": "string",
        "title": "The last_name schema",
        "description": "An explanation about the purpose of this instance.",
        "default": "",
        "examples": [
          "Menning"
        ]
      },
      "rating": {
        "$id": "#/properties/passenger/properties/rating",
        "type": "integer",
        "title": "The rating schema",
        "description": "An explanation about the purpose of this instance.",
        "default": 0,
        "examples": [
          2
        ]
      }
      },
      "additionalProperties": true
      }
      },
      "additionalProperties": true
      }
      
    9. Revision Comment: "Initial Creation of Schema"
    10. Click Save

Design PaymentCharged Event

So now that we have constructed the payload format for the PaymentCharged event, it is time to design the event itself. What's involved? Well we need to apply our best practices as it comes to the Topic name!

  1. Click into the Designer component of the Event Portal
  2. Double Click on the NYC Modern Taxi Co - Back Office Application Domain
  3. On the Upper Right Corner, Click the Create button and select Create Event
    1. Name: PaymentCharged
    2. Shared: YES
    3. Description: NONE
    4. Topic Scheme: Solace
    5. Topic

      1. As you can see the domain aleady has some of the "Event Topic Root"
      2. We need to apply the best practice of Domain/ObjectType/Verb/Version/Locality/SourceID/ObjectID to this event
      3. We will use the topic name of: _taxinyc/backoffice/payment/charged/v1/${payment_status}/${driver_id}/${passengerid}
    6. Value:

      1. Keep the Schema radio button selected
      2. Choose the Schema "PaymentCharged" that we created in the previous step
    7. Owner: Assign Yourself
    8. Tags: NONE
    9. Revision Comment: "Initial Creation of Event"
    10. Click Save

Design ProcessPayment Application

Now for the fun part! We need to design the event-driven interface of the ProcessPayment Application. This is pretty easy as it has one input which triggers a single output.

  1. Click into the Designer component of the Event Portal
  2. Double Click on the NYC Modern Taxi Co - Back Office Application Domain
  3. On the Upper Right Corner, Click the Create button and select Create Application
    1. Name: ProcessPayment
    2. Description: NONE
    3. Application Class: Unspecified
    4. Owners: Assign Yourself
    5. Tags: NONE
    6. Associated Events:

      1. Click the Manage link

        1. Select the Sub button next to the RideUpdated event
        2. Select the Pub button next to the PaymentCharged event
        3. Click Save
    7. Revision Comment: "Initial Creation of Application"
    8. Click Save
  4. You should now see the newly added application on the graph!

Design InvoiceSystem Application

Remember back to our use case... We have designed how we process payment but still have to deal with invoicing customers when the payment_status says to invoice. Therefore, our plan is to create an application that integrates with our invoicing system.

  1. Click into the Designer component of the Event Portal
  2. Double Click on the NYC Modern Taxi Co - Back Office Application Domain
  3. On the Upper Right Corner, Click the Create button and select Create Application
    1. Name: InvoiceSystem
    2. Description: NONE
    3. Application Class: Unspecified
    4. Owners: Assign Yourself
    5. Tags: NONE
    6. Associated Events:

      1. Click the Manage link

        1. Select the Sub button next to the PaymentCharged event
        2. Click Save
    7. Revision Comment: "Initial Creation of Application"
    8. Click Save
  4. You should now see the newly added application on the graph!

✅ Know your Audience
The events which you have are used to enable Realtime collaboration between systems and solve a problem for a specific industry and organization. These events are integrated into applications by software developers/engineers but they are not all the same and can be decomposed into:

✅ Capture Business Point of View and Moment

✅ Technical Requirements

✅ Link to other References

✅ Provide Examples

✅ Terms of Use

✅ Tags

Update Documentation of PaymentCharged Event

Update Documentation of ProcessPayment Application

Lets enhance the documentation of the ProcessPayment Application and put our Documentation Best Practices to work!

  1. Click into the Designer component of the Event Portal
  2. Double Click on the NYC Modern Taxi Co - Back Office Application Domain
  3. Double Click on the ProcessPayment Application in the graph
    1. Click on the Edit button
      1. Copy and Paste the following into the Description field:
        Description of Business Capability
        
        	Overview: 
        
        		The ProcessPayment application solely exists in order to monitor for when Passenger Rides are completed such that final billing can be performed against the passengers credit card. Because this application will need to look up the passenger's billing information it is important that security be taken into account as it will need to be PCI compliant. Upon successful payment, the application shall emit an event to signify that payment has happened.
        
        
        
        Technical Requirements
        
        Java Version:  OpenJDK 11.0.4
        Spring Cloud Version:  Hoxton.SR8
        Number of Instances: 1
        Cloud: AWS us-east
        Security Level: PCI 
        Event Broker Profile: Solace
        
        
        Source Code Repository
        
        github repo
        
        
        
        Terms of Use
        
        N/A
        
      2. Lets make it nicer to read by using bullets, bold, italics etc
      3. Lets add a hyperlink to the github repo that points to https://github.com
      4. Lets now also add Tags
        1. Click Add/Remove Tags
          1. Type PCI in the box and Select (Create a new tag) below.
          2. Optionally add other tags.
          3. Click Done
      5. The documentation should look something like:
        asyncapi_doc2
      6. Click Save

Most organizations already leverage event driven architecture (EDA) and have one or more event brokers. Today the Solace PubSub+ Event Portal supports the ability to scan, catalog and reverse engineer the following Event Brokers:

  1. Kafka – Confluent Kafka, Amazon MSK, Apache Kafka
  2. Solace PubSub+ Event Broker – Coming Soon!

If you have a non-supported Event Broker type/configuration, then you will need add the schemas, events and applications to the Event Portal manually by using your existing documentation. While this may seem like a lot of work, it may be possible to capture this metadata and use the PubSub+ Event Portal's APIs in order to automate the ingestion of this data. The benefits of doing this from a dependency management perspective is enormous as your EDA evolves and enables you to begin to manage and expose the existing event-driven capabilities implemented.

Automated Discovery and Data Importation from Kafka

Once you have decided on the application domains that are required for your enterprise, it is time to start the data importation process.

If you have an event broker type/configuration that is supported by the discovery agent then an automated discovery process not only provides a faster path to managing and governing your existing EDA assets, it also ensures that the data is valid and up to date.

Event Portal Discovery with Kafka Code Lab

A critical aspect of the Event Portal is the capability to capture the EDA design and documentation in a central place to enable cross organizational learnings. These learnings come in multiple forms from creating new ideas, to enabling and training members of the team on the architecture to performing change impact analysis and more. The purpose of this section is to outline some of these scenarios and for you to think about ways to incorporate them into your organization.

Ideate

To create new business value you must be able to imagine or conceive of a new solution to an existing problem. These ideas can be derived from two different directions. First, I have a known problem and I am searching for a solution or secondly, let us look at what is available and uncover unique solutions for problems we were not actively looking for. The Event Portal enables learnings from both directions as without it, you do not have a central location to capture all of the events that are available, nor do you have a way to understand whether a given event stream solves your problem. The search and filter functionality enable the user to perform keyword searches which range from data level attributes to metadata within the description. This helps when you are aware of the problem and are looking for ideas of how to solve them with events. For example, let's say you're a Taxi company and are getting complaints about drivers speeding and you want to in real-time analyze the problem. You know the data has an attribute called "speed", but what event streams have that data? You can simply search for speed in the schemas section of the catalog, review the matches, decide which schema is of interest and navigate to which events actually capture the moment you want to analyze. But what if you don't have a specific problem and are simply wanting to think about the art of the possible? This is where browsing the event catalog can be key. Maybe you want to improve an area of your business and simply want to see what events are available in that area. Filter by that application domain and view the events like a menu of business capabilities that when combined could fundamentally transform that business area.

Once a new idea has been formulated, you can jump to the Design phase and start down the path of defining the new business capability in detail. Of course, in that phase you should consider making this new capability event-driven so that your colleges can ideate and solve more problems. The more events you have, the more ideation that can occur.

Organizational Enablement

Organizational changes happen all the time. How ready are you to take over another groups EDA implementation? How about enable new members on yours? What if your current architect were to resign, are you capturing everything you should be?

Tribal knowledge happens and is dangerous. The above organizational changes showcase the multitude of scenarios that can occur that leave the business in limbo and result in reverse engineering something that was already engineered. If you get into the habit and develop the muscle memory around designing/documenting and continuously validating your EDA, tribal knowledge is eliminated as its now available centrally and kept up to date. While most organizations believe they have a software development and governance process that will prevent this from happening, it is typically comprised of multiple conflicting sources of truth, none of which actually representing the current truth. This leads the team to constantly as the question "so how does this actually work" and wasting time trying to investigate vs simply using a tool that captures the information and ensures it matches reality.

So next time you are faced with the questions presented above, your answer should be an emphatic YES for your event-driven architecture.

Change Impact Analysis

Changes happen. The question is what is the effect and who is affected? In the synchronous world changes to an API of course may/will affect the clients, so changes are rolled out, clients notified, and changes implemented. The challenge in the EDA world is that consumers are decoupled from producers and vice/versa. In addition, the ripple effect can be large in that integrations though connectors and integration capabilities can move events between different groups which further casts a fog upon dependency management.

The AsyncAPI Initiative is an open source initiative that provides both the AsyncAPI specification to define your asynchronous APIs, and open source tools to enable developers to build and maintain an event-driven architecture.

The AsyncAPI Generator allows you to generate a wide variety of things from an AsyncAPI document depending on what template you choose. The latest list of templates can be found here

asyncapiSpecExample

asyncapiGeneratorTemplates

Install the AsyncAPI Generator

Now that we've defined the architecture for our use case in the Event Portal we're ready to write some code! But we don't want to have to write everything from scatch so we're going to use the AsyncAPI Generator

In order to use the AsyncAPI Generator we first need to install the CLI.

If you have the prequisites installed as defined earlier in the "What You'll Need" section you should be able to pop open your terminal and use the command below to install the CLI.

npm install -g @asyncapi/generator@0.53.1

Develop the ProcessPayment Microservice

🚕 🚖 🚕 🚖 🚕 🚖 🚕 🚖 🚕 🚖 🚕 🚖 🚕 🚖 🚕
On to developing the ProcessPayment App. As defined during the design sections of this codelab, we determined that this will be a microservice written using Java & Spring. We are going to use the Spring Cloud Stream framework to develop this microservice since it was created for the exact purpose of developing event-driven microservices. We'll also keep the business logic to a minimum to focus on the process of creating an event-driven microservice with AsyncAPI + Spring Cloud Stream and getting it running!

Generate the Code Skeleton

In the Solace Event Portal right click on the ProcessPayment application, Choose AsyncAPI, Choose YAML and click Download

processPaymentAsyncapi

Let's add a few of the template's configuration options to the downloaded AsyncAPI document.

✅ After adding those configuration options your channels section of the AsyncAPI document should look like the image below.

channels:
  'taxinyc/backoffice/payment/charged/v1/${payment_status}/${driver_id}/${passenger_id}':
    publish:
      x-scs-function-name: processPayment
      message:
        $ref: '#/components/messages/PaymentCharged'
  'taxinyc/ops/ride/updated/v1/${ride_status}/${driver_id}/${passenger_id}/${current_latitude}/${current_longitude}':
    subscribe:
      x-scs-function-name: processPayment
      x-scs-destination: test/taxinyc/PaymentProcessorQueue
      message:
        $ref: '#/components/messages/RideUpdated'

🚀 Our AsyncAPI document is now ready to generate the actual code so go over to your terminal and enter the command in the code snippet below.

Note the different pieces of the command:

✅ After running the command you should see output that ends with where you can find your generated files.

Done! ✨
Check out your shiny new generated files at /private/tmp/codelab/ProcessPayment.

Import and Explore the Generated Project

The generated project is a Maven project so head over to your IDE and import the project so we can add our business logic. Once imported you should see something like the image below.
projectsetup2

A few notes on the project:

Subscribe to dropoff events

As of the writing of this codelab, dynamic topics are not yet supported by the Event Portal or the AsyncAPI Code Generator template. Because our Taxis are publishing their RideUpdate events to a dynamic topic structure of taxinyc/ops/ride/updated/v1/${ride_status}/${driver_id}/${passenger_id}/${current_latitude}/${current_longitude} we need to update the application.yml file to subscribe to only dropoff events. To do this change the queueAdditionalSubscriptions parameter value to taxinyc/ops/ride/updated/v1/dropoff/>

Publish to a personalized topic for uniqueness

Because there are potentially multiple people using a shared broker participating in this codelab at the same time we need to make sure we publish to a unique topic. Change your spring.cloud.stream.bindings.processPayment-out-0.destination to be test/taxinyc/<YOUR_UNIQUE_NAME>/ops/payment/charged/v1/accepted. Be sure to replace with your name or some unique field; and remember it for later!

✅ After updating the spring.cloud.stream portion of your application.yml file should look something like this:

spring:
  cloud:
    stream:
      function:
        definition: processPayment
      bindings:
        processPayment-out-0:
          destination: test/taxinyc/yourname/backoffice/payment/charged/v1/accepted
        processPayment-in-0:
          destination: test/taxinyc/ProcessPaymentQueue
      solace:
        bindings:
          processPayment-in-0:
            consumer:
              queueAdditionalSubscriptions: 'taxinyc/ops/ride/updated/v1/dropoff/>'

Fill in the Business Logic

Obviously in the real world you'd have more complex business logic but for the sake of showing simplicity we're just going to log the RideUpdated events as they're received and create a new PaymentCharged event for each.

Open the Application.java file and modify the processPayment method to log the events. When you're done it should look something like the code below.

@Bean
public Function<RideUpdated, PaymentCharged> processPayment() {
	return rideUpdated -> {
		logger.info("Received Ride Updated Event:" + rideUpdated);
		//TODO Process Payment
		PaymentCharged pc = new PaymentCharged();
		pc.setRideId(rideUpdated.getRideId());
		pc.setAmountCharged(rideUpdated.getMeterReading());
		pc.setPaymentStatus("accepted");
		pc.setPaymentChargedId(UUID.randomUUID().toString());
		pc.setInvoiceSystemId("PSG-" + RandomUtils.nextInt());
	    pc.setInformationSource("ProcessPayment Microservice");
		pc.setTimestamp(Instant.now().toString());
		pc.setEntityType("Driver");
		logger.info("Created PaymentCharged Event:" + pc);
		return pc;
	};
}

That's it! The app development is complete.

🚀🚀🚀 Was that simple enough for you!? 🚀🚀🚀

Run the app!

Now that our app has been developed let's run it!

If your IDE has support for Spring Boot you can run it as a Spring Boot App.

Or run it from the terminal by navigating to the directory with the pom and running the mvn clean spring-boot:run command.

Once running you should see that for each RideUpdated event that is received a PaymentCharged Event is created which is being published back out onto the broker for downstream apps to consume. The output should look something like the below.

2020-11-12 14:25:54.451  INFO 97106 --- [pool-2-thread-1] org.taxi.nyc.Application                 : Received Ride Updated Event:RideUpdated [ rideId: f3ce97cb-e2df-4ed2-bb07-ab6afe9db629 heading: 168 latitude: 40.666628 passengerCount: 2 pointIdx: 1025 informationSource: RideDispatcher speed: 22 driver: Driver [ driverId: 16 rating: 2.37 lastName: Sawyer carClass: Coupe firstName: Miwa ] passenger: Passenger [ passengerId: 13817844 rating: 4.43 lastName: Bateman firstName: Chantal ] meterIncrement: 0.0198049 longitude: -73.85236 timestamp: 2020-11-12T14:25:54.206-05:00 meterReading: 20.3 rideStatus: dropoff ]
2020-11-12 14:25:54.453  INFO 97106 --- [pool-2-thread-1] org.taxi.nyc.Application                 : Created PaymentCharged Event:PaymentCharged [ rideId: f3ce97cb-e2df-4ed2-bb07-ab6afe9db629 entityType: Driver amountCharged: 20.3 driver: null paymentChargedId: 59d3caed-cad1-438b-9e9a-b37b8660efe7 passenger: null paymentStatus: accepted invoiceSystemId: PSG-616368280 informationSource: ProcessPayment Microservice timestamp: 2020-11-12T19:25:54.452Z ]

🤯🤯 The Microservice is now Running, connected to the Solace Event Broker and processing events! 🤯🤯

Develop the InvoiceSystem Python App

🚕 🚖 🚕 🚖 🚕 🚖 🚕 🚖 🚕 🚖 🚕 🚖 🚕 🚖 🚕
On to developing the InvoiceSystem python app that we previously designed. We are going to be using the Python Paho library to communicate with our event broker and will leverage the Python Paho AsyncAPI Generator Template to bootstrap our app creation.

Generate the Code Skeleton

In the Solace Event Portal right click on the InvoiceSystem, Choose AsyncAPI, Choose YAML and click Download

invoiceSystemAsyncapi

🚀 Our AsyncAPI document is now ready to generate the actual code so go over to your terminal and enter the command in the code snippet below.

Note the different pieces of the command:

✅ After running the command you should see output that ends with where you can find your generated files.

Done! ✨
Check out your shiny new generated files at /private/tmp/codelab/InvoiceSystem.

Import and Explore the Generated Project

The generated project is a Maven project so head over to your IDE and import the project so we can add our business logic. Once imported you should see something like the image below.
projectsetup2

A few notes on the project:

Subscribe to PaymentCharged events

As of the writing of this codelab, dynamic topics are not yet supported by the Event Portal or the AsyncAPI Code Generator template. Because our ProcessPayment microservice is publishing the PaymentCharged events to a dynamic topic structure of taxinyc/backoffice/payment/charged/v1/${payment_status}/${driver_id}/${passenger_id}
we need to update our subscription to subscribe to all PaymentCharged events no matter their payment_status, driver_id or passenger_id. To do this change the subscription to taxinyc/backoffice/payment/charged/v1/#

Fill in the Business Logic

That's it! The app development is complete.

🚀🚀🚀 Was that simple enough for you!? 🚀🚀🚀

Run the app!

Now that our app has been developed let's run it!

If your IDE has support for Spring Boot you can run it as a Spring Boot App.

Or run it from the terminal by navigating to the directory with the pom and running the mvn clean spring-boot:run command.

🤯🤯 The Python app is now Running, connected to the Solace Event Broker and processing events! 🤯🤯

AsyncAPI Code Generators

Generate Custom Code

Since the AsyncAPI Specification provides a machine readable way to define your Asynchronous applications it allows for the creation of custom code generators. The easiest way to likely do this is to leverage the tooling that the AsyncAPI Initiative has already put in place and create a new template for the AsyncAPI Generator

Use an Integration Platforms

Dell Boomi Connector

✅ The Solace PubSub+ Event Portal is an excellent tool to design and visualize your Event-Driven Architecture, discover what events exist, collaborate with your team and kickstart development via exporting of AsyncAPI documents.
✅ AsyncAPI Generator templates allow developers to consistently create event-driven applications by generating code skeletons that are pre-wired with the events and channels defined in the AsyncAPI documents.
✅ < Fill IN TAKEAWAY 3>

Soly Image Caption

Thanks for participating in this codelab! Let us know what you thought in the Solace Community Forum! If you found any issues along the way we'd appreciate it if you'd raise them by clicking the Report a mistake button at the bottom left of this codelab.