Most organizations are adopting an event-driven architecture (EDA) to compete in a world where customer satisfaction requires real-time outcomes.
In this code lab we'll build and expand your toolbox by learning how an Event Portal, paired with industry standard specifications and frameworks, enable a smooth journey to bring your EDA from initial architecture and design to code running in production while also setting your team up for success as the business needs, architecture and applications themselves are enhanced over time.
Throughout this workshop we will get hands on and talk about:
PubSub+ Event Portal
AsyncAPI
Positive : Artifacts created or used throughout this codelab can be found in this Github repo
🛠 This page covers the setup needed to perform this codelab. 🛠
✅ Install instructions available here
node -v
)npm -version
)We'll install the generator itself later 👍
✅ Spring Cloud Stream just requires Java and Maven to use 🚀
java -version
)mvn -version
) brew install maven
✅ There are only a few requirements for the Node.js steps!
node -v
)✅ The credentials below are for a public event feed found on the Solace feed Marketplace that we'll use during this codelab.
tcp://taxi.messaging.solace.cloud:55555
mqtt://taxi.messaging.solace.cloud:8883
nyc-modern-taxi
public-taxi-user
iliketaxis
✅ Note that this client-username has permissions to subscribe to taxinyc/>
and test/taxinyc/>
and permissions to publish to test/taxinyc/>
✅ If you already have a Solace Cloud account just login, otherwise please sign-up for a free Solace Cloud Account using this link. Note that no credit card is required. You will receive an email to activate the account and will then be prompted to start the free trial.
✅ Download the Application Domain export file: EventPortalExport_Initial.json
You can download the file via curl or by cloning the git repo
curl -k -XGET https://raw.githubusercontent.com/Mrc0113/ep-design-workshop/main/EventPortalExport_Initial.json -o EventPortalExport_Initial.json
OR
git clone https://github.com/Mrc0113/ep-design-workshop.git
✅ Inside your logged into Solace Cloud Account navigate to the Event Portal Designer by clicking "Designer" in the menu on the left.
✅ Then import the previously downloaded Application Domain file by clicking the Import
button at the top right of the Designer and importing the file.
🚀 Setup complete! Let's get going! 🚀
You are a member of the engineering team at the NYC Modern Taxi Co, a fictional taxi cab company based in New York City. Your team is playing from behind and racing to catch up with technology innovation introduced to the industry by Rideshare competitors such as Uber and Lyft. In order for the company to survive and eventually thrive your team has convinced the board that transforming the companies' IT systems is of utmost importance. Your team has done its research and determined that moving to an Event-Driven Architecture is essential to future rapid innovation and has already kicked this initiative off by deploying a Solace Event Mesh and updating the taxi fleet to stream real-time events that include ride and location information. We know what the fleet is up to! Now it's time to start to continually improve and provide a world class customer experience.
In order to react in a real-time manner the team has decided that we want to process the updates as they stream in from the fleet of taxis instead of putting them directly into a datastore and then having to retrieve them to do processing later. To prototype this work, you'll see a high level design in the diagram below. Since we already have the taxi fleet streaming their updates into our PubSub+ Event Mesh we need to do the following:
Positive : The dataset you will be using in this lab originally comes from the NYC Taxi & Limousine Commission's open data release of more than a billion taxi ride records. Google then extended one week worth of data (3M taxi rides) from their original pickup and drop-off points into full routes in order to simulate a fleet of taxis roaming the streets of NYC. Solace is streaming this data over Solace PubSub+ for you to analyze and process.
Terms of Use: This dataset is publicly available for anyone to use under the following terms provided by the Dataset Source — https://data.cityofnewyork.us/ — and is provided "AS IS" without any warranty, express or implied, from Solace. Solace disclaims all liability for any damages, direct or indirect, resulting from the use of the dataset.
Before we dive deeper, let's ensure we are all aligned with terminology of the objects and concepts we will use in PubSub+ Event Portal.
An application domain represents a namespace where applications, events, and schemas can live. Within this namespace, you can create a suite of applications, events, and schemas that are independent of other application domains. In our NYC Taxi use case we introduced earlier, we may group applications into different domains, for ex. we may have a domain for our rideshare apps and services, one for our back-office apps where invoicing and background checks are being processed, and maybe another domains for analytics where we group apps that are responsible for analyzing the successful operation of our rideshare services.
Events are an important part of the Event Portal. Think of an event as a concept of the publish-subscribe (pub/sub) architectural pattern. Topics are used to route data or events (in the form of messages) between distributed applications, often using a message broker or an event broker.
A Solace topic and an Apache Kafka topic might seem fundamentally the same, but there are quite a few differences between them.
Here are some examples from our use case:
Kafka Topics:
Solace Topics:
In simple terms, a schema represents the contract to describe the payload of an event. Producers and consumers of an event can trust that the event's payload matches the schema definition assigned to that event. Schemas define a type of payload through JSON, AVRO, XML, Binary, or Text. JSON, AVRO, and XML schemas have content that describes each property of the schema.
In our use case all events are in JSON Schema format.
An application represents a piece of software that produces and consumes events. Applications connect to the event broker in an event-driven architecture and communicate with other applications via events. A single application represents a class of applications that are running the same code base; therefore, a Kafka consumer group can be associated with an Application object in the Event Portal.
It is important to consider how your enterprise is organized so that it can be decomposed using the Application Domain construct. An Application Domain provides the ability to organize and decompose an enterprise into logical groupings. These groupings could be based on-line of business, related functional capabilities or based on team dynamics. The benefits of doing this includes:
The topic of which an event is addressed seems like a pretty simple decision, but in reality, it can result in some negative consequences if not planned in advance. A topic is more than an address, it is metadata that describes the event and can be used for several purposes such as routing, access control and versioning. Thus, it is important to properly govern and manage the topic structure. Regardless of your broker type, it is a good practice to make topics structured and hierarchical the same way a RESTful Resource uses hierarchical addressing. In other words we want to produce hierarchical topics that rank from least specific to most specific.
The event topic structure has two parts:
Domain/ObjectType/Verb/Version/
Positive : Complete Event Topic Format: Putting together an Event Topic Root and Event Topic Properties creates an event topic that describes the event with a series of fields from least specific to most specific.
Domain/ObjectType/Verb/Version/Locality/SourceID/ObjectID
For more information about topic best practices, review the [Topic Architecture Best Practices Guide] (https://docs.solace.com/Best-Practices/Topic-Architecture-Best-Practices.htm)
There are multiple Event Exchange Patterns (EEP) that should be considered when using EDA:
Positive : The advantage of this pattern however is that the data is smaller and can thus reduce latency and bandwidth when important. In general, the source of that event should be the single authoritative source for all published attributes.
Negative : The disadvantage to this pattern is it increases the latency of the interaction as all the data is not available within the event and puts more complexity on the client and its behavior.
Negative : The challenge in this case is that the publishing application may not be the authoritative source of all attributes published. Additionally, the event may become large and increase latency/decrease performance.
Positive : The benefit however is that decoupling has been achieved in that it will support a variety of use cases and the publisher does not need to be aware of the client's usage of the data.
For more information about discovering, organizing and enhancing your Event-Driven Architecture, review [Understand, Manage, and Enhance Your Event-Driven Architecture Lifecycle ] (https://docs.solace.com/Cloud/Event-Portal/get-started-event-portal-user-scenario.htm)
By designing a new event-driven application or extending your event-driven architecture, you're able to deliver new real-time business capabilities in a decoupled and reusable fashion. There are however several key elements which should be considered when designing events, schemas and applications including topic best practices, options for exchanging event data and sharing/visibility rules. Considering these things early will put you on the road to success and enable better reusability down the road.
Now that you're familiar with the use case 🚕 🚖 🚕 and you've imported the application domain into the Event Portal, let's update our Event-Driven Architecture (EDA).
Lets say that your tasked with working within the Back Office team (where the cool kids all work) and are asked to architect the way in which we will charge our passengers for their rides and if the passenger is part of a commercial account, send to our Invoicing System. This is composed of 4 steps
So essentially we need to consider, is there a business event that would help us trigger on the moment when the ride has been completed?
Positive : Event-Driven Ideation: To create new business value you must be able to imagine or conceive of a new solution to an existing problem. These ideas can be derived from two different directions. First, I have a known problem, and I am searching for a solution or secondly, let us look at what is available and uncover unique solutions for problems we were not actively looking for. The Event Portal enables learnings from both directions as without it, you do not have a central location to capture all the events that are available, nor do you have a way to understand whether a given event stream solves your problem. The search and filter functionality enable the user to perform keyword searches which range from data level attributes to metadata within the description.
Next we should decide what we want the data to look like once we have processed a payment.
{
"$schema": "http://json-schema.org/draft-07/schema",
"$id": "http://example.com/example.json",
"type": "object",
"title": "The root schema",
"description": "The root schema comprises the entire JSON document.",
"default": {},
"examples": [
{
"payment_charged_id": "23232323",
"timestamp": "2020-06-03T16:51:47.29612-04:00",
"information_source": "ProcessPayment",
"payment_status": "accepted",
"invoice_system_id": "PSG-32923",
"amount_charged": 12.32,
"ride_id": "545496c5-a334-4344-9662-efde68c0b98a",
"entity_type": "Driver",
"driver": {
"driver_id": 1234132,
"first_name": "Frank",
"last_name": "Smith",
"rating": 4,
"car_class": "SUV"
},
"passenger": {
"passenger_id": 2345243,
"first_name": "Jesse",
"last_name": "Menning",
"rating": 2
}
}
],
"required": [
"payment_charged_id",
"timestamp",
"information_source",
"payment_status",
"invoice_system_id",
"amount_charged",
"ride_id",
"entity_type",
"driver",
"passenger"
],
"properties": {
"payment_charged_id": {
"$id": "#/properties/payment_charged_id",
"type": "string",
"title": "The payment_charged_id schema",
"description": "An explanation about the purpose of this instance.",
"default": "",
"examples": ["23232323"]
},
"timestamp": {
"$id": "#/properties/timestamp",
"type": "string",
"title": "The timestamp schema",
"description": "An explanation about the purpose of this instance.",
"default": "",
"examples": ["2020-06-03T16:51:47.29612-04:00"]
},
"information_source": {
"$id": "#/properties/information_source",
"type": "string",
"title": "The information_source schema",
"description": "An explanation about the purpose of this instance.",
"default": "",
"examples": ["ProcessPayment"]
},
"payment_status": {
"$id": "#/properties/payment_status",
"type": "string",
"title": "The payment_status schema",
"description": "An explanation about the purpose of this instance.",
"default": "",
"examples": ["accepted"]
},
"invoice_system_id": {
"$id": "#/properties/invoice_system_id",
"type": "string",
"title": "The invoice_system_id schema",
"description": "An explanation about the purpose of this instance.",
"default": "",
"examples": ["PSG-32923"]
},
"amount_charged": {
"$id": "#/properties/amount_charged",
"type": "number",
"title": "The amount_charged schema",
"description": "An explanation about the purpose of this instance.",
"default": 0,
"examples": [12.32]
},
"ride_id": {
"$id": "#/properties/ride_id",
"type": "string",
"title": "The ride_id schema",
"description": "An explanation about the purpose of this instance.",
"default": "",
"examples": ["545496c5-a334-4344-9662-efde68c0b98a"]
},
"entity_type": {
"$id": "#/properties/entity_type",
"type": "string",
"title": "The entity_type schema",
"description": "An explanation about the purpose of this instance.",
"default": "",
"examples": ["Driver"]
},
"driver": {
"$id": "#/properties/driver",
"type": "object",
"title": "The driver schema",
"description": "An explanation about the purpose of this instance.",
"default": {},
"examples": [
{
"driver_id": 1234132,
"first_name": "Frank",
"last_name": "Smith",
"rating": 4,
"car_class": "SUV"
}
],
"required": [
"driver_id",
"first_name",
"last_name",
"rating",
"car_class"
],
"properties": {
"driver_id": {
"$id": "#/properties/driver/properties/driver_id",
"type": "integer",
"title": "The driver_id schema",
"description": "An explanation about the purpose of this instance.",
"default": 0,
"examples": [1234132]
},
"first_name": {
"$id": "#/properties/driver/properties/first_name",
"type": "string",
"title": "The first_name schema",
"description": "An explanation about the purpose of this instance.",
"default": "",
"examples": ["Frank"]
},
"last_name": {
"$id": "#/properties/driver/properties/last_name",
"type": "string",
"title": "The last_name schema",
"description": "An explanation about the purpose of this instance.",
"default": "",
"examples": ["Smith"]
},
"rating": {
"$id": "#/properties/driver/properties/rating",
"type": "number",
"title": "The rating schema",
"description": "An explanation about the purpose of this instance.",
"default": 0,
"examples": [4]
},
"car_class": {
"$id": "#/properties/driver/properties/car_class",
"type": "string",
"title": "The car_class schema",
"description": "An explanation about the purpose of this instance.",
"default": "",
"examples": ["SUV"]
}
},
"additionalProperties": true
},
"passenger": {
"$id": "#/properties/passenger",
"type": "object",
"title": "The passenger schema",
"description": "An explanation about the purpose of this instance.",
"default": {},
"examples": [
{
"passenger_id": 2345243,
"first_name": "Jesse",
"last_name": "Menning",
"rating": 2
}
],
"required": ["passenger_id", "first_name", "last_name", "rating"],
"properties": {
"passenger_id": {
"$id": "#/properties/passenger/properties/passenger_id",
"type": "integer",
"title": "The passenger_id schema",
"description": "An explanation about the purpose of this instance.",
"default": 0,
"examples": [2345243]
},
"first_name": {
"$id": "#/properties/passenger/properties/first_name",
"type": "string",
"title": "The first_name schema",
"description": "An explanation about the purpose of this instance.",
"default": "",
"examples": ["Jesse"]
},
"last_name": {
"$id": "#/properties/passenger/properties/last_name",
"type": "string",
"title": "The last_name schema",
"description": "An explanation about the purpose of this instance.",
"default": "",
"examples": ["Menning"]
},
"rating": {
"$id": "#/properties/passenger/properties/rating",
"type": "number",
"title": "The rating schema",
"description": "An explanation about the purpose of this instance.",
"default": 0,
"examples": [2]
}
},
"additionalProperties": true
}
},
"additionalProperties": true
}
So now that we have constructed the payload format for the PaymentCharged event, it is time to design the event itself. What's involved? Well we need to apply our best practices as it comes to the Topic name!
taxinyc/backoffice/
taxinyc/backoffice/payment/charged/v1/{payment_status}/{driver_id}/{passenger_id}
{payment_status}, {driver_id} and {passenger_id}
as variablesNow for the fun part! We need to design the event-driven interface of the ProcessPayment Application. This is pretty easy as it has one input which triggers a single output.
Positive : Pro Tip!: If you wanted to develop/implement this application you could right-click on the ProcessPayment Application in graph and export an AsyncAPI Document that could be used to generate code!
Remember back to our use case... We have designed how we process payment but still have to deal with invoicing customers when the payment_status says to invoice. Therefore, our plan is to create an application that integrates with our invoicing system.
Getting reuse of your events is an important part of proving return on investment (ROI) and also enables other applications and teams to integrate with realtime data.
In this scenario we will act as though we are members of the "Ops" team (they are not as cool as us back office kids, but oh well). They have a use case that Payment charged events should go to the Rider Mobile Application. Lets make it happen!
You should now see the relationship on the graph where we are subscribed to the PaymentCharged event and the dependency on the Back Office App domain!
Positive : Change Impact Analysis: Changes happen. The question is what is the effect and who is affected? In the synchronous world changes to an API of course may/will affect the clients, so changes are rolled out, clients notified, and changes implemented. The challenge in the EDA world is that consumers are decoupled from producers and vice/versa. In addition, the ripple effect can be large in that integrations though connectors and integration capabilities can move events between different groups which further casts a fog upon dependency management. The Event Portal enables you to navigate the relationships you just designed and understand impact.
💡 Know your Audience
The events which you have are used to enable real-time collaboration between systems and solve a problem for a specific industry and organization. These events are integrated into applications by software developers/engineers, but they are not all the same and can be decomposed into:
💡 Capture Business Point of View and Moment
💡 Technical Requirements
💡 Link to other References
💡 Provide Examples
💡 Terms of Use
💡 Tags
Positive : Organizational Enablement: Organizational changes happen all the time. How ready are you to take over another groups EDA implementation? How about enable new members on yours? What if your current architect were to resign, are you capturing everything you should be? Tribal knowledge happens and is dangerous. The above organizational changes showcase the multitude of scenarios that can occur that leave the business in limbo and result in reverse engineering something that was already engineered. If you get into the habit and develop the muscle memory around designing/documenting and continuously validating your EDA, tribal knowledge is eliminated as its now available centrally and kept up to date. While most organizations believe they have a software development and governance process that will prevent this from happening, it is typically comprised of multiple conflicting sources of truth, none of which actually representing the current truth. This leads the team to constantly as the question "so how does this actually work" and wasting time trying to investigate vs simply using a tool that captures the information and ensures it matches reality.
Remember how we did not provide any description or tags for the Events and Applications we created before? Well, lets go in and follow our best practices fixing this. Lets enhance the documentation of the PaymentCharged Event
Description of Business Moment
Overview:
The PaymentCharged Event exists in order to notify other systems that we have attempted to charge the passenger. There are also other states such as:
accepted - customer credit card on file has been charged
declined - customer credit card on file has been declined
org - the customer is part of a B2B org and does not provide automated payment
Technical Requirements
Format: JSON
Security Level: PCI
Terms of Use
N/A
Lets enhance the documentation of the ProcessPayment Application and put our Documentation Best Practices to work!
Description of Business Capability
Overview:
The ProcessPayment application solely exists in order to monitor for when Passenger Rides are completed such that final billing can be performed against the passengers credit card. Because this application will need to look up the passenger's billing information it is important that security be taken into account as it will need to be PCI compliant.
Upon successful payment, the application shall emit an event to signify that payment has happened.
Technical Requirements
Java Version: OpenJDK 11.0.4
Spring Cloud Version: Hoxton.SR8
Number of Instances: 1
Cloud: AWS us-east
Security Level: PCI
Event Broker Profile: Solace
Source Code Repository
github repo
Terms of Use
N/A
The AsyncAPI Initiative is an open source initiative that provides both the AsyncAPI specification to define your asynchronous APIs, and open source tools to enable developers to build and maintain an event-driven architecture.
Positive : Learn More in the AsyncAPI Docs
The AsyncAPI Generator allows you to generate a wide variety of things from an AsyncAPI document depending on what template you choose. The latest list of templates can be found here
Now that we've defined the architecture for our use case in the Event Portal we're ready to write some code! But we don't want to have to write everything from scratch, so we're going to use the AsyncAPI Generator
In order to use the AsyncAPI Generator we first need to install the CLI.
If you have the prerequisites installed as defined earlier in the "What You'll Need" section you should be able to pop open your terminal and use the command below to install the CLI.
npm install -g @asyncapi/generator
Negative : Note that the AsyncAPI project is continuously updated so if you previously installed the generator you can also use the command above to update to the latest.
🚕 🚖 🚕 🚖 🚕 🚖 🚕 🚖 🚕 🚖 🚕 🚖 🚕 🚖 🚕 On to developing the ProcessPayment App. As defined during the design sections of this codelab, we determined that this will be a microservice written using Java & Spring. We are going to use the Spring Cloud Stream framework to develop this microservice since it was created for the exact purpose of developing event-driven microservices. We'll also keep the business logic to a minimum to focus on the process of creating an event-driven microservice with AsyncAPI + Spring Cloud Stream and getting it running!
Open the NYC Modern Taxi Co - Back Office
Application Domain in the Solace Event Portal, right-click on the ProcessPayment application, Choose AsyncAPI, Choose YAML and click Download
Positive : The AsyncAPI Java Spring Cloud Stream Generator Template includes many Configuration Options that allow you to change what the generated code will look like.
If you want a pre-populated file with changes and additions, you can download using the following command.
curl -k -XGET https://raw.githubusercontent.com/Mrc0113/ep-design-workshop/main/ProcessPayment.yml -o ProcessPayment.yaml
After downloading, directly go to Code Generation step.
Alternatively, you can follow the instruction and make changes to the AsyncAPI document.
Let's add a few of the template's configuration options to the downloaded AsyncAPI document.
x-scs-function-name: processPayment
immediately after the subscribe operation and the publish operation under our two channels. By adding this you are telling the generator the name of the function you would like to handle events being exchanged and by adding the same function-name for both subscribe and publish operation you are saying you want them handled by the same function!x-scs-destination: test/taxinyc/PaymentProcessorQueue
immediately after the subscribe operation. By adding this and using the Solace binder you are specifying the durable queue name if you're using a Consumer Group, or part of the temporary queue name if you're not. This will also add a topic subscription matching the channel specified in the AsyncAPI document to the queue.✅ After adding those configuration options your channels section of the AsyncAPI document should look like the image below.
channels:
'taxinyc/backoffice/payment/charged/v1/{payment_status}/{driver_id}/{passenger_id}':
subscribe:
x-scs-function-name: processPayment
x-scs-destination: test/taxinyc/PaymentProcessorQueue
message:
$ref: '#/components/messages/PaymentCharged'
....
....
'taxinyc/ops/ride/updated/v1/{ride_status}/{driver_id}/{passenger_id}/{current_latitude}/{current_longitude}':
publish:
x-scs-function-name: processPayment
message:
$ref: '#/components/messages/RideUpdated'
Negative : To work-around an issue in the AsncAPI code generator, the $id
fields need to be removed. You can use the following command in the terminal to accomplish this.sed -i '/$id:/d' ProcessPayment.yaml
🚀 Our AsyncAPI document is now ready to generate the actual code so go over to your terminal and enter the command in the code snippet below.
Note the different pieces of the command:
ag
is the AsyncAPI Generator command-o
is the output directory-p
allows you to specify parameters defined for the template you're usingbinder
is the Spring Cloud Stream binder you wish to use, in this case SolaceartifactId
& groupId
configure Maven params of the same namesjavaPackage
specifies the Java Package to place the generated classes intohost
, username
, password
and msgVpn
allow you to set binder connection information (use the connection detailed noted from the previous step).@asyncapi/java-spring-cloud-stream-template
is the AsyncAPI generator template that we are using.ag -o ProcessPayment -p binder=solace -p dynamicType=header -p artifactId=ProcessPayment -p groupId=org.taxi.nyc -p javaPackage=org.taxi.nyc -p host=taxi.messaging.solace.cloud:55555 -p username=public-taxi-user -p password=iliketaxis -p msgVpn=nyc-modern-taxi ProcessPayment.yaml @asyncapi/java-spring-cloud-stream-template
✅ After running the command you should see output that ends with where you can find your generated files.
Done! ✨
Check out your shiny new generated files at /private/tmp/codelab/ProcessPayment.
The generated project is a Maven project so head over to your IDE and import the project, so we can add our business logic. Once imported you should see something like the image below.
Negative : If you are wondering why the POJO name has a character 1 – it is because there already exists a schema with the same name.
A few notes on the project:
org.taxi.nyc
package that we specified.PaymentCharged
and RideUpdated
POJOs were generated from the schemas defined in our AsyncAPI document and includes getters/setters/toString/etc.Application.java
contains a processPayment
method which is a Function
that takes in a RideUpdated
POJO and returns a PaymentCharged
POJO.application.yml
file contains the Spring configuration which tells our app how to connect to Solace using the SCSt binder as well as which message channels to bind our methods to.pom.xml
file contains the dependencies needed for the microservice. These include the solace-cloud-starter-stream-solace
dependency which allows you to use the Solace SCSt. Binder.Negative : You can use a Java IDE like Eclipse or Spring Tool Suite to work with the generated Spring Boot application. Alternatively, you can also use a simple text editor like Visual Source Code or other text editors to make changes to the files.
There are certain updates required to the applicaion.yaml file to reflect destination names and subscriptions. You can use the following block of code and replace the content of application.yml file in the Spring project or manually make changes by following steps in the Changes to application.yml section.
spring:
cloud:
function:
definition: processPayment
stream:
bindings:
processPayment-out-0:
destination: 'test/taxinyc/YOUR_NAME/backoffice/payment/charged/v1/accepted'
processPayment-in-0:
destination: test/taxinyc/PaymentProcessorQueue
solace:
bindings:
processPayment-in-0:
consumer:
queueAdditionalSubscriptions: "taxinyc/ops/ride/updated/v1/dropoff/>"
binders:
solace-binder:
type: solace
environment:
solace:
java:
host: 'taxi.messaging.solace.cloud:55555'
msgVpn: nyc-modern-taxi
clientUsername: public-taxi-user
clientPassword: iliketaxis
logging:
level:
root: info
org:
springframework: info
After updating the file, directly go to Implement the Business Logic step.
If you choose to make manual changes, please follow the instructions here.
Dynamic topics (topics with variables enclosed by {,} characters) is not supported by the AsyncAPI Code Generator in cloud stream bindings.
Let us make the following changes to the application.yml file.
test/taxinyc/YOUR_UNIQUE_NAME/backoffice/payment/charged/v1/accepted
Be sure to replace YOUR_UNIQUE_NAME with your name or some unique field; and remember it for later!. Because there are potentially multiple people using a shared broker participating in this codelab at the same time we need to make sure we publish to a unique topic.
solace:
bindings:
processPayment-in-0:
consumer:
queueAdditionalSubscriptions: "taxinyc/ops/ride/updated/v1/dropoff/>"
Now we have the ProcessPayment application subscribing to a dynamic topic taxinyc/ops/ride/updated/v1/dropoff/>
and publishing to test/taxinyc/YOUR_UNIQUE_NAME/backoffice/payment/charged/v1/accepted
topic.
Positive : Note that the >
symbol, when placed by itself as the last level in a topic, is a multi-level wildcard in Solace which subscribes to all events published to topics that begin with the same prefix. Example: animals/domestic/>
matches animals/domestic/cats
and animals/domestic/dogs
. More wildcard info, including a single level wildcard, can be found in docs
Obviously in the real world you'd have more complex business logic but for the sake of showing simplicity we're just going to log the RideUpdated events as they're received and create a new PaymentCharged event for each.
Open the Application.java file and modify the processPayment
method to build and publish PaymentCharged event. Copy the following code block and replace the processPayment
bean. Upon receiving a RideUpdated message, thie bean function constructs a PaymentCharged message and publishes to the unique topic (distinguished by your unique name).
@Bean
public Function<RideUpdated1, Message<PaymentCharged>> processPayment() {
return rideUpdated -> {
logger.info("Received Ride Updated Event:" + rideUpdated);
// Process Payment
PaymentCharged pc = new PaymentCharged();
pc.setRideId(rideUpdated.getRideId());
pc.setAmountCharged(rideUpdated.getMeterReading());
pc.setPaymentStatus("accepted");
pc.setPaymentChargedId(UUID.randomUUID().toString());
pc.setInvoiceSystemId("PSG-" + RandomUtils.nextInt());
pc.setInformationSource("ProcessPayment Microservice");
pc.setTimestamp(Instant.now().toString());
pc.setEntityType("Driver");
org.taxi.nyc.RideUpdated1.Driver driver = rideUpdated.getDriver();
pc.setDriver(new org.taxi.nyc.PaymentCharged.Driver(
driver.getDriverId(), driver.getRating(), driver.getLastName(),
driver.getCarClass(), driver.getFirstName()));
org.taxi.nyc.RideUpdated1.Passenger passenger = rideUpdated.getPassenger();
pc.setPassenger(new org.taxi.nyc.PaymentCharged.Passenger(
passenger.getPassengerId(), passenger.getRating(),
passenger.getLastName(), passenger.getFirstName()
));
logger.info("Created PaymentCharged Event:" + pc);
String topic = "test/taxinyc/YOUR_UNIQUE_NAME/backoffice/payment/charged/v1/accepted";
Message<PaymentCharged> message = MessageBuilder
.withPayload(pc)
.setHeader(BinderHeaders.TARGET_DESTINATION, topic)
.build();
return message;
};
}
Be sure to replace YOUR_UNIQUE_NAME with your name or some unique field in the following line of code:
String topic = "test/taxinyc/YOUR_UNIQUE_NAME/backoffice/payment/charged/v1/accepted";
Negative : After updating the code, ensure that all "Instant cannot be resolved" errors due to missing imports. Or simply insert the following import statements at the top of the file.
import java.time.Instant;
import java.util.UUID;
import org.apache.commons.lang.math.RandomUtils;
That's it! The app development is complete.
Now that our app has been developed let's run it!
If your IDE has support for Spring Boot you can run it as a Spring Boot App.
Or run it from the terminal by navigating to the directory with the pom and running the following command in the terminal.
mvn clean spring-boot:run
Negative : If you get an error that says something like Web server failed to start. Port XXXX was already in use.
then change the server.port
value in application.yml
to an open port.
Once running you should see that for each RideUpdated event that is received a PaymentCharged Event is created which is being published back out onto the broker for downstream apps to consume. The output should look something like the below.
2022-03-31 09:50:58.267 INFO 26107 --- [pool-4-thread-1] org.taxi.nyc.Application :
Received Ride Updated Event:RideUpdated1 [ rideId: 056b4e73-60d6-4e90-bd29-0261d5824b5c heading: 35 latitude: 40.7888 passengerCount: 1 pointIdx: 545 informationSource: RideDispatcher speed: 18 driver: Driver [ driverId: 325 rating: 2.89 lastName: Ferro carClass: Coupe firstName: Florence ] passenger: Passenger [ passengerId: 66574939 rating: 4.39 lastName: Cast firstName: Sally ] meterIncrement: 0.0317431 longitude: -73.974224 timestamp: 2022-03-31T00:20:58.184-04:00 meterReading: 17.3 rideStatus: dropoff ]
2022-03-31 09:50:58.270 INFO 26107 --- [pool-4-thread-1] org.taxi.nyc.Application :
Created PaymentCharged Event:PaymentCharged [ rideId: 056b4e73-60d6-4e90-bd29-0261d5824b5c entityType: Driver amountCharged: 17.3 driver: Driver [ driverId: 325 rating: 2.89 lastName: Ferro carClass: Coupe firstName: Florence ] paymentChargedId: 02917306-79ff-457c-b2bc-73a547bcfdbe passenger: Passenger [ passengerId: 66574939 rating: 4.39 lastName: Cast firstName: Sally ] paymentStatus: accepted invoiceSystemId: PSG-1656881013 informationSource: ProcessPayment Microservice timestamp: 2022-03-31T04:20:58.269Z ]
🤯🤯 The Microservice is now Running, connected to the Solace Event Broker and processing events! 🤯🤯
🚕 🚖 🚕 🚖 🚕 🚖 🚕 🚖 🚕 🚖 🚕 🚖 🚕 🚖 🚕 On to developing the InvoiceSystem application. We will be using the Node.js service that uses Hermes package to communicate with our event broker over MQTT. To do this we will leverage the Node.js AsyncAPI Generator Template to bootstrap our app creation. Note that MQTT is an open standard messaging protocol very popular in Internet of Things (IoT) world and is designed to be extremely lightweight and
Open the NYC Modern Taxi Co - Back Office
Application Domain in the Solace Event Portal, right-click on the InvoiceSystem, Choose AsyncAPI, Choose YAML and click Download
If you want a pre-populated file with changes and additions, you can download using the following command.
curl -k -XGET https://raw.githubusercontent.com/Mrc0113/ep-design-workshop/main/InvoiceSystem/asyncapi.yaml -o InvoiceSystem.yaml
After downloading, directly go to Code Generation step.
Alternatively, you can follow the instruction and make changes to the AsyncAPI document.
channels:
'test/taxinyc/YOUR_UNIQUE_NAME/backoffice/payment/charged/v1/accepted':
publish:
operationId: receivePaymentCharged
Be sure to replace YOUR_UNIQUE_NAME with your name or some unique field in the following line of code:
name
parameter next to the PaymentCharged
message. messages:
PaymentCharged:
name: PaymentCharged
servers:
production:
url: 'mqtt://taxi.messaging.solace.cloud:8883'
protocol: mqtt
🚀 Our AsyncAPI document is now ready to generate the actual code so go over to your terminal and enter the command in the code snippet below.
Note the different pieces of the command:
ag
is the AsyncAPI Generator command-o
is the output directory-p
is the parameter to specify server configuration to be utilized@asyncapi/nodejs-template
is the AsyncAPI generator template that we are using.ag InvoiceSystem.yaml @asyncapi/nodejs-template -o InvoiceSystem -p server=production
✅ After running the command you should see output that ends with where you can find your generated files.
Done! ✨
Check out your shiny new generated files at /private/tmp/codelab/InvoiceSystem.
The AsyncAPI Generator generated a nodejs project in the directory specified by the -o
parameter so head over to your favorite Editor/IDE and open it up. Once opened you should see something like the image below.
A few notes on the project:
test-taxinyc-YOUR_UNIQUE_NAME-backoffice-payment-charged-v1-accepted.js
in the src/api/handlers
directory contains callback function that will receive subscribed messages. This is the place where you can implement your business logic (message processing).common.yml
file in the config
directory contains broker and application details.In the common.yml in the config directory:
url
and protocol
fields.Before coding our nodejs app let's go ahead and put our credentials in place.
broker:
mqtt:
url: mqtts://taxi.messaging.solace.cloud:8883
topics: ["test/taxinyc/YOUR_UNIQUE_NAME/backoffice/payment/charged/v1/accepted"]
qos:
protocol: mqtts
retain:
subscribe: true
config:
username: "public-taxi-user"
password: "iliketaxis"
That's it! The app development is complete.
🚀🚀🚀 Was that simple enough for you!? 🚀🚀🚀
Now that our app has been developed let's run it!
# Go to the generated server
$ cd InvoiceSystem
# Build generated application
$ npm i
# Start server - Once running you should see PaymentCharged Events are received from the Broker.
# The output should look something like the below.
$ npm start
> invoice-system@0.0.1 start /Users/xxxx/gitsolace/work/InvoiceSystem
> node src/api/index.js
SUB Subscribed to test/taxinyc/YOUR_UNIQUE_NAME/backoffice/payment/charged/v1/accepted
InvoiceSystem 0.0.1 is ready!
🔗 MQTT adapter is connected!
← test/taxinyc/YOUR_UNIQUE_NAME/backoffice/payment/charged/v1/accepted was received:
{
driver: {
rating: 2.24,
driver_id: 77,
last_name: 'Van Zandt',
car_class: 'Coupe',
first_name: 'Pablo'
},
passenger: {
rating: 4.65,
passenger_id: 93731728,
last_name: 'Schlicht',
first_name: 'Aaron'
},
timestamp: '2022-03-31T04:23:34.055Z',
ride_id: 'effd0ef8-6358-4592-ac97-2be6173bfa4d',
entity_type: 'Driver',
amount_charged: 14.3,
payment_charged_id: '553ef9a9-dd18-4112-b79d-970156d529d2',
payment_status: 'accepted',
invoice_system_id: 'PSG-1557019248',
information_source: 'ProcessPayment Microservice'
}
🤯🤯 The Node.js app is now Running, connected to the Solace Event Broker receiving and logging events! 🤯🤯
You can create event driven applications in a wide variety of different options as shown here: ![APIs and Protocols] (img/Solace-PubSub-Platform-Diagram-1.png)
Since the AsyncAPI Specification provides a machine-readable way to define your Asynchronous applications it allows for the creation of custom code generators. The easiest way to likely do this is to leverage the tooling that the AsyncAPI Initiative has already put in place and create a new template for the AsyncAPI Generator
✅ Event Driven Architecture does not have to be hard if you understand some key fundamentals and follow best practices.
✅ The Solace PubSub+ Event Portal is an excellent tool to design, visualize and document your Event-Driven Architecture, discover what events exist, collaborate with your team and kick start development via exporting of AsyncAPI documents.
✅ AsyncAPI Generator templates allow developers to consistently create event-driven applications by generating code skeletons that are pre-wired with the events and channels defined in the AsyncAPI documents.
Thanks for participating in this codelab! Let us know what you thought in the Solace Community Forum! If you found any issues along the way we'd appreciate it if you'd raise them by clicking the Report a mistake button at the bottom left of this codelab.