Negative : Note that this CodeLab uses the Annotation based Spring Cloud Stream code which is effectively deprecated in Cloud Stream v3. Please see the Spring Cloud Stream Basics Codelab to learn how to leverage Spring Cloud Function which is preferred in v3+.

Using Java & Spring Cloud Streams (SCS) to create Event-Driven Applications with PubSub+ in PCF

story_section1

Positive : Developer Resources Spring Cloud Stream Project Home: https://spring.io/projects/spring-cloud-stream The latest version of Spring Clouds Streams as of the creation of this lab is Greenwich.SR1 (2.1.4) The Reference Guide for that current version is available here.

Negative : The SCS framework allows for building messaging-driven applications without having to explicitly write any code for publishing or receiving events. While many microservices-oriented applications today are based on synchronous, request/response interactions based on protocols such as HTTP or gRPC, the asynchronous, event-driven nature of communications using SCS allows for building highly scalable, efficient and responsive distributed systems that can run on-premise or in the cloud. When combined with the high-performance Solace PubSub+ Event Broker which can be deployed in virtually any environment, you can create powerful and flexible applications that support both hybrid and multi-cloud capabilities, all operating in real-time with high throughput and low latency.

Negative : Note that this CodeLab uses the Annotation based Spring Cloud Stream code which is effectively deprecated in Cloud Stream v3. Please see the Spring Cloud Stream Basics Codelab to learn how to leverage Spring Cloud Function which is preferred in v3+.

story_section2

Developer IDE & Code Access

IDE Setup

The recommended IDE for the workshop is Spring Tools Suite (STS) Download Here. STS comes with some niceties, such as autodeploy, when used with spring-boot-devtools. Participants can of course use another IDE if preferred. It is also recommended that you begin the workshop with an empty STS workspace to avoid any unforeseen issues with existing projects/configurations/code.

Required libraries:

Code Access

$ git clone -b pcf https://github.com/SolaceTraining/solace-workshop-scs.git
$ git clone -b pcf git@github.com:SolaceTraining/solace-workshop-scs.git

After importing everything you should see the following projects in STS:

Negative : Note: There will be errors associated with the template projects as they are incomplete and will be addressed in the exercises that follow.

$ cd ~/git/solace-workshop-scs/01-scs-workshop-common/
$ mvn clean install

Create and/or Verify access to a Solace PubSub+ Service

PubSub+ Service in Solace Cloud

If you want to stand up your Solace PubSub+ Service in Solace Cloud go ahead and login or signup at the Cloud Signup Page. Note that a free tier is available and will work for this workshop.

Local Solace PubSub+ Instance

When developing your application, you may want to test using a local instance of the Solace PubSub+ Event Broker. Refer to the Solace Docker Getting Started Guide to get you up and running quickly with a broker instance running in Docker. You may skip this step if you decide to use a broker running in PCF or Solace Cloud.

Negative : Note that this CodeLab uses the Annotation based Spring Cloud Stream code which is effectively deprecated in Cloud Stream v3. Please see the Spring Cloud Stream Basics Codelab to learn how to leverage Spring Cloud Function which is preferred in v3+.

story_section3

Application Architecture

At the end of this section we will have created the apps below! The Source will send out tweets that will be received by the marketing Sink.

1 Application Architecture

Negative : SCS Provides 3 Binding Interfaces: 1: Sink - "Identifies the contract for the message consumer by providing the destination from which the message is consumed." 2: Source - "Identifies the contract for the message producer by providing the destination to which the produced message is sent." 3: Processor - "Encapsulates both the sink and the source contracts by exposing two destinations that allow consumption and production of messages."

Deploying a Source

Before our company can do anything with the tweets we have to start to receive an incoming stream of them! Let's get started! Please navigate to the "02-scs-source-tweets" project in your IDE.

Learn the Project Structure

Before we take a look at the code, let's take a quick look at the structure of a Spring Cloud Streams project.

SCS Project Structure

Negative : Spring Cloud Streams is built on top of Spring Boot. A great resource for creating your own Spring Boot applications is Spring Initializr. A publically hosted version is hosted here: start.spring.io

SCS Maven Dependencies

Negative : "Spring Cloud Stream is built on the concepts and patterns defined by Enterprise Integration Patterns and relies in its internal implementation on an already established and popular implementation of Enterprise Integration Patterns within the Spring portfolio of projects: Spring Integration framework." By using Spring Integration we can make use of familiar annotations such as @InboundChannelAdapater, @Transformer or @ServiceActivator

SCS Sample Implementation

SCS Application Config File

Positive : SCS apps are not restricted to only using one binder at a time. This allows a SCS app the flexibility of receiving events from one binder/location/environment/etc, performing business logic and then sending new events to another binder/location/environment/etc. Also note that because bindings are dynamically configured at run-time you don't have to touch the code to switch out your binder of choice, environment info, etc.

Deploy our 02-scs-source-tweets app

$ mvn spring-boot:run

story_section3_g2

To do this we will deploy a sink app. Recall that a sink app binds to an INPUT channel.

Deploying a Sink

Negative : Note that Spring Cloud Streams provides message converters which enable the conversion of a payload to/from a specified format/shape (such as a Tweet POJO in our case) based on the argumentType in the method signature and the contentType specified on the message. The default content type is application/json.

Positive : You now have a source application sending events to a sink application via an external eventing system, but notice that you didn't need to use any messaging APIs! SCS provides this abstraction and makes it possible for developers to concentrate on their business logic rather than learning proprietary messaging APIs!

Negative : Note that this CodeLab uses the Annotation based Spring Cloud Stream code which is effectively deprecated in Cloud Stream v3. Please see the Spring Cloud Stream Basics Codelab to learn how to leverage Spring Cloud Function which is preferred in v3+.

story_section4

Configure a Cloud Foundry Target in STS

STS provides integrated support for deploying, running and debugging your SCS services in PCF. In the Boot Dashboard view, configure a connection to your PCF deployment by clicking the "+" button as seen in the image below.

Configuring a connection to Cloud Foundry in STS(use button circled in red)

Follow the dialog prompts and fill in the username / password associated with your PCF account. You may need to skip SSL validation if your PCF deployment uses self-signed certificates.

Cloud Foundry CLI Setup

$ cf login -a <API_URL> -u <USERNAME>
$ Password>

Verify PubSub+ Service in Pivotal Cloud Foundry (PCF)

If you are using PCF, your administrator will have created an org and space for your workshop demo in which you can deploy and run your microservices. Moreover, a Solace PubSub+ service instance will have been created so that it can be bound by any app running in the space and automatically lookup credentials to connect to a broker instance running in PCF. You should determine the name of this service instance before deploying or running your application to avoid any service binding errors. You can do this through the Apps Manager or via the cf CLI. You'll need this service instance name later so don't forget it.

$ cf services
Getting services in org test-org / space development as user1...
name                    service             plan                       bound apps
solace-pubsub-service   solace-pubsub       Enterprise Shared Plan     sample-app

Deploy the Source to PCF

IF PARTICIPATING IN AN INSTRUCTOR LED WORKSHOP THE INSTRUCTOR WILL PUSH TO PCF. YOU ARE WELCOME TO RUN LOCALLY IF YOU WOULD LIKE

$ mvn clean install
$ cf push
$ cf logs 02-scs-source-tweets

Positive : Notice that you did not have to add any credentials for your PubSub+ service instance running in PCF. This is because the Solace Spring Cloud Connector allows for Auto-Configuration in PCF. Just specify the service name and the service configuration is automatically looked up and injected into your app from VCAP_SERVICES! This allows you to leave your local credentials in place for even easier development and also refrain from storing credentials in your code repository.

Deploy the Sink to PCF

$ mvn clean install
$ cf push
$ cf logs 03-scs-sink-analytics

Negative : Note that this CodeLab uses the Annotation based Spring Cloud Stream code which is effectively deprecated in Cloud Stream v3. Please see the Spring Cloud Stream Basics Codelab to learn how to leverage Spring Cloud Function which is preferred in v3+.

story_section5_g1

Application Architecture

At the end of this section we will have added the Factory Tweet Board Sink.

2 Application Architecture

Creating the Tweet Board Sink

We obviously don't have a giant LED board that we can use so we're going to settle for logging the tweets as they come in.

Negative : Spring Cloud Streams supports multiple messaging models. We are going to use two different ones in this workshop 1: Publish-Subscribe allows for an application to process all events sent to the defined subscription. It also allows new applications to be added to the topology without disruption of the existing flow.
2: Consumer Groups allow for a set of applications to participate in a "group" to consume messages. This option is commonly used to allow the ability for an application to scale horizontally (creating multiple instances of the same application) while only processing each event once.

Deploying the Tweet Board

At this point we have created our "04-scs-sink-tweetboard" application and it needs to be deployed.

$ mvn clean install
$ cf push
$ cf logs 04-scs-sink-tweetboard-**ATTENDEE_NAME**

story_section5_g2

Positive : Notice that the publisher (Source) application did not need to be modified in order for another consumer (Sink) application to receive the stream of tweets. There are two takeaways here: 1: The publish-subscribe paradigm allows for the publisher to send data once and not care whether 0,1,2 or 100 applications are subscribed on the other end. It just send the data and moves on. 2: Developing event driven applications allows for decoupling of your sending and receiving applications. This powerful concept allowed our company to add new functionality without touching our already operational applications.

Negative : Note that this CodeLab uses the Annotation based Spring Cloud Stream code which is effectively deprecated in Cloud Stream v3. Please see the Spring Cloud Stream Basics Codelab to learn how to leverage Spring Cloud Function which is preferred in v3+.

So far in this workshop we have created source or sink applications. In this section we will create our first processor.

story_section6_g1

Application Architecture

In order to meet our new goal we will add the Features processor and a new Sink as seen below.

3 Application Architecture

Create the Feature Processor

story_section6_g2

Processor with a Custom Binding Interface

Positive : Custom binding interfaces can be defined in order for your SCS app to have additional input or output channels. They also allow for custom naming of channels.

Processor using Dynamic Destinations

Negative : At this point you might be thinking "Okay these custom binding interfaces are great, but what if I don't know how many or what to call my output channels at design time?" Spring Cloud Streams supports the use of Dynamic Destinations for this exact situation! Dynamic destinations allow you to use business logic to define your destinations at runtime.

Negative : From the JavaDocs, the BinderAwareChannelResolver is "A DestinationResolver implementation that resolves the channel from the bean factory and, if not present, creates a new channel and adds it to the factory after binding it to the binder."

Create the Feature Sink for the Boss

Update the Tweet Board Subscription

Note that our processor that we created earlier in this lab publishes to multiple topics essentially splitting our feed into two. Due to our new requirements to not show new features on the twitter board we need to update that sink appropriately.

$ cd /path/to/app
$ mvn install
$ cf push

Negative : spring-boot-devtools is handy for development and adds features such as automatic restart and remote debugging. Click here for a high level overview of some of the functionality it provides.

Positive : Notice that the use of topic hierarchies provides an organized way of keeping track of your event streams. This ability, combined with the ability to wildcard in PubSub+ gives your applications an easy way to subscribe to the exact event streams necessary to meet your need. Don't underestimate the value of creating a well-defined topic hierarchy! A few best practices to keep in mind are: start general and become more specific as more levels are added, use short words or abbrevations as topics can only be so long, and don't mix upper and lower case! A good topic hierarchy might be defined as "<country>/<state>/<locality>" which would allow you to subscribe to all events in a given country "canada/>" or even subscribe to events from all 1,716 localities named San Jose no matter what country or state they're in "*/*/sanjose"

Negative : Note that this CodeLab uses the Annotation based Spring Cloud Stream code which is effectively deprecated in Cloud Stream v3. Please see the Spring Cloud Stream Basics Codelab to learn how to leverage Spring Cloud Function which is preferred in v3+.

story_section7

Negative : Spring Cloud Streams currently has two different ways to follow a Reactive (Functional) Programming Model: Spring Cloud Functions and spring-cloud-stream-reactive. We're going to concentrate on Spring Cloud Functions since the latter option is marked for deprecation.

Application Architecture

We're going to add a "No Yelling" processor in our event driven architecture in order to meet this new need.

4 Application Architecture

Deploying a SCS Processor using Spring Cloud Functions

Negative : Spring Cloud Functions supports 3 types that conveniently map to our 3 SCS binding interfaces. java.util.function.Function maps to a SCS Processor java.util.function.Supplier maps to a SCS Source java.util.function.Consumer maps to a SCS Sink

Positive : From the Spring docs, Spring Cloud Functions allows you to "Decouple the development lifecycle of business logic from any specific runtime target so that the same code can run as a web endpoint, a stream processor, or a task." Read more here: https://spring.io/projects/spring-cloud-function

Negative : Note that this CodeLab uses the Annotation based Spring Cloud Stream code which is effectively deprecated in Cloud Stream v3. Please see the Spring Cloud Stream Basics Codelab to learn how to leverage Spring Cloud Function which is preferred in v3+.

story_section8

Negative : Obviously this company has some morality issues :) In the real world you should definitely not modify people's speech or create fake news!

Application Architecture

A processor will be added to our architecture in order to convert negative words to positive ones.

5 Application Architecture

Create the Processor

Let's get started and hopefully have a bit of fun!

Positive : Notice that multiple processors can easily be connected together in order to form a processing chain.

Update the Tweet Board Subscription

$ cd /path/to/app
$ mvn install
$ cf push

Negative : Note that a cf restage would rebuild with the same code, but would pickup changes to library dependencies or the buildpack itself and a cf restart will just restart the already built droplet

Negative : Note that this CodeLab uses the Annotation based Spring Cloud Stream code which is effectively deprecated in Cloud Stream v3. Please see the Spring Cloud Stream Basics Codelab to learn how to leverage Spring Cloud Function which is preferred in v3+.

story_section9

Application Architecture

To meet this new requirement we are going to add the MQTT Web App shown in the diagram below:

6 Application Architecture

Positive : Since we're using Solace PubSub+ as our event broker we support a bunch of open standards and protocols. Even though the SCS apps are sending/receiving events using the Java API other applications can still use their language/protocol of choice.

Obtain PubSub+ Credentials for an App that can't use the Cloud Connector & Auto-config

IF PARTICIPATING IN AN INSTRUCTOR LED WORKSHOP THE INSTRUCTOR WILL PERFORM THIS SECTION. YOU ARE WELCOME TO RUN LOCALLY IF YOU WOULD LIKE

Create the Web App

IF PARTICIPATING IN AN INSTRUCTOR LED WORKSHOP THE INSTRUCTOR WILL PERFORM THIS SECTION. YOU ARE WELCOME TO RUN LOCALLY IF YOU WOULD LIKE

$ cf app 10-spring-boot-mqttwebapp

Negative : Note that the MqttWebApp is actually running locally in your browser. This paradigm of creating credentials in the PubSub+ service can be used to connect other external apps as well!

Negative : Note that this CodeLab uses the Annotation based Spring Cloud Stream code which is effectively deprecated in Cloud Stream v3. Please see the Spring Cloud Stream Basics Codelab to learn how to leverage Spring Cloud Function which is preferred in v3+.

Review

story_section10

Positive : Hopefully you not only learned how to use Spring Cloud Streams today, but also how it enables developers to concentrate on achieving business goals by removing the need to learn messaging APIs. You should also now have a solid understanding of how implementing an event-driven architecture allows for loose coupling between your apps which enables rapid addition of new functionality.

Continued learning topics:

This course was just an introduction to Spring Cloud Streams, but we've included some resources below if you're interested in learning more about it or some of the features that complement it! Happy Learning :)