As NYC Modern Taxi begins its quest to compete with ride sharing companies, it faces many challenges. One of them is winning over existing taxi drivers, who aren't excited about using the new app.
To that end, NYC Modern Taxi Company decided to use Salesforce to treat each cabdriver like a potential customer. Account managers have been assigned 30 taxicab drivers each and have been instructed to spare no amount of free coffee to get them using the new app. Time is of the essence, so as soon as a driver agrees to join the app, the data in Salesforce needs to be available in the operations database. Simultaneously, NYC Taxi conducts a background check.
In this scenario, there's no time for data re-entry, or delays in data transmission: It's gotta be event-driven. If a driver tries to login to the app and can't because the information isn't there...fuhgettaboutit.
This codelab covers a lot of ground. You'll learn about:
✅ How to design an Event Driven Architecture with the Solace's Event Portal
✅ How to seamlessly implement your architecture using the PubSub+ Connector for Boomi and distribute real-time data around your enterprise
✅ How to get events flowing out of Salesforce using its Change Data Capture and Platform Events capabilities
✅ How to use Boomi's Salesforce Platform Events connector to receive those events and transform them
✅ How to interact with Azure SQL, a cloud-based database, using Boomi's connector.
Open the NYC Modern Taxi Co – Back Office domain by double clicking on it.
Here's the basic flow of events through the system
👉 After being convinced by cash, coffee or other means, a driver agrees to use the app
👉 The account representative changes the driver's account from inactive to active
👉 Salesforce sends an event to Boomi
👉 A Boomi process transforms the event payload, removing unnecessary fields and publishes the event to the Solace event broker.
👉 Solace simultaneously distributes copies of the driver's information across the company:
Update Driver Database Boomi process receives the event, and inserts or updates the driver information in an Azure Cosmos database
If the active field on the account is set to activated, the Background Check Boomi process runs a background check on the driver. This takes 3 minutes. When finished, the Update Driver Database process updates the database with the results. (You already got a lot on our plate, so that'll be an optional exercise at the end)
The account information goes to the Analytics department's data lake for real-time analysis. (Once again, you'll save that part for another CodeLab)
Do a deep dive into the individual applications, events and schemas by double clicking on them. Note that Event Portal houses all the schemas and topic strings that you'll need for the CodeLab.
With event-driven design, each component is decoupled from one another. That makes development and testing each component in isolation easier. You'll take that approach here: one at a time, we'll get Solace, Salesforce, Boomi and Azure SQL working in "Hello World" fashion. Then you will link them all together using the PubSub+ Event Broker to complete the solution.
Designing the architecture in Solace Event Portal means that the schemas, topics and events are in one place, versioned and governed. It also means that the Solace PubSub+ Connector for Boomi can import them, saving you from manually re-entering information into Boomi.
⚙️ Deeper Dive into Salesforce Event Architecture
An account rep updates or adds an Account using the Salesforce UI
Salesforce alerts an Apex trigger that an account has changed
The trigger maps the new values for the Account into a custom platform AccountChanged event
Salesforces publishes the event on its internal message bus
Boomi's Salesforce Platform Events connector listens for AccountChanged events
A Boomi process maps the Salesforce Platform Event into a Solace event, standardizing the data format and creating a topic string that allows for pinpoint data distribution.
You'll generate the token needed to connect to the Event Portal from Boomi. Click on the very last icon in the left hand column takes you to the User Profile (the label will be your account name, not Solace CTO 😄) Note: if you've already generated an Event Portal token for another CodeLab, feel free to reuse it.
Click on Token Management
Click on Create Token
Name the token BoomiUIAccess. Give it the "Event Portal Read" permission:
Click on Generate Token
Copy the generated token to a text editor. You will use it in the next section.
⚙️ Boomi configuration
⚙️ Connect the process to listen for Solace events
Open the Hello World – Listen for Solace Event process you just added to your workspace.
Click on the start shape, then click on the pencil icon next to the Solace Event Portal and PubSub+ Connection field
Fill in the Solace Event Portal and Event Broker connection information you gathered in the last section.
Click on Test Connection. Select an atom, then click Next. You should get a message indicating that the test connection was successful. Then save and close the Connection.
Next, modify the Connector Operation by clicking on the pencil icon.
Click on the Import button
Pick an atom, and use the Event Portal connection that you just modified.
Event Portal returns a list of events currently available on the Event Portal. Select the DriverUpserted event, and click on OK. Boomi grabs the schema from the Event Portal and creates a response profile (you'll use this for mapping in a later section). It also brings the topic subscription format from Event Portal.
Change the destination to NYCTAXI.BACKOFFICE.BOOMI.UPDATEDRIVERDB This will be the Solace queue that the process binds to. Solace will create it automatically when you deploy the process for the first time.
Replace the topic string structure with taxinyc/backoffice/driver/upserted/v1/salesforce/> The > character at the end is a wildcard that means that your process will get all DriverUpserted events, regardless of the upsert source or the account status.
Make sure that the check box next to Create Queue and TS is checked.
Save and Close the Connector operation and start shape.
⚙️ Package and deploy
Save the Process, then click on Create Packaged Component using default options. Proceed to Deploy the new components to your atom.
Ensure that the Solace listener is up and running by going to Manage > Atom Management
Click on your atom
In the middle column, select Listeners
You should see the listener up and running, with a green circle next to it.
And Now-One Quick Fix
Go back to the Solace UI, click on Messaging Services in the upper left, then select the messaging service that you created.
Click on Manage
Click on Queues
Click on the +Queue button to create a new queue to hold events that have failed processing
In the popup, name the new queue #DEAD_MSG_QUEUE and hit Done.
Click on the input queue for your Boomi Process: NYCTAXI.BACKOFFICE.BOOMI.UPDATEDRIVERDB queue
Click on the Edit button (with the pencil) in the upper right
Click on Show Advanced Settings in the upper right
Change the Maximum Redelivery Count field to 3. This moves a failed event to a queue named #DEAD_MSG_QUEUE after 3 attempts.
You will reuse the Solace connector you created in the last step, so you'll just need to modify the Solace connector operation.
Open the "Hello World - Publish Solace Event" process, and click on the Publish DriverUpsertedEvent operation
You'll note that the sample code has already filled in the destination topic.
Since this process doesn't use a Listen, we can just use Boomi's test functionality. Click on the Test button and choose your atom.
Once the test completes, go to Manage > Process Reporting
You should see an execution of the Hello World Listener
Click to view the logs and see that the process received the message successfully. So what did you just do? You now have a complete Boomi and Solace integration, with the first Boomi process publishing a message on a topic. Solace examines the topic string on the event, finds a matching subscription and puts the event on a queue. A second Boomi process is listening for events that land on that queue, so when a event does land there, the process instantly triggers.
⚙️ Salesforce configuration
Log into your Salesforce Developer account
Switch to the Setup view using the button in the upper right hand corner
In left hand column, expand out Apps, then click on App Manager
Click on "New Connected App"
Configure the Connected App
For the callback URL, insert your Boomi account ID. If you're logged into Boomi, it will be the part immediately after accountId= For example, if your Boomi URL is: https://platform.boomi.com/AtomSphere.html#build;accountId=trainingjessemenning-WX1LP2 Then your account id is: trainingjessemenning-WX1LP2 And your callback URL is: https://platform.boomi.com/account/trainingjessemenning-WX1LP2/oauth2/callback
For the Oauth Scopes, move Full Access into the Selected OAuth Scopes.
Click on the Configure ID Token check box, and make the token valid for 720 minutes
1.Click on Save. In the next screen, Copy and paste the consumer key and consumer secret into a text editor. You'll use the values when you're configuring Boomi's Salesforce Platform Event connector.
⚙️ Define Platform Event structure
Type "platform" into the quick find box in the left-hand column and click on "Platform Events"
Click on New Platform Event
Use AccountChanged as the label for the event and click on Save.
Add the following Custom Fields by repeatedly clicking on the "New" button and following the instructions. Note that spelling does matter, so please be careful.
⚙️ Create a trigger that populates and publishes the Platform Event
In the upper right-hand portion of the screen, click on Developer Console
Go to File > New > Apex Trigger
In the popup, enter the following values and click on submit:
In a new tab, go to https://workbench.developerforce.com/login.php. Accept the term and click Login with Salesforce
Select "Stream Push Topics" from the Jump To menu
Click on the Generic Subscriptions radio button
In the subscription field, enter /event/AccountChanged__e
Back in Salesforce, leave the Setup section of Salesforce, and go to the Sales Console
Create a new account.
Enter the name of the driver, phone number and billing address information. Be sure to set the Active dropdown to of the account to "active", as only active accounts are sent over to the Driver database.
Move back to the workbench tab. You should see data from your newly created account.
Open the "Hello World - Listen for Salesforce Platform Event" sample process.
Click on the Start shape, then select "Salesforce Platform Events" in the dropdown
Click on the plus sign in the Connection box to create a new connection to Salesforce
Cut and paste the Client ID from the previous step into the Client ID box
Cut and paste the Client Secret into the Client Secret box.
Scroll to the bottom of the screen and click on Generate
You'll see the following if it's successful:
Click on Save and Close
Click on the Plus sign next to Connector operation to create a new operation.
Click on Import in the upper right corner
Fill out the Import wizard with your local atom, and your newly created connection.
You should see your account change event listed.
Before plunging further, deploy your testing flow using the typical deployment routine. Then make a change within Salesforce and see if it reaches Boomi.
Copy mssql-jdbc-8.2.2.jre8.jar to <Boomi install directory>/userlib/database . If those directories don't exist, just create them.
End result should look like this, where the highlighted portion depends on how you installed the Atom
OK, back to the fun stuff: In another tab, head back to the Boomi UI
Open the "Azure SQL Database Connection – taxinyc"
Open the "Hello World - Select from/Insert into Azure SQL" process
Fill in the connection details you saved in the last section. The value for the host will be different than the one listed here.
Click on Test Connection, you should get a "success" message. If not, check your connection values. Save and close.
Open the Hello World - Select from/Insert into Azure SQL process.
Click on the "Test" button
Once it runs, return to the Azure UI, and get the data you just inserted.
Return to Boomi, double click on the "Sample data for testing" shape, and change the first name
Rerun the test and confirm that the data has changed.
Now that we have all the components tested individually, we can use the Solace event broker to tie them all together.
First, undeploy all the processes we used for testing. Go to Deploy > Deployments
Undeploy any "Hello World" processes that are deployed
Open the "Listen for Salesforce Platform Account, publish Solace DriverUpserted" process. Expanding on the Hello World functionality, here are the differences:
Rather than using the same topic, here the code change the topic string based on whether the account is active, just was activated or was deactivated
There is now a mapping from the Salesforce event over into the standard Solace event.
At the end of the process, Boomi publishes the message to Solace, on the topic string you generate earlier in the process.
You don't need to make any changes, just create a packaged component and deploy it as usual
Open up the "Listen for Solace DriverUpserted, Insert into Drivers" process. This process also expands on the Hello World flows. Rather than just using canned data, it listens for events on the queue you created earlier. As Solace publishes DriverUpserted events, they land on the queue. The process grabs them and inserts or updates the Driver table.
Create a Packaged Component and deploy.
The world is your oyster. Log back into Salesforce and add and modify Accounts. As you do, they should stream in real time over Boomi and Solace, and instantly land in the SQL database
Congratulations, you now have a functioning, real-time, event-driven Salesforce implementation. No more worries about your operational database lagging behind Salesforce. Because no one likes an angry cabbie. Although seeing event move quickly though an enterprise is pretty cool, this is just the beginning. Event enabling Salesforce vastly simplifies adding on additional information consumers. Like we mentioned in the beginning—need to add a background check? Just add another process with a subscription and add the logic. Want to add real-time analytics? Send the DriverUpserted events to Kafka. Now that you have real-time data flowing, the possibilities are endless.
Thanks for participating in this codelab! Let us know what you thought in the Solace Community Forum! If you found any issues along the way we'd appreciate it if you'd raise them by clicking the Report a mistake button at the bottom left of this codelab.