Mulesoft

Introduction Recently I’ve been working with Mule, which is an ESB (Enterprise Service Bus). When I started off working on it, I had numerous questions and grey areas but understood over a period of time that community support can be very hard to find. Hence I wanted to share an introduction and tips that might help those who want to grasp this subject.  Let’s start with the basic concepts. Wait, what is an ESB? The Mule website describes it best: An Enterprise Service Bus (ESB) is fundamentally an architecture. It is a set of rules and principles for integrating numerous applications together over a bus-like infrastructure. ESB products enable users to build this type of architecture but vary in the way that they do it and the capabilities that they offer. The core concept of the ESB architecture is that you integrate different applications by putting a communication bus between them and then enable each application to talk to the bus. This decouples systems from each other, allowing them to communicate without dependency on or knowledge of other systems on the bus. The concept of ESB was born out of the need to move away from point-to-point integration, which becomes brittle and hard to manage over time. Point-to-point integration results in custom integration code being spread among applications with no central way to monitor or troubleshoot. This is often referred to as “spaghetti code” and does not scale because it creates tight dependencies between applications. Hmm, but why ESB? Time to market is increasingly important, and ESBs play an important role in improving time to market, especially with a diverse array of connected services. Using an ESB as an ecosystem makes your enterprise applications pluggable and facilitates communication between them. For example, let’s say we have a use case where we have data flowing from multiple applications (D1, D2, D3) in different formats which needs to be fed into an application(A1) as below: Here A1 receives data from D1 using the FTP protocol and transforms the data as necessary before consuming it. Again, it transforms the data from D2 for its usage and the same in the case of D3.  So effectively A1 = A1(what the application is supposed to do) + Transformer (for the transformations of the data from different data sources) which is not ideal. Not only do D1, D2, and D3 need to know destinations for data delivery (in this case, A1), but also A1 needs to understand what format each individual data source serves. Moreover, if there is another D4 and another A2, the mess is compounded by this point-to-point communication. We have already understood the problems from the above ecosystem; let’s reconsider the same scenario with an ESB. The ESB serves as a destination for all the data sources (D1, D2, D3) which in turn sends data to its registered destinations. What about the transformations? The ESB takes care of the transformation as per the consumer application which in this particular case is A1. Wait, did we also talk about scalability? Yes, there we go! Fair enough, what’s Mule? Mule is an implementation of the ESB pattern. Glossary:

• “Mule” is a runtime engine
• Anypoint Studio is the IDE in which to create applications
• Design Center is an application where APIs can be designed

Mule ESB facilitates easy integration and communication between a number of systems using built-in modules with scalability. It is easy to add custom modules as well. They can be dragged and dropped and configured through the Mule IDE. A few of the modules are Amazon S3 connector, Anyoint MQ, Database connector, JMS connector, Mongo DB connector, SMTP connector, Salesforce connector, Jetty, Quartz, etc and the list goes on.  If code customization is required, Mule offers two scripting options: an expression language, MEL(Mule Expression Language) and DataWeave.  MEL can be used for short snippets like a quick one-liner code, and DataWeave is better suited to more complicated logic. Personally, being a developer, I feel at ease with DataWeave as one can stretch out and write code with whatever best coding practices one believes in. Since Mule 4, DataWeave has become the main expression language for custom logic. Let’s consider an example in Mule where we need to write a RESTful API that fetches us the current WWE RAW World Champion. Suppose we have this WWE information in a Mongo DB in the database WWE_RAW, which we intend to serve through the above API using the steps below: 1. Build API spec:

#%RAML 1.0
title: WWE RAW Apis
version: v1.0

securitySchemes:
  basic:
    description: |
      This API supports Basic Authentication.
    type: Basic Authentication

securedBy: [basic]

traits:
  hasHeaders:
    headers:
      Authorization:
        displayName: Authorization
        description: Basic authentication base 64 encoded string
        type: string
        required: true
      Content-type:
        displayName: Content-type
        description: json
        type: string
        required: true

types:
 wrestler:
  properties:
    first_name : string
    last_name : string
    weight_lbs : integer
    current_champion : boolean

/raw/wrestlers/champion:
    is: ["hasHeaders"]
    description: api for fetching current champion
    get:
      responses:
        200:
          body:
            application/json:
              type: wrestler
        404:
          body:
            application/json:
              properties:
                message: string
              example: |

                  "message" : "No don't have a champ yet!"

2. Generate an API using RAML: We create a Mule project in Anypoint Studio and copy the RAML we designed above  into the project.  We then generate a “flow” from the RAML. This “flow” represents our API with whatever information our RAML provided such as path, payload, body, query parameters, etc. if any. It will be an empty flow that needs to be completed as shown below by drag and drop of the components from Mule Palette into the IDE. Above is the message flow, and below is the configuration XML (the third tab in the above image). One could drag-drop and create flows or reverse engineer flows by writing the  configuration XML. Configuration XML is the XML representation of the message flow.

<?xml version="1.0" encoding="UTF-8"?>
<mule xmlns="http://www.mulesoft.org/schema/mule/core" xmlns:apikit="http://www.mulesoft.org/schema/mule/mule-apikit" xmlns:doc="http://www.mulesoft.org/schema/mule/documentation" xmlns:ee="http://www.mulesoft.org/schema/mule/ee/core"
xmlns:http="http://www.mulesoft.org/schema/mule/http" xmlns:mongo="http://www.mulesoft.org/schema/mule/mongo" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.mulesoft.org/schema/mule/core
http://www.mulesoft.org/schema/mule/core/current/mule.xsd http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd http://www.mulesoft.org/schema/mule/mule-apikit
http://www.mulesoft.org/schema/mule/mule-apikit/current/mule-apikit.xsd http://www.mulesoft.org/schema/mule/mongo http://www.mulesoft.org/schema/mule/mongo/current/mule-mongo.xsd
http://www.mulesoft.org/schema/mule/ee/core http://www.mulesoft.org/schema/mule/ee/core/current/mule-ee.xsd">
    <configuration-properties file="app.properties"/>
    <http:listener-config name="wwe_raw_apis-httpListenerConfig">
        <http:listener-connection host="${app.hostname}" port="${app.port}" />
    </http:listener-config>
    <apikit:config name="wwe_raw_apis-config" raml="wwe_raw_apis.raml" outboundHeadersMapName="outboundHeaders"
    httpStatusVarName="httpStatus" />
    <mongo:config name="MongoDB_Config" doc:name="MongoDB Config" doc:id="fa250676-40f4-499d-8c78-c7d1bc609f3b">
        <mongo:connection username="${db.username}" password="${db.password}" database="${db.name}" />
    </mongo:config>
    <apikit:config outboundHeadersMapName="outboundHeadersMapName" httpStatusVarName="httpStatus" doc:name="Router"
    doc:id="b268a874-716a-4b9f-ab60-a6e4e0bb4c6a" name="Router" raml="wwe_raw_apis.raml" />
    <flow name="wwe_raw_apis-main">
        <http:listener config-ref="wwe_raw_apis-httpListenerConfig" path="/api/*">
            <http:response statusCode="#[vars.httpStatus default 200]">
                <http:headers>#[vars.outboundHeaders default {}]</http:headers>
            </http:response>
            <http:error-response statusCode="#[vars.httpStatus default 500]">
                <http:body>#[payload]</http:body>
                <http:headers>#[vars.outboundHeaders default {}]</http:headers>
            </http:error-response>
        </http:listener>
        <apikit:router config-ref="Router" />
        <error-handler>
            <on-error-propagate type="APIKIT:BAD_REQUEST">
                <ee:transform xsi:schemaLocation="http://www.mulesoft.org/schema/mule/ee/core http://www.mulesoft.org/schema/mule/ee/core/current/mule-ee.xsd">
                    <ee:message>
                        <ee:set-payload><![CDATA[%dw 2.0
output application/json
---
{message: "Bad request"}]]></ee:set-payload>
                    </ee:message>
                    <ee:variables>
                        <ee:set-variable variableName="httpStatus">400</ee:set-variable>
                    </ee:variables>
                </ee:transform>
            </on-error-propagate>
            <on-error-propagate type="APIKIT:NOT_FOUND">
                <ee:transform xsi:schemaLocation="http://www.mulesoft.org/schema/mule/ee/core http://www.mulesoft.org/schema/mule/ee/core/current/mule-ee.xsd" doc:id="4f5e1524-c26f-4885-b0c2-97871c0db96b">
                    <ee:message>
                        <ee:set-payload><![CDATA[%dw 2.0
output application/json
---
{message: "Resource not found"}]]></ee:set-payload>
                    </ee:message>
                    <ee:variables>
                        <ee:set-variable variableName="httpStatus"><![CDATA[404]]></ee:set-variable>
                    </ee:variables>
                </ee:transform>
            </on-error-propagate>
            <on-error-propagate type="APIKIT:METHOD_NOT_ALLOWED">
                <ee:transform xsi:schemaLocation="http://www.mulesoft.org/schema/mule/ee/core http://www.mulesoft.org/schema/mule/ee/core/current/mule-ee.xsd" doc:id="2c47904e-ba10-4909-872c-6deee0f3b86b">
                    <ee:message>
                        <ee:set-payload><![CDATA[%dw 2.0
output application/json
---
{message: "Method not allowed"}]]></ee:set-payload>
                    </ee:message>
                    <ee:variables>
                        <ee:set-variable variableName="httpStatus"><![CDATA[405]]></ee:set-variable>
                    </ee:variables>
                </ee:transform>
            </on-error-propagate>
            <on-error-propagate type="APIKIT:NOT_ACCEPTABLE">
                <ee:transform xsi:schemaLocation="http://www.mulesoft.org/schema/mule/ee/core http://www.mulesoft.org/schema/mule/ee/core/current/mule-ee.xsd" doc:id="531f1c92-02e4-415d-b09c-95c612d3b6f6">
                    <ee:message>
                        <ee:set-payload><![CDATA[%dw 2.0
output application/json
---
{message: "Not acceptable"}]]></ee:set-payload>
                    </ee:message>
                    <ee:variables>
                        <ee:set-variable variableName="httpStatus"><![CDATA[406]]></ee:set-variable>
                    </ee:variables>
                </ee:transform>
            </on-error-propagate>
            <on-error-propagate type="APIKIT:UNSUPPORTED_MEDIA_TYPE">
                <ee:transform xsi:schemaLocation="http://www.mulesoft.org/schema/mule/ee/core http://www.mulesoft.org/schema/mule/ee/core/current/mule-ee.xsd">
                    <ee:message>
                        <ee:set-payload><![CDATA[%dw 2.0
output application/json
---
{message: "Unsupported media type"}]]></ee:set-payload>
                    </ee:message>
                    <ee:variables>
                        <ee:set-variable variableName="httpStatus">415</ee:set-variable>
                    </ee:variables>
                </ee:transform>
            </on-error-propagate>
            <on-error-propagate type="APIKIT:NOT_IMPLEMENTED">
                <ee:transform xsi:schemaLocation="http://www.mulesoft.org/schema/mule/ee/core http://www.mulesoft.org/schema/mule/ee/core/current/mule-ee.xsd">
                    <ee:message>
                        <ee:set-payload><![CDATA[%dw 2.0
output application/json
---
{message: "Not Implemented"}]]></ee:set-payload>
                    </ee:message>
                    <ee:variables>
                        <ee:set-variable variableName="httpStatus">501</ee:set-variable>
                    </ee:variables>
                </ee:transform>
            </on-error-propagate>
        </error-handler>
    </flow>
    <flow name="wwe_raw_apis-console">
        <http:listener config-ref="wwe_raw_apis-httpListenerConfig" path="/console/*">
            <http:response statusCode="#[vars.httpStatus default 200]">
                <http:headers>#[vars.outboundHeaders default {}]</http:headers>
            </http:response>
            <http:error-response statusCode="#[vars.httpStatus default 500]">
                <http:body>#[payload]</http:body>
                <http:headers>#[vars.outboundHeaders default {}]</http:headers>
            </http:error-response>
        </http:listener>
        <apikit:console config-ref="wwe_raw_apis-config" />
        <error-handler>
            <on-error-propagate type="APIKIT:NOT_FOUND">
                <ee:transform xsi:schemaLocation="http://www.mulesoft.org/schema/mule/ee/core http://www.mulesoft.org/schema/mule/ee/core/current/mule-ee.xsd" doc:id="444222a1-ff70-48a8-b118-d939dac3afa0">
                    <ee:message>
                        <ee:set-payload><![CDATA[%dw 2.0
output application/json
---
{message: "Resource not found"}]]></ee:set-payload>
                    </ee:message>
                    <ee:variables>
                        <ee:set-variable variableName="httpStatus"><![CDATA[404]]></ee:set-variable>
                    </ee:variables>
                </ee:transform>
            </on-error-propagate>
        </error-handler>
    </flow>
    <flow name="get:rawwrestlerschampion:Router">
        <logger level="INFO" message="get:rawwrestlerschampion:Router" />
		<ee:transform doc:name="Create query" doc:id="24144e23-51a6-4c09-9d82-ebb7222d9862" >
			<ee:message />
			<ee:variables >
				<ee:set-variable variableName="query" ><![CDATA[%dw 2.0
output application/json
---
{
	"current_champion" : true
}]]></ee:set-variable>
			</ee:variables>
		</ee:transform>
		<mongo:find-one-document collectionName="wrestlers" doc:name="Execute query" doc:id="b027e174-f9b9-4827-a66c-37b083f873cd"
		config-ref="MongoDB_Config" returnId="false" >
			<mongo:find-query ><![CDATA[#[vars.query]]]></mongo:find-query>
		</mongo:find-one-document>
		<set-payload value="#[payload]" doc:name="Set Payload" doc:id="2cf2cf6a-9b03-4c5b-b136-fe95ff1b98e8" />
    </flow>
</mule>

We run the project, and this should start our application at a URL configured in the configuration XML with values such as ${app.hostname}, ${app.port}, context (/api/) and flow name - get:rawwrestlerschampion. Here “get” in the flow name serves as the HTTP verb. Time to taste our recipe. It better be sweet! It worked! Many other transformation and integrations are possible, which involve integration of different components and their orchestration. From my experience, the initial time to set up and configure the first flows take a bit of time, and the rest is a cakewalk. Finally, I suggest perusing the blog written by Ross Mason (founder of Mulesoft) To ESB or not to ESB which serves a good guideline for the best usage of ESB as an architecture.