Enterprise Sales and Procurement Model (ESPM) Cloud Native

Description

Enterprise Sales and Procurement Model (ESPM) Cloud Native is a reference application to showcase how Resilience patterns can be implemented in a Cloud Native application. It’s built based on microservices architecture principles. Each microservice is built as a Spring Boot application. The current scope of the application showcases the below resilience patterns.

Architecture

Alt text

The ESPM application consists of five microservices and one external service.

  1. Customer Service - This service process Customer and Shopping cart information

  2. Product Service - This service can be used to process products and stock information

  3. Sales Service - Sales Orders are processed by this service. Each time a sales order is created, it’s not directly inserted into the database, but inserted into a queue. A background process called worker picks the message from queue and inserts to the database. The rationale behind this approach is explained later in the document. For read operation on sales order, its directly read from the database.

  4. Worker - Background process which picks the Sales Order from the queue and inserts it into the database.

  5. Gateway - It’s an optional component and acts as entry point for the complete application. It also acts as a reverse proxy and routes the request to the appropriate microservice. The UI for the application is integrated into the Gateway module. Then UI of the application consists of two parts

    Webshop: An application where an authenticated Customer can buy products by creating Sales Order

    Retailer: An application where an authenticated and authorized Sales Manager known as Retailer can approve/reject sales orders. Only a user with retailer role will be able to access the end point.

  6. External Tax Service - This is a service which is external to the application and used to do tax calculation. This Tax calculation service is provided, to be used along with the implementation of Circuit Breaker, Quarantine pattern. This service is also used in showcasing the app-to-app communication between two microservices deployed in the same subaccount, but bounded to two different Authorization and Trust Management services. For more information, see referencing the application in the documentation for SAP Cloud Platform.

A Domain Driven Design approach was used to decide the capabilities of each microservices. The Customer and Cart entities are part of the Customer Microservice and Product and Stock entities are part of the Product Service. To keep things simple, there is only one entity in Sales Service which is the Sales Order entity. In real world scenarios, Sales Entity might have Sales Order Header and Sales Order Line Items Entity and more. The Product and Customer service has its own database while Sale and worker shares the same database.

Each of the resilience patterns has been fit into architecture of the ESPM Application to showcase how they can make an application resilient during potential failures. These are some of the potential places where the pattern could be applied. There could be more points in the application where the pattern could have been applied to make it more resilient.

Retry

In a distributed environment some resources may not be reachable or unavailable due to network latency or network glitches. A simple retry might cause the execution of a task to succeed which would have failed, if no retry was attempted. This pattern is showcased by wrapping the database calls in Product and Customer Service with a retry. This ensures that if the database is not momentarily reachable a retry will ensure that the task succeeds.

Timeout

It's usually not possible to predict how long it will take for response while calling an external service. Defining a timeout ensures that the caller be interrupted and does not wait indefinitely if the no response is received. The timeout is implemented in the Sales Service while calling the external Tax Service. This ensures that Sales Service is not indefinitely blocked by calls to Tax Service.

Circuit Breaker

This pattern addresses the challenge in communicating with an external system. The status of the external system is not known, and it could be under load and not responding. The circuit breaker tackles these problems by introducing a kind of circuit for each external dependency. If a problem is identified, the circuit on the caller side controls the behavior of the calls in future. The circuit breaker is implemented in the Sale Service of ESPM application for communicating with the external Tax service. The Tax service could be temporarily, unavailable, under load or non-responsive. The Circuit Breaker ensures that if Tax service is not reachable the circuit is opened, and no future calls goes Tax service and a fall back service or mechanism is used for Tax Calculation.

Bounded Queue

Introduction of a queue brings the application closer to an asynchronous processing paradigm. It based on assumption that computing resources like CPU and memory are not infinite. The bounded queue implementation in Sale Service can ensure that in case there are spikes in the rate at which Sales Orders are created, they can be slowed down by inserting into the queue first. The number of requests, the application can process at a point in time can be decided by the size of queue. If the queue becomes full, it creates a back pressure by rejecting messages. This ensures that application is not getting overloaded and does not crash. Also, a secondary advantage is that, if due to network latency, database is not available momentarily, the data can remain in the queue. Once the database is available, the worker can pick the data from queue and write to database. In cloud platform, Enterprise Messaging service provides an unbounded queue. It follows a pay per message model.

Shed load

This pattern focuses on handling the rate at which requests are coming and reject requests before processing, if the system can't handle it. Each request consumes memory. If the system tries to process too many requests than it can handle, it can crash. Shedding the load by rejecting requests which it can't handle as early as possible, ensures that the application remains healthy and does not crash. The system can define a fixed rate for accepting request or be elastic and decide at runtime the current load on resources and decide to accept or reject the request. The Shed Load pattern is implemented in Product and Customer Service to avoid spike in the number of concurrent requests handled by the application. The number of requests which can be processed at a point in time is fixed to specific number and the requests exceeding this number is rejected.

Unit Isolation

The focus of this pattern is on the design of the failure unit. A failure unit is the entity of an application that can fail without overall availability of the entire application being affected. The microservices architecture paradigm itself brings in a level of unit isolation while applying methodology of domain driven design to define the units.

REST API

Swagger API Definition

Customer Service

https://customer-service.cfapps.eu10.hana.ondemand.com

Product Service

https://product-service.cfapps.eu10.hana.ondemand.com

Sales Service

https://sales-service.cfapps.eu10.hana.ondemand.com

Requirements

Before running ESPM application one would need

For Running locally:

Message server

SQL Database Server

For Cloud:

cf install-plugin multiapps

cf add-plugin-repo CF-Community https://plugins.cloudfoundry.org

Deploying the ESPM application locally

Follow steps below to run each microservice of ESPM one by one. Please ensure that message server and SQL Database server are running before you start.

Customer Service

Product Service

Worker

Tax Service

Tax service is an external service. It can be deployed locally or on SAP Cloud Platform Neo Environment or SAP Cloud Platform Cloud Foundry Environment. This service does Tax calculation while a sales order is created. Tax Service can be locally deployed by following these steps.

Sales Service

Lifecycle Life Cycle Status Name Note
N New When the Sales Order is created
I In Progress
C Cancelled When the product is Out Of Stock
S Shipped When the Sales Order is Shipped
R Rejected When the Sales Order is Rejected by Retailer
D Delivered

Gateway

Accessing the Local API Endpoints

The below are the list of local service API endpoints of all the microservices.

Customer Service

Create Customer
Endpoint URL http://localhost:9991/customer.svc/api/v1/customers/
Header Content-Type:application/json
Method POST
Body {"emailAddress": "[email protected]", "phoneNumber": "0123456789", "firstName": "new", "lastName": "customer", "dateOfBirth": "19900911", "city": "Bang, KR", "postalCode": "112233", "street": "100ft Road", "houseNumber": "123", "country": "IN"}
Get Customer by Email ID
Endpoint URL http://localhost:9991/customer.svc/api/v1/customers/{emailAddress}
Method GET
Create Cart
Endpoint URL http://localhost:9991/customer.svc/api/v1/customers/{customerId}/carts/
Header Content-Type:application/json
Method POST
Body {"productId": "HT-1000","checkOutStatus": "false","quantityUnit": 3}
Get Cart by Customer ID
Endpoint URL http://localhost:9991/customer.svc/api/v1/customers/{customerId}/carts/
Method GET
Update Cart by Item ID
Endpoint URL http://localhost:9991/customer.svc/api/v1/customers/{customerId}/carts/{itemId}
Header Content-Type:application/json
Method PUT
Body "productId": "HT-1000","quantityUnit": 10,"checkOutStatus": false}
Delete Cart by Item ID
Endpoint URL http://localhost:9991/customer.svc/api/v1/customers/{customerId}/carts/{itemId}
Method DELETE

Product Service

Get All Products
Endpoint URL http://localhost:9992/product.svc/api/v1/products
Method GET
Get Product by Product ID
Endpoint URL http://localhost:9992/product.svc/api/v1/products/{productId}
Method GET
Get Stock by Product ID
Endpoint URL http://localhost:9992/product.svc/api/v1/stocks/{productId}
Method GET

The stock is updated by quantity specified in the payload. e.g. if the current quantity is 50 and in payload in the body for update stock request quantity is provided as 20 the quantity will be updated to 70.

Update Stock by Product ID
Endpoint URL http://localhost:9992/product.svc/api/v1/stocks/{productId}
Header Content-Type:application/json
Method PUT
Body {"productId": "HT-1000","quantity": 20}

Sales Service

Create Sales Order
Endpoint URL http://localhost:9993/sale.svc/api/v1/salesOrders
Header Content-Type:application/json
Method POST
Body {"customerEmail": "[email protected]","productId": "HT-1000","currencyCode": "EUR", "grossAmount":956,"quantity":4}
Get Sales Order by Sales Order ID
Endpoint URL http://localhost:9993/sale.svc/api/v1/salesOrders/{salesOrderId}
Method GET
Get Sales Order by Customer Email ID
Endpoint URL http://localhost:9993/sale.svc/api/v1/salesOrders/email/{emailAddress}
Method GET
Get All Sales Order
Endpoint URL http://localhost:9993/sale.svc/api/v1/salesOrders/
Method GET

Tax Service (External Service)

Get Tax Amount
Endpoint URL http://localhost:9994/tax.svc/api/v1/calculate/tax?amount=1000
Method GET

Test the ESPM Application Locally

To test the ESPM application, Postman REST Client can be used. A Postman collection which is provided here has all the request URLs and sample request body payloads (in case of a POST request).

Deploying the ESPM application on Cloud Foundry

To run the application on Cloud Foundry you need an account on SAP Cloud Platform Cloud Foundry Environment Productive account. Please note that in SAP Cloud Platform Cloud Foundry Environment, for a trial account, there is limited resource and you get a RAM of 2 GB which is not sufficient to run the complete ESPM application.

Check if the Cloud Foundry Space you will be deploying the application has the following entitlements:

Service Plan Number of Instances
Destination lite 1
Enterprise Messaging default 1
SAP HANA Schemas & HDI Containers schema 1
SAP HANA Service 64standard 1
Application Runtime 7

Create SAP HANA Service instance with plan 64standard as described here. If the SAP HANA Service instance is present in another space share with your space as described here

If there are multiple instances of SAP HANA Service in the space where you plan to deploy this application, please modify the mta.yaml as shown below. Replace with the id of the database you would like to bind the application with :

# Hana Schema
- name: espm-hana-db
type: com.sap.xs.hana-schema
parameters:
service: hana
service-plan: schema
config:
database_id: <database_guid>

The ESPM application has a dependency to Tax Service Application which is a mock external service and needs to be separately deployed. Tax service is bound to its own instance of the Authorization and Trust Management service(XSUAA).

Please note that the ESPM application and Tax Service application should be deployed on the same CF space.

Security Implementation

The security implementation in the ESPM application is based on Spring Security. Spring applications using the Spring-security libraries can integrate with the SAP Cloud Platform Authorization and Trust Management service as described here. ESPM Application implements app-to-app communication so that two microservices can securely communicate with each other. This application showcases how to implement a secure communication using two different ways:

Propagating a Business User

In this approach, the business user is authenticated and his authorizations are used to call another microservice. The user is therefore known to the microservice that he is calling.

Using a Technical User

In this approach, a technical user is used to access data from another microservice. The called microservice grants the calling application the necessary rights without identifying a user.

Both methods have their use cases, depending on whether or not you need to identify the business user and grant access based on his authorizations or using a technical user is sufficient.

Implementing Authentication and Authorization

The steps below describe how authentication and authorization is implemented in the ESPM application.

Business User Implementation

As a pre prerequisite, the sale-service and product-service should be bound to same xsuaa instance.

  1. Add the application security descriptor file (xs-security.json) to the project.

    This file can be found in the root folder of the project.

  2. Define a role Retailer within the application security descriptor.

    Only a person assigned the Retailer role will be able to access the retailer UI of the ESPM application to process the sales orders.

  3. Configure scope checks for validating jwt tokens.

    This is done in the sale-service and product-service by extending the WebSecurityConfigurerAdapter class.

  4. Implement app-to-app communication for the business user in the createSalesOrder method of class com.sap.refapps.espm.controller.SalesOrderController in the sale-service microservice

  5. Implement app-to-app communication for the business user in the UpdateStockbyProductID method in the com.sap.refapps.espm.controller.ProductController class of the product-service microservice.

    When a Retailer logs in to accept a sales order created by a customer, the business user is propagated from the sale-service to product-service for a stock check before accepting a sales order. This ensures that enough stock is available before a sales order is accepted and only a user with the Retailer role has the permission to do a stock check.

    Technical User Implementation

    App-to-app communication for the technical user is implemented between the sale-service and the tax-service using client-credential flow. The sale-service and the tax-service are bound to different XSUAA instances.

  6. The sale-service is bound to the instance espm-xsuaa(which uses xs-security.json).

  7. The tax-service is bound to the instance espm-xsuaa-tax(which uses xs-security-tax.json).

  8. The tax-service grants a scope to the sales-service using the property "grant-as-authority-to-apps" in the xs-security-tax.json. This property has the value ["$XSAPPNAME(application,espm-cloud-native-uaa)"] where espm-cloud-native-uaa is the xs-appname of the espm-xsuaa service.

  9. The sales-service accepts the granted authorities. This is achieved by the property "$ACCEPT_GRANTED_AUTHORITIES" in the xs-security.json. This ensures that the tax-service trusts the sale-service and hence technical user communication between the two services is achieved using client credentials flow.

    For more information, refer to section referencing the application in the documentation for SAP Cloud Platform.

Configuring Enterprise Messaging

Tax Service Application Deployment

The Tax Service Application can be deployed in two ways:

Please note that the ESPM application and Tax Service application should be deployed on the same CF space.

CF Manifest

Deploy Service

From the tax-service folder where mta.yaml is kept for tax-service application run the command:

  mbt build -p=cf

This will package your application to be ready for deployment.

To Deploy MTAR, run the command:

cf deploy mta_archives/cloud-espm-cloud-native-tax_1.1.0.mtar

Create Destination

Destination will be used by ESPM Application to consume the Tax Service which is an external service.

Build and Deploy ESPM Application

Using CF manifest

cf cs enterprise-messaging default espm-em -c em-default.json

  • In case you are using a different name for the HANA instance, please update the hana configuration file, manifest file as well with the same name.

  • It’s possible to use different HANA instances for each of the microservices too, in that case, you would have to keep a copy of hana configuration file in the config folder of each of the microservices with the corresponding HANA instance names.

For simplicity all the microservices are bound to one database instance espm-hana-db. If required three database instances can be created (e.g. esmp-customer, espm-product and espm-sales) and individual microservice can be bound to them.

note: In case you are changing the xsappname in xs-security.json, please update it in xs-security-tax.json as well. This is because tax service authorizes only an app with the xsappname mentioned here.

Using CF deploy service

This will package your application to be ready for deployment.

To Deploy MTAR, run the command:

cf deploy mta_archives/cloud-espm-cloud-native_1.1.0.mtar

Running the Application

Setup Role collections

The ESPM application defines a role template called as Retailer and a role collection called as Retailer-RoleCollection in the application security descriptor (xs-security.json). Users need this Retailer role collection to accept sales orders. Creation of sales orders can be done by anonymous users. For more information about adding roles to role collection, see Add Roles to Role Collections in the documentation for SAP Cloud Platform.

Assign Role to the user

We need to assign the role which we have created in the previous step to the user. For more information about assigning role collections, see Assign Role Collections in the documentation for SAP Cloud Platform.

Enterprise Message Queue creation

Alt text

Acessing the application UI

Lifecycle Life Cycle Status Name Note
N New When the Sales Order is created
C Cancelled When the product is Out Of Stock
S Shipped When the Sales Order is Shipped
R Rejected When the Sales Order is Rejected by Retailer

Alt text

Accessing the application API Endpoints

The below are the list of local service API endpoints of all the microservices.

Customer Service

Create Customer
Endpoint URL https://-espm-customer-svc.cfapps.eu10.hana.ondemand.com/customer.svc/api/v1/customers/
Header Content-Type:application/json
Method POST
Body {"emailAddress": "[email protected]", "phoneNumber": "0123456789", "firstName": "new", "lastName": "customer", "dateOfBirth": "19900911", "city": "Bang, KR", "postalCode": "112233", "street": "100ft Road", "houseNumber": "123", "country": "IN"}
Get Customer by Email ID
Endpoint URL https://-espm-customer-svc.cfapps.eu10.hana.ondemand.com/customer.svc/api/v1/customers/{emailAddress}
Method GET
Create Cart
Endpoint URL https://-espm-customer-svc.cfapps.eu10.hana.ondemand.com/customer.svc/api/v1/customers/{customerId}/carts/
Header Content-Type:application/json
Method POST
Body {"productId": "HT-1000","checkOutStatus": "false","quantityUnit": 3}
Get Cart by Customer ID
Endpoint URL https://-espm-customer-svc.cfapps.eu10.hana.ondemand.com/customer.svc/api/v1/customers/{customerId}/carts/
Method GET
Update Cart by Item ID
Endpoint URL https://-espm-customer-svc.cfapps.eu10.hana.ondemand.com/customer.svc/api/v1/customers/{customerId}/carts/{itemId}
Header Content-Type:application/json
Method PUT
Body {"itemId": {itemId},"productId": "HT-1000","quantityUnit": 10,"checkOutStatus": false}
Delete Cart by Item ID
Endpoint URL https://-espm-customer-svc.cfapps.eu10.hana.ondemand.com/customer.svc/api/v1/customers/{customerId}/carts/{itemId}
Method DELETE

Product Service

Get All Products
Endpoint URL https://-espm-product-svc.cfapps.eu10.hana.ondemand.com/product.svc/api/v1/products
Method GET
Get Product by Product ID
Endpoint URL https://-espm-product-svc.cfapps.eu10.hana.ondemand.com/product.svc/api/v1/products/{productId}
Method GET

In order to access the below endpoint, the user needs retailer role and token has to be passed in the header.

Execute the below command and make note of url, clientid, clientsecret.

cf env <unique_id>-espm-product-svc

Get New access Token
Access token URL <url>/oauth/token
Client ID <clientid>
Client Secret <clientsecret>
Grant Type Client Credentials
Get Stock by Product ID
Endpoint URL https://-espm-product-svc.cfapps.eu10.hana.ondemand.com/product.svc/api/v1/stocks/{productId}
Method GET
Header Content-Type:application/json , Authorization:Bearer <Get New Access Token>

The stock is updated by quantity specified. e.g. if the current quantity is 50 and in the body for update stock request quantity is provided as 20 the quantity will be updated to 70

Below URL requires the retailer role to be added to user and hence if you are executing the same from postman, make sure you have the role, and inorder to get the Access token with scopes of Retailer role execute the following request from postman.

Access token with scopes of Retailer role
Endpoint URL Access token URL
Header Content-Type:application/x-www-form-urlencoded
Method POST
Body x-www-form-urlencoded

The payload of the request needs to have following form-url-encoded values:

grant_type: set to password to define that the client and user credentials method has to be used for the token determination

username: set user name of authorized user

password: password of the authorized user

client_id: the client id determined for the application

client_secret: the client secret determined for the application

response_type: set to token to indicate than an access token is requested

Update Stock by Product ID
Endpoint URL https://-espm-product-svc.cfapps.eu10.hana.ondemand.com/product.svc/api/v1/stocks/{productId}
Header Content-Type:application/json , Authorization:Bearer <Access token with scopes of Retailer role>
Method PUT
Body {"productId": "HT-1000","quantity": 20}

Sales Service

In order to access the below endpoint, the user needs retailer role and token has to be passed in the header.

Execute the below command and make note of url, clientid, clientsecret.

cf env <unique_id>-espm-sales-svc

Get New access Token
Access token URL <url>/oauth/token
Client ID <clientid>
Client Secret <clientsecret>
Grant Type Client Credentials
Create Sales Order
Endpoint URL https://-espm-sales-svc.cfapps.eu10.hana.ondemand.com/sale.svc/api/v1/salesOrders
Header Content-Type:application/json
Method POST
Body {"customerEmail": "[email protected]","productId": "HT-1000","currencyCode": "EUR", "grossAmount":956,"quantity":4}
Get Sales Order by Sales Order ID
Endpoint URL https://-espm-sales-svc.cfapps.eu10.hana.ondemand.com/sale.svc/api/v1/salesOrders/{salesOrderId}
Method GET
Header Content-Type:application/json , Authorization:Bearer <Get New Access Token>
Get Sales Order by Customer Email ID
Endpoint URL https://-espm-sales-svc.cfapps.eu10.hana.ondemand.com/sale.svc/api/v1/salesOrders/email/{emailAddress}
Method GET
Header Content-Type:application/json , Authorization:Bearer <Get New Access Token>
Get All Sales Order
Endpoint URL https://-espm-sales-svc.cfapps.eu10.hana.ondemand.com/sale.svc/api/v1/salesOrders/
Method GET
Header Content-Type:application/json , Authorization:Bearer <Get New Access Token>

Resilience Patterns in action

Retry

Retry patterns is implemented in Customer and Product Service to retry interactions with the database. The database might not be reachable momentarily due to network latency. But a simple retry might ensure that the next request might succeed. This ensures that the operation does not fail. To see this pattern in action in the Customer Service, follow these steps-

Similarly to see this pattern in action in the Product Service, follow the below steps:

Timeout

This pattern is implemented in Sales Service along with Circuit Breaker pattern. It's used to ensure that any request from Sales Service to Tax service does not wait indefinitely but times out after a preconfigured time for 1.2 seconds and a fall back is used for Tax calculation. To see these patterns in action, follow these steps:

Bounded Queue

The Sales service along with Worker implements the Bounded Queue pattern. To achieve reliable messaging, Consumer Acknowledgement and Publisher Confirms. This ensures that messages are not lost and delivered reliably to consumers. To see the pattern in action, follow these steps-

Unit Isolation

ESPM has a microservice-based architecture, where all the services are independent of each other and have been isolated against each other here by bringing in Unit Isolation.

Circuit Breaker

In ESPM this pattern is showcased via sale service. This service needs to compute the tax amount for a Sales Order. This is done by hitting an external Tax Service. If the Tax Service is unreachable, instead of throwing an error, a fallback mechanism executes the logic and default tax value is returned. Resilience4j library is used to implement Circuit breaker patterns. To see the pattern in action follow these steps-

Shed Load

Shed Load pattern to limit the rate of request handling is implemented in the Product and the Customer Service. The same can be implemented on Sales Service but since the implementation is same it has not been implemented in the Sales Service.

The strategy to Shed Load, implemented in Customer and Product Service is to limit the concurrent requests. The simple and efficient way to limit the concurrent requests is to implement a Semaphore. Now that the concurrent requests can be rate limited, HTTP requests needs to be filtered. There multiple approaches to solve this problem. It could be solved via a servlet filter or a tomcat valve. Tomcat valve has been chosen to solve the problem of filtering the HTTP requests because filtering / rejecting happens very early in the request processing chain i.e. even before requests enters servlet context. Since its implemented at the Tomcat Container level, it ensures that all servlets/api endpoints are rate limited.

Currently all api endpoints have same rate limitation with respect maximum concurrent request which can be handled. The number of concurrent requests that can be handled is configured based on the max.requests property in the application.properties. This can be modified to have separate rate limitation for each api endpoint by having URL pattern matching in the Shed Load implementation.

Each application component must decide how many concurrent threads it can allow.

To see the Shed Load Pattern in action we will use Apache JMeter as performance benchmarking tool.

Known issues

None

Support

Please use GitHub issues for any bugs to be reported.

License

Copyright (c) 2018 SAP SE or an SAP affiliate company. All rights reserved. This file is licensed under the Apache Software License, version 2.0 except as noted otherwise in the LICENSE file.