Introducing PayloadVault project Part-2

by

Overview

Contents hide
1 Overview
2 Check out the full project :
3 Prerequisite
4 Create Queue
5 Build a JMS publish service
6 Insert data into the Database
6.1 Create JDBC Insert Adapter :
6.2 Configure Global variable :
6.3 Create service to insert Data into table :
6.4 Enable audit logging :
6.5 Create JMS trigger to subscribe events :

This is the continuation part of the PayloadVault project, if you have not checked our article on part-1 click the link below to check that out. In this article, we will focus on below tasks.

  • We will create a queue and build both publish and subscriber services.
  • We will enable audit logging in subscriber service which will help us to reprocess failed transactions.

Check out the full project :


Prerequisite

  • Basics of JDBC adapter.
  • Basics of SQL queries.
  • Basics of DSP page with HTML, CSS, and javascript.

Create Queue

  • Check the UM connection, if the UM server is down then start the UM server.
  • Open Universal Messaging Enterprise Manager, log in to your UM connection, and create a queue named ‘Q_PayloadVault‘.

Build a JMS publish service

  • Navigate to the HG_PayloadVault.v1.Common.doc location in Designer and create a document named PublishPayload with the fields shown in the screenshot below.

  • Create a flow service with the name publishPayload in HG_PayloadVault.v1.PayloadVault_pub location.
  • In the input section of the ‘publishPayload‘ service, create a document reference named ‘PublishPayload‘ by referencing the ‘HG_PayloadVault.v1.Common.doc:PublishPayload‘ document.
  • Invoke ‘pub.jms:send‘ service in the ‘publishPayload‘ service, and follow the below mapping.
    • connectionAliasName hardcode it with your JMS connection name ex- ‘DEFAULT_IS_JMS_CONNECTION‘.
    • destinationName hardcode it with ‘Q_PayloadVault‘.
    • destinationType -> QUEUE.
    • JMSMessage/body/data ->PublishPayload.
    • useCSQ -> true.
  • Invoke the Map step and drop all the variables as we don’t need them.
  • Provide comments for each step.

Insert data into the Database

Create JDBC Insert Adapter :

  • Create a JDBC adapter named ‘storePayload‘ in ‘HG_PayloadVault.v1.PayloadVault_Sub.adapter‘, Use the JDBC connection where you have the table and utilize the JDBC Insert template.
  • In the ‘storePayload‘ adapter select the ‘PayloadVault’ table and in the insert tab select all columns and make sure to update INSERTED_DATE field type to string and PAYLOAD field type to byte array as shown in the screenshot below.

  • In the insert tab set the query time out to 300 or more.

Configure Global variable :

  • Login to the webMethods Administration page, navigate to the “Global Variables” section under the Settings tab and configure the following values as global variables.
    Global variables

Create service to insert Data into table :

  • Create a flow service with name ‘PayloadVault_sub‘ in ‘HG_PayloadVault.v1.PayloadVault_Sub‘ location.
  • In the input of the service, create a document named ‘JMSMessage‘ by referencing ‘pub.jms:JMSMessage‘.
  • Invoke the TRY-CATCH step inside the service.
  • TRY :
    • Invoke the Map step and perform the below mapping.
      • Create a document reference with name ‘PublishPayload‘ from ‘HG_PayloadVault.v1.Common.doc:PublishPayload‘ location and map it from ‘JMSMessage/body/data‘.
      • Create a string variable with the name ‘payloadSize‘ and hard code with ‘%PayloadVault.PayloadSize%‘ and make sure to enable the perform Global variable substitution option.
      • Create a string variable with the name ‘datePattern‘ and hard code with ‘%PayloadVault.datePattern%‘ and make sure to enable the perform Global variable substitution option.
    • Insert One BRANCH step and in the switch property put ‘PublishPayload/PayloadInfo/document‘ which will help us to check if the payload is a document or not.
      • Inside the BRANCH insert one MAP step and in the level put ‘/.+/’ which will check if  ‘PublishPayload/PayloadInfo/document‘ exists or not in the pipeline.
      • In the MAP step invoke ‘pub.xml:documentToXMLString‘ as a transformer and follow the below mapping.
      • PublishPayload/PayloadInfo/document -> document.
      • Create a string variable with the name payload in the pipeline out and map it from xmldata.
    • Insert One BRANCH step and in the switch property put ‘/PublishPayload/PayloadInfo/string‘ which will help to check if the payload is a string or not.
      • Inside the BRANCH insert one MAP step and in the level put ‘/.+/’ .
      • In the MAP step create a string variable with the name payload in the pipeline out and map it from ‘PublishPayload/PayloadInfo/string‘.
    • Now we need to check the size of the payload in MegaBytes and if it exceeds our configured value in the global variable then we should not store it in PayloadVault, to check the size of the payload create a java service with the name ‘checkSizeOfString‘ in ‘HG_PayloadVault.v1.PayloadVault_Sub.subService‘ location.
    • In the input section of the java service create a string variable with the name ‘inString‘ and in the output of the java service create a string variable with the name ‘sizeInMB‘.
    • update the Java service with the below script to provide the size of the string as output.
      package HG_PayloadVault.v1.PayloadVault_Sub.subService;

import com.wm.data.*;
import com.wm.util.Values;
import com.wm.app.b2b.server.Service;
import com.wm.app.b2b.server.ServiceException;
import java.nio.charset.StandardCharsets;
import java.text.DecimalFormat;

public final class checkSizeOfString_SVC

{

	/** 
	 * The primary method for the Java service
	 *
	 * @param pipeline
	 *            The IData pipeline
	 * @throws ServiceException
	 */
	public static final void checkSizeOfString(IData pipeline) throws ServiceException {
		// pipeline
		IDataCursor pipelineCursor = pipeline.getCursor();
			String	inString = IDataUtil.getString( pipelineCursor, "inString" );
		pipelineCursor.destroy();
		
		String sizeInMB=getStringSizeInMB(inString);
		// pipeline
		IDataCursor pipelineCursor_1 = pipeline.getCursor();
		IDataUtil.put( pipelineCursor_1, "sizeInMB", sizeInMB );
		pipelineCursor_1.destroy();
		
			
	}
	
	// --- <<IS-BEGIN-SHARED-SOURCE-AREA>> ---
	
	
	public static String getStringSizeInMB(String input) {
	    if (input == null) {
	        return "0.0";
	    }
	
	    // Convert the string to bytes using UTF-8 encoding
	    byte[] byteArray = input.getBytes(StandardCharsets.UTF_8);
	
	    // Calculate the size in bytes
	    int sizeInBytes = byteArray.length;
	
	    // Convert bytes to megabytes (1 MB = 1024 * 1024 bytes)
	    double sizeInMB = (double) sizeInBytes / (1024 * 1024);
	
	    // Use a more precise format
	    DecimalFormat df = new DecimalFormat("#.#####");
	    return df.format(sizeInMB);
	}
		
	
	// --- <<IS-END-SHARED-SOURCE-AREA>> ---

	final static checkSizeOfString_SVC _instance = new checkSizeOfString_SVC();

	static checkSizeOfString_SVC _newInstance() { return new checkSizeOfString_SVC(); }

	static checkSizeOfString_SVC _cast(Object o) { return (checkSizeOfString_SVC)o; }

}
    • Invoke the ‘HG_PayloadVault.v1.PayloadVault_Sub.subService:checkSizeOfString‘ service and follow the below mapping.
      • payload ->inString.
      • sizeInMB ->sizeInMB.
    • Invoke the Branch step and set the Evaluate Labels to true.
      • Invoke ‘pub.string:stringToBytes‘ service inside the BRANCH step and in the level put ‘%sizeInMB%<=%payloadSize%‘ and follow the below mapping.
        • payload -> string.
        • encoding -> Hard code to to ‘UTF-8’.
      • Invoke the Exit step and follow the below setup.
        • Label -> $default.
        • Signal ->FAILURE.
        • Exit From -> $flow.
        • Failure Message ->’Payload size is %sizeInMB% which is greater than %payloadSize% so the message is discarded.‘.
    • Invoke ‘pub.date:getCurrentDateString‘ to get the current date time and follow the below mapping.
      • datePattern ->pattern.
      • value -> create a variable with the name ‘currentDate‘ and map.
    • Invoke ‘HG_PayloadVault.v1.PayloadVault_Sub.adapter:storePayload‘ adapter and follow the below mapping.
      • PublishPayload/interfaceId ->storePayloadInput/INTERFACE_ID.
      • PublishPayload/source ->storePayloadInput/SOURCE.
      • PublishPayload/target -> storePayloadInput/TARGET.
      • PublishPayload/interfaceName ->storePayloadInput/INTERFACE_NAME.
      • PublishPayload/trackingId -> storePayloadInput/TRACKING_ID.
      • PublishPayload/documentType->storePayloadInput/DOCUMENT_TYPE.
      • PublishPayload/documentName->storePayloadInput/DOCUMENT_NAME.
      • currentDate->storePayloadInput/INSERTED_DATE.
      • PublishPayload/transactionId->storePayloadInput/TRANSACTION_ID.
      • PublishPayload/identifier1->storePayloadInput/IDENTIFIER1.
      • PublishPayload/identifier2->storePayloadInput/IDENTIFIER2.
      • PublishPayload/identifier3->storePayloadInput/IDENTIFIER3.
      • PublishPayload/identifier4->storePayloadInput/IDENTIFIER4.
      • bytes->storePayloadInput/PAYLOAD.
  • CATCH:
    • Invoke ‘pub.flow:getLastError‘ in the Catch block to extract error details.
    • Invoke ‘pub.flow:debugLog‘ and follow the below mapping to write the log into the server log.
      • message -> Hard code it to ‘Error occured while storeing the paylod into PyloadVault with TrackingID: %PublishPayload/trackingId% and TransactionId : %PublishPayload/transactionId%, Error : %lastError/error%.’ and enable ‘Perform variable substitution’.
      • level -> Error.
    • Invoke ‘pub.flow:throwExceptionForRetry‘ service and map ‘lastError/error’ to ‘message‘.

Enable audit logging :

  • Open properties of ‘HG_PayloadVault.v1.PayloadVault_Sub:PayloadVault_sub‘ service and set the below properties.
    • Enable auditing -> Always.
    • Log on -> Error Only.
    • Include pipeline -> On errors only.
  • Now if the service is failing due to any error then you will be able to view the error in MWS under Monitoring->Integration->Services section like below.

  • As you can see in the above screenshot, you can use the resubmit button to resubmit the transaction, you can also update the payload for this transaction.

Create JMS trigger to subscribe events :

  • Create a JMS trigger in ‘HG_PayloadVault.v1.PayloadVault_Sub‘ with the name ‘PayloadVault_JMSTrigger‘ and configure the below properties as shown in the screenshot.

In the next article, we will focus on loading the data into the database and populating it in the front end. Additionally, we will implement search functionality, we will store it. Stay tuned.

Post Views: 347

Leave a Comment