Real time use cases of Deployment Synchronizer with a WSO2 ESB Cluster – Part 1

In this blog post I will explain some of the real time use cases of Deployment Synchronizer (DepSync) in a WSO2 ESB Cluster environment. If you want to know more about DepSync follow my previous post.

Before going further into the topic I'm going to list down the use cases so anyone can simply understand the possible scenarios happened here.

1. Create an artifact using Management console
2. Remove an artifact using Management console
3. Create and save an artifact in local repository of the manager node
4. Remove an artifact in local repository of the manager node
5. Create and save an artifact in local repository of a worker node
6. Remove an artifact in local repository of a worker node

Usecase 1 - Create an artifact using Management console

In here, I'm going to create a sample address endpoint and check how the synchronization process happened in manager and worker nodes.

1. Go to management console of the manager node, select Endpoints under Service Bus.

2. Then select 'Add Endpoint' by choosing 'address endpoint' as the type

3. Give a name and address to the Endpoint and press 'Save and Close'

Now observe the terminals of both manager and worker nodes.

Manager Node

Worker node

Once an artifact has been deployed in central repository (We use a SVN repository), manager node will send an hazlecast message to the all worker nodes indicating that the autocommiting of a change has been done. So worker nodes receive the same hazlecast message ( check messageId ) and the process of checking out (autocheckout) the changes from the central repository to its local repository happens.

Usecase 2 - Remove an artifact using Management console

Same process like the Usecase 1 happens here. Once an artifact has been removed from the repository, manager node will commit it in central repository and parallel send a message to worker nodes to checkout the changes. But check the below output of the worker node which shows the undeployment of the removed artifact from its local repository.

Worker Node

Usecase 3 - Create and save an artifact in local repository of the manager node

In this use case I'm going to save an endpoint manually in the local repository (<PRODUCT_HOME>/repository/deployment/server/synapse-configs/default) of the manager node.

Following is the sample configuration of a proxy which includes a send mediator

<proxy name="PassTroughTest" startonload="true" trace="disable" transports="http https" xmlns="http://ws.apache.org/ns/synapse">
<target>
<endpoint>
<address uri="http://10.100.7.11:9768/java_first_jaxws/services/hello_world">
</address>
</endpoint>
<outsequence>
<send>
</send></outsequence>
</target>
</proxy>

Go to proxy-services folder of the local repository and create an Xml file with the name 'SamplePassThrough' and save above configuration.

Now we have created a sample proxy service successfully. Let's check the terminals of both manager and worker nodes to understand what happened here.

Manager Node

Worker Node

As soon as the file created in the local repository of the manager node, it will auto commit the new configured artifact in central SVN repository with the name of the proxy service. Although the message communication happens after that and worker node receives the message, checking out from the Svn repo and deployed the proxy service in local repository of the each worker node.

I will explain other 3 use cases in my next post. If you have questions, don't hesitate to drop a comment.

Use SVN Deployment Synchronizer to sync artifacts in multiple nodes of a WSO2 cluster

Deployment Synchronizer has the ability to sync deployed artifacts in a multiple nodes cluster as soon as the deployment happened. It's necessary to have the same configuration among each node of the cluster since the whole mechanism is to work as a single system (virtually). If a particular node doesn't have the same configuration like the rest of the nodes, it breaks the clustering model. As an example if a proxy service which doesn't include in Node 1, has been invoked through the load balancer and it won't fulfill the work division among nodes as it intended to be.

WSO2 products use SVN (SubVersioN) deployment synchronizer in order to have the syncing process of deployed artifacts like endpoints, proxy services, sequences and etc. The Subversion repository is used to sync the contents in the local sync directory (the axis2 repo directory, /repository/deployment/server, by default).

Now let's see how to setup the SVN DepSync repository with WSO2 products. Before going further into setup, first of it's necessary to select the suitable SVN version. It's advised to only select SVN version 1.7 or 1.8 for the products based on Carbon 4.4.x ( if you're using products based on Carbon 4.2.x and below, please use only SVN version 1.6 ).

Following files needed to be downloaded and save in the product environment to continue with SVN Dep Sync configuration.

• Download SVNKit from here and save it into the <PRODUCT_HOME>/repository/components/dropins folder. 

• In order to function the SVNKit properly, download trilead-ssh2-1.0.0-build215.jar file from here and save it in <PRODUCT_HOME>/repository/components/lib folder. 

After completing the pre-requesties, let's move to configure DepSync on manager and worker nodes. 

Deployment synchronizer configuration done in /repository/conf/carbon.xml file and go to <DeploymentSynchronizer> sub section in carbon.xml to make necessary changes.


Steps to Enable DepSync on the manager node

• To enable the Deployment synchronizer, the value should be set to 'true' for <Enabled> tag 
• Since manager node responsible for receiving server requests and committing changes in local repository to central repository, the value for <AutoCommit> should be 'true' 
• <AutoCheckout> value should changed to 'true'. Once changes are committed in central repository, it should automatically checked out them to local repository of the node. 
• We're using Subversion as repository type here :
  <RepositoryType>svn</RepositoryType>
• Location of the SVN repository should be specify:
  <SvnUrl>https://svn.example.org/depSync</SvnUrl>
• <SvnUsername> and <SvnPassword> must specified with the Username and password of SVN repository respectively. If the tenant specific configuration has to be made, set the value for to 'true'. 

That's it with the manager node configuration. So the manager node deployment synchronization configuration should looks like this,

<DeploymentSynchronizer>
<Enabled>true</Enabled>
<AutoCommit>true</AutoCommit>
<AutoCheckout>true</AutoCheckout>
<RepositoryType>svn</RepositoryType>
<SvnUrl>https://svn.example.com/depsync.repo/</SvnUrl>
<SvnUser>rep1</SvnUser>
<SvnPassword>reppass</SvnPassword>
<SvnUrlAppendTenantId>true</SvnUrlAppendTenantId>
</DeploymentSynchronizer>

Now move to worker node configuration.

Steps to Enable DepSync on the worker nodes

Worker node has the same configuration for DepSync like the manager node except the usage of <AutoCommit>. Since worker node doesn't receive any server requests to be handled and the committing part isn't associated with worker nodes, <AutoCommit> value set to be 'false'. This is the DepSync configuration for Worker nodes.

<DeploymentSynchronizer>
<Enabled>true</Enabled>
<AutoCommit>false</AutoCommit>
<AutoCheckout>true</AutoCheckout>
<RepositoryType>svn</RepositoryType>
<SvnUrl>https://svn.example.com/depsync.repo/</SvnUrl>
<SvnUser>rep1</SvnUser>
<SvnPassword>reppass</SvnPassword>
<SvnUrlAppendTenantId>true</SvnUrlAppendTenantId>
</DeploymentSynchronizer>

So that's it about how to configure deployment synchronizer in a clustered environment. I will discuss about some of the use cases of DepSync in my upcoming posts.

How to create a custom log for a proxy service in WSO2 ESB

Log files are vital to identifying any associated errors, security threats and sequence of service execution while using WSO2 carbon products. Usually application and other service logs are write into server logs but if you want to have a separate log for a particular service, you just need to simply add few lines to log4j.properties file. So with that in mind I will explained how to create a customized log file for a particular proxy service using wso2 Enterprise Service Bus.

First open log4j.properties file ( /repository/conf/log4j.properties ) and simply add the following entry.

log4j.category.SERVICE_LOGGER.TestLogProxy=INFO, PROXY_APPENDER
log4j.additivity.PROXY_APPENDER=false
log4j.appender.PROXY_APPENDER=org.apache.log4j.DailyRollingFileAppender
log4j.appender.PROXY_APPENDER.File=${carbon.home}/repository/logs/${instance.log}/testlogproxy${instance.log}.log
log4j.appender.PROXY_APPENDER.Append=true
log4j.appender.PROXY_APPENDER.layout=org.apache.log4j.PatternLayout
log4j.appender.PROXY_APPENDER.layout.ConversionPattern=%d{HH:mm:ss,SSS} [%X{ip}-%X{host}] [%t] %5p %c{1} %m%

In here, 'TestLogProxy' is the name of the proxy service. You can define a location as the value for 'log4j.appender.PROXY_APPENDER.File' property. I have used default path for logs ( ${carbon.home}/repository/logs/) in here.

Then create a proxy service with log mediator ( log category – Info and log level – Full ) and add a send mediator with a service endpoint. You can see the customized log file under specified log path and logs for the 'TestLogProxy' will be logged in this file.