Tuesday 25 September 2012

Spring Batch in the Enterprise


Large organizations by their nature are reluctant to move wholesale over to new technologies and this can result in a mishmash of new, legacy and bespoke components, all of which want a piece of the action.  They also stick with what they know works.  I'm always looking for software solutions which are a good fit, and sometimes I'm not so sure when it comes to spring batch and the enterprise.

Why Spring batch?

The great thing about spring batch, is that it provides the majority of what you need in  terms of the framework.  This allows you to invest your effort and time writing business code (this is the genius of spring).  What it doesn't do so well, is to bridge onto legacy components, bespoke components and working patterns that prevail in the enterprise.

How can  spring batch be made a better fit for the enterprise?

1) Work out batch patterns which fit with the enterprise. i.e. How does hibernate fit with batch?
2) Bridge onto enterprise components (i.e. the enterprise scheduler)
3) Work with enterprise data strategies (i.e. stream from database, rather than a file)
4) Monitoring tools (i.e. need to know if errors have occured)
5) Testing tools - testing to align with system testers.  In the enterprise it's all about integration testing.

What is SBENT?


Spring Batch Extensions for the Enterprise (SBENT) is a thin software bridge built to span 'out of the box' spring batch and the corporate enterprise. It's a set of Readers, Writers, batch admin extensions, XML processing tasklets and design patterns.

SBENT is released under Apache License Version 2.0.

The sample batch project

The best way to describe SBENT is via a reference batch project.  spent-sample makes use of the various
ItemWriters, ItemReaders and so on provided by SBENT.

The Batch scenario

 1) Bank to push a return in the form of XML onto an AMQP Queue (RabbitMQ).  The return contains all transactions its customers have made across a given period:

<return>
  <header>
    <bankName>Brookside Bank PLC</bankName>
    <periodEnd>2012-10-10</periodEnd>
  </header>
  <customer>
    <name>Mark Middleton</name>
    <nino>AB123001A</nino>
    <transaction><amount>45.11</amount></transaction>
  </customer>
  <customer>
    <name>John Smith</name>
    <nino>AB123002A</nino>
    <transaction><amount>44.44</amount></transaction>
  </customer>
</return>   
   
2) Pick up the XML submission, and save it to the database.
3) Parse the XML submission and save into staging tables
4) Process the staged data, saving it into the "domain model" 
5) Generate "ticker" reporting customers overall transaction balance (a person can be a customer in many banks)
6) Publish ticker to AMQP Queue.

Running the sample batch project

Install the following:
MySQL Workbench - create a database called 'test'

Instructions:

mkdir src
cd src
git clone git://github.com/coder36/sbent.git
cd sbent
mvn clean install
cd sbent-sample
mvn jetty:run    (wait for jetty to start up)

Open another prompt:

cd src/sbent/sbent-sample/bin
sample.bat  (this emulates what a scheduler would do and see)

You should see something along the lines of:



Monitoring links:

Spring batch admin front end - http://localhost:8080/sbent-sample



RabbitMQ Queues - http://localhost:55672/mgmt (guest/guest)



Source code:






No comments:

Post a Comment