Monday, 17 December 2012

How to create a JSF 2 component

Java Server Pages (JSF) is an extensible JEE web framework.  In this blog post I will demonstrate how to create a simple JSF component.

When developing a JSF component one of the technical aim is to make it as simple and obvious as possible for a developer to use it.  A user of our component shouldn't have to worry about handling call backs, updating state etc.  With this in mind lets begin:


Step 1) Create the java component:  


public class TextToUpperComponent {

  private String input;
  private String output;

  public void convert() {
    output = input.toUpperCase();
  }

  + getters and setters
}

This contains the state and the call back method ( convert() ) to handle button presses.

Step 2) Create the JSF Manatged Bean:.

@ManagedBean
public class MyBean {

  private TextToUpperComponent a = new TextToUpperComponent();
  private TextToUpperComponent b = new TextToUpperComponent();
  private TextToUpperComponent c = new TextToUpperComponent();

  + getters and setters
}

What we've done here, is to make the TextToUpperComponent component a property of the ManagedBean (3 times).   Other listeners/callbacks can be defined to inspect these properties and apply business logic as required.


Step 3)  Define the component -  webapp/resources/texttoupper.xhtml:

<html xmlns="http://www.w3.org/1999/xhtml"
      xmlns:h="http://java.sun.com/jsf/html"
      xmlns:composite="http://java.sun.com/jsf/composite"
      xmlns:f="http://java.sun.com/jsf/core"
      xmlns:ui="http://java.sun.com/jsf/facelets">

  <composite:interface>
    <composite:attribute name="title" />
    <composite:attribute name="jcomponent" />
  </composite:interface>

  <composite:implementation>
    <h:form id="myform">
      <h:panelGrid>
        <h:outputLabel value="#{cc.attrs.title}: " for="word" />
        <h:inputText id="word" value="#{cc.attrs.jcomponent.input}" />
        <h:outputText value="#{cc.attrs.jcomponent.output}" />
        <h:commandButton value="To Upper">
          <f:ajax listener="#{cc.attrs.jcomponent.convert()}" execute="myform" render="myform"/>
        </h:commandButton>
        <br/>
      </h:panelGrid>
    </h:form>
  </composite:implementation>
</html>


As we're placing it under webapp/resources/coder36, the component namespace will be: xmlns:c="http://java.sun.com/jsf/composite/coder36".

A point to note is that once the JSF page is rendered, the components are completely independent of each other, so there is no need to worry about clashing id's etc.


Step 4)  Create index.xhtml Page

Don't forget to include the namespace: 


<html xmlns="http://www.w3.org/1999/xhtml"
      xmlns:p="http://primefaces.org/ui"
      xmlns:h="http://java.sun.com/jsf/html"      
      xmlns:f="http://java.sun.com/jsf/core"
      xmlns:c="http://java.sun.com/jsf/composite/coder36"
      xmlns:ui="http://java.sun.com/jsf/facelets"> 

 <h:head></h:head> 
 <h:body>
   <c:texttoupper jcomponent="#{myBean.a}" title="First"/>      
   <c:texttoupper jcomponent="#{myBean.b}" title="Second"/>
   <c:texttoupper jcomponent="#{myBean.c}" title="Third"/>
 </h:body> 
</html>


And that's it!  With very little effort we have created a simple reusable JSF component and demonstrated how to integrate it.






Wednesday, 14 November 2012

Setting up a Jboss 7.1 Cluster

Sadly, setting up clustering in jboss 7.1.1 is not as straight forward as it should be.  After hours of googling and numerous false starts, I finally got it all working.  Here are some of the gotcha's which tripped me up.

My starting assumption is that we're running a domain.  Out of the box, the Master node defines 3 servers:
 Server-1    - main-server-group
 Server-2    - other-server-group
 Server-3    - main-server-group

Server-1 and Server-2 are configured to start automatically.  In a cluster these are of no use - disable them.  We're only interested in Server-3 which uses the full-ha profile.

Before we go on, a quick heads up:

  • You only need to amend the domain.xml on the master node
Some helpfull links:


Gotcha #1

When you start up the nodes using the domain.bat scripts, always provide the bind ip and ports.  If you don't then you seem to get some strange behaviours:

domain.sh-b 192.168.0.24 -bmanagement 192.168.0.24     (this is the master)
and
domain.sh-b 192.168.0.25 -bmanagement 192.168.0.25


Gotcha #2

Server-3 does not start cleanly out of the box, due to a mismatch between the full-ha progile and the socket bindings.  In domain.xml, amend as follows:


<server-group name="other-server-group" profile="full-ha">
  <jvm name="default"
    <heap size="64m" max-size="512m"/>
  </jvm>
  <socket-binding-group ref="full-ha-sockets"/>
</server-group>


This will allow Server-2 to start cleanly.

Gotcha #3

The hornetq-server needs configuring with a user name and password.  If you don't you will get an exception along the lines of:

[Server:server-three] 16:24:36,875 ERROR [org.hornetq.core.protocol.core.impl.Ho
rnetQPacketHandler] (Old I/O server worker (parentId: 2128830893, [id: 0x7ee361a
d, /192.168.0.24:5695])) Failed to create session : HornetQException[errorCode=1
05 message=Unable to validate user: HORNETQ.CLUSTER.ADMIN.USER]

Update domain.xml with:
<subsystem xmlns="urn:jboss:domain:messaging:1.1">
  <hornetq-server>                
  <cluster-user>fred</cluster-user>
  <cluster-password>password</cluster-password>


It looks as though you can use any user and password.  It doesn't appear to be authenticated against anything!

Gotcha #4

Use the correct version of @Clustered annotation!   If you don't then in a failover situation you will get the "No EJBReceiver" exception.  The version of @Clustered your'e after is org.jboss.ejb3.annotation.Clustered.  This comes from the jboss-ejb3-ext-api dependency, which is only available from the jboss repo.  Assuming that you are using maven add the following to the pom.xml:

<dependencies>
 ...

  <dependency>
    <groupId>org.jboss.ejb3</groupId>
    <artifactId>jboss-ejb3-ext-api</artifactId>
    <version>2.0.0-beta-3</version>
    <scope>provided</scope>
  </dependency>

<dependencies>
...
<repositories>
  <repository>
    <id>JBoss repository</id>
    <url>http://repository.jboss.org/nexus/content/groups/public/</url>
  </repository>
<repositories>

Then correctly annotate the bean:

import org.jboss.ejb3.annotation.Clustered;

@Stateful
@Clustered
@Remote( coder36.demo.service.MyService.class )
public class MyServiceImpl implements MyService {
...

Gotcha #5

When injecting the EJB do not use the @Inject annotation!   Use @EJB on it's own.  No JNDI name is required:


@ManagedBean
public class PersonController implements Serializable {
  @EJB
  private MyService myService;


Note - the @EJB annotation is available via the following dependency:


<dependency>
  <groupId>org.jboss.spec.javax.ejb</groupId>
  <artifactId>jboss-ejb-api_3.1_spec</artifactId>
</dependency>


Gotcha #6

To make the web application cluster-able ensure that webapp/WEB-INF/web.xml exists, and has the <distributable/> tag:


<web-app xmlns="http://java.sun.com/xml/ns/javaee"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://java.sun.com/xml/ns/javaee 
    http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd"
    version="3.0"> 

  <distributable/> 
</web-app>


To be extra helpful, I've provided the web.xml namespaces :)

Gotcha #7

Assuming you have downloaded and installed mod_cluster 1,2.Final from here.

Out of the box, mod_cluster does't  not work with jboss 7.1.1. A few tweaks are needed.  Keeping things as simple as possible, I've stripped the provided httpd.conf down the bare minimum:


LoadModule authz_host_module modules/mod_authz_host.so
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_ajp_module modules/mod_proxy_ajp.so
LoadModule proxy_http_module modules/mod_proxy_http.so
LoadModule proxy_cluster_module modules/mod_proxy_cluster.so
LoadModule manager_module modules/mod_manager.so
LoadModule slotmem_module modules/mod_slotmem.so
LoadModule advertise_module modules/mod_advertise.so

Listen 192.168.0.24:80
Listen 192.168.0.24:10001

<VirtualHost 192.168.0.24:10001>
  <Location />
    Order deny,allow
    Deny from all
    Allow from all
  </Location>

  KeepAliveTimeout 300
  MaxKeepAliveRequests 0

  EnableMCPMReceive

  <Location /mod_cluster_manager>
    SetHandler mod_cluster-manager
    Order deny,allow
    Deny from all
    Allow from all
  </Location>
</VirtualHost>
ErrorLog "logs/error_log"


I used port 10001 as the default port 6666 didn't work for me.  This httpd.conf should be enough to get you going.

The domain.xml also needs updating to link it up with the mod_cluster instance:


<subsystem xmlns="urn:jboss:domain:modcluster:1.0">
  <mod-cluster-config advertise-socket="modcluster" 
                      proxy-list="192.168.0.24:10001">
  ...


and

<subsystem xmlns="urn:jboss:domain:web:1.1" 
            default-virtual-server="default-host" 
            instance-id="${jboss.node.name}" 
            native="false">
...


Testing

To test everything is working, fire up the domain, deploy an application and start httpd (on windows 7 I needed to run it as administrator otherwise it complained about file permissions).  Navigate to:  http://192.168.0.24:10001/mod_cluster_manager.  You should see:



The web application can be now accessed through the mod_cluster proxy sitting on http://192.168.0.24 and http://192.168.0.24:10001:

ie. http://192.168.0.24/demo/index/jsf

Monday, 12 November 2012

How to set up a Jboss 7.1 domain

With the new JBOSS 7.1 domain controller, a cluster of jboss instances can be administered via a single console.  In this tutorial we will step through the simple process of setting up a domain controller, and 3 nodes.

Target Architecture:

Simplistically we are aiming for the following architecture:



I've assumed IP addresses, passwords and names to keep thing simple.

Later on we will need base64 version of the identity.  I used this website to generate the following base64 strings:

Identity     Base64
node1id      bm9kZTFpZA==
node2id      bm9kZTJpZA==
node3id      bm9kZTNpZA==

1) Configure the ManagementRealm

The nodes authenticates with the domain controller (DC) by passing it's node name and password.  The DC does a name/password lookup against the ManagementRealm.

so... the first step is to configure the ManagementRealm.

On the domain controller run:
add-user node1    (when prompted set password to node1id)
add-user node2    (when prompted set password to node1id)
add-user root      (when prompted set password to password)

(The root user is used for logging into the jboss console.)

Start up the domain controller:
domain -b 192.168.0.24 -bmanagement 192.168.0.24

2) Configure Node 1

Open host.xml (under jboss-as-7.1.1.Final\domain\configuration) and edit as follows:

a) Update <host tag to read:
<host name="node1" xmlns="urn:jboss:domain:1.2">

b) Add server-identities section to <security-realm name="ManagementRealm">


<security-realm name="ManagementRealm">
  <server-identities>
    <secret value="bm9kZTFpZA=="/>
  </server-identities>

  <authentication>
    ...
  </authentication>
</security-realm


The secret value is the base64 encoded version of the Node password.

c) Edit the <domain-controller> section to read:

<domain-controller>

  <remote host="${jboss.domain.master.address:192.168.0.24}"
          port="${jboss.domain.master.port:9999}" 
          security-realm="ManagementRealm"/>
</domain-controller>


Don't forget to add the the security-realm attribute otherwise you will get messages like:


[Host Controller] 16:26:06,454 ERROR [org.jboss.remoting.remote.connection] (Rem
oting "host1:MANAGEMENT" read-1) JBREM000200: Remote connection failed: javax.se
curity.sasl.SaslException: Authentication failed: all available authentication m
echanisms failed
[Host Controller] 16:26:06,469 ERROR [org.jboss.as.host.controller] (Controller
Boot Thread) JBAS010901: Could not connect to master. Aborting. Error was: java.
lang.IllegalStateException: JBAS010942: Unable to connect due to authentication
failure.


d) Start up the node
domain.bat -b 192.168.0.25 -bmanagment 192.168.0.25

3) Configure Node 2

Configure as above,  setting <host name="node2" and <secret value="bm9kZTJpZA=="

4) Configure Node 3

Configure as above,  setting <host name="node3" and <secret value="bm9kZTNpZA=="

Testing

If everything is working, then open a browser and navigate to: http://192.168.0.24:9990/console using root/password when prompted.  On the left hand side, there should be a list of nodes.


Friday, 9 November 2012

How to enable security logging in Jboss 7

To view what's happening within the jboss 7 security subsystem, you need to enable logging of org.jboss.security.

Assuming standalone.xml, update the <subsystem xmlns="urn:jboss:domain:logging:1.1"> section, adding a new console-handler which deals with TRACE level messages only:


<console-handler name="CONSOLE_TRC">
  <level name="TRACE"/>
  <formatter>
    <pattern-formatter pattern="%d{HH:mm:ss,SSS} %-5p [%c] (%t) %s%E%n"/>
  </formatter>
</console-handler>

and link it up to a new org.jboss.securty logger:

<logger category="org.jboss.security">
  <level name="TRACE"/>
  <handlers>
    <handler name="CONSOLE_TRC"/>
  </handlers>
</logger>

Finally, restart the jboss server :)

Thursday, 8 November 2012

Securing JBoss 7 management console using SSL

This tutorial describes how to secure the JBOSS 7 web management console and the CLI management interface.   For clarity, it assumes you're running a stand alone jboss server.

Create a java keystore

Start by creating a java keystore with a self signed certificate:
c:
mkdir ssl
cd ssl
keytool -genkey -keyalg RSA -alias mycert -keystore keystore.jks -storepass   password -keypass password -validity 360 -keysize 2048 -dname "cn=coder36.com, O=Coder36 L=Newcastle, S=England, C=GB"

The keystore password is: password and the private key password is: password

View the certificate using:
keytool -list -v -keystore keystore.jks -alias mycert -storepass password

c:/ssl/keystore.jks will now  contain your private key and a self signed certificate.

Enable SSL

The next step is to enable SSL on the management interfaces.  Edit standalone.xml.  Search for the <management> tags and update.  I've highlighted what's changed from out of the box:




<management>
  <security-realms>
    <security-realm name="ManagementRealm">
      <server-identities>
        <ssl protocol="TLSv1">
          <keystore path="C:/ssl/keystore.jks" password="password"/>
        </ssl>
      </server-identities>
      <authentication>
        <properties path="mgmt-users.properties" relative-to="jboss.server.config.dir"/>
      </authentication>
    </security-realm>
    <security-realm name="ApplicationRealm">
      <authentication>
        <properties path="application-users.properties" relative-to="jboss.server.config.dir"/>
      </authentication>
    </security-realm>
  </security-realms>
  <management-interfaces>
    <native-interface security-realm="ManagementRealm">
      <socket-binding native="management-native"/>
    </native-interface>
    <http-interface security-realm="ManagementRealm">
      <socket-binding https="management-https"/>
    </http-interface>
  </management-interfaces>
</management>

Testing

Restart the JBoss standalone server and test:
Open a web and navigate to: https://localhost:9443/console. You will be presented with the usual warnings about using untrusted certificates   Interestingly, the https console does not work with Google Chrome, but works fine with Internet Explorer, and what ever is built into eclipse.

Test the CLI using:
jboss-cli.sh -c


Using this solution we can prove that the jboss server we are calling is who we think we are calling.  It's also more secure in that it encrypts traffic to and from the server.

Comments

With a few changes, it straight forward to get the management console protected with SSL.

I'm of the opinion that java key stores are a bit clunky and unfriendly.  I much prefer the .pem format with openssl etc.  From my commercial experience, it's generally a bad idea to use java to do the SSL work.  Architecturally it would be better to offload the SSL termination to dedicated hardware (CISCO provides ACE modules to do this kind of work), or dedicated software like apache - see one of my earlier posts for how this could be done.

 



Saturday, 3 November 2012

TVHeadend on the Raspberry PI

Tvheadend

Tvheadend is a linux service which converts a dvb-t feed into a network stream.  This  enables you to stream live tv feed from your raspberry pi to any device on your LAN.  The aim of this tutorial to set up a Tvheadend streaming server on the Raspberry PI, then watch it via an XBMC client on the Nexus 7.


Pre-requisites - Hardware

1) Raspberry PI  (I've got the newer 512MB version)
2) Powered USB hub (New Link 4 Port USB Hub with Mains Adaptor from modmypi )
3) USB DVB-T Tuner (Hauppauge WinTV Nova-TD)
4) USB WIFI adapter (Edimax EW-7811UN Wireless)
5) Memory card (Lexar class 10 32GB micro SD inside an SD card adapter)
6) Keuboard and mouse (I've got a wireless "MINI ULTRA SLIM DESIGN LAPTOP USB KEYBOARD MOUSE COMBO" from here)




Pre-requisistes OS

The operating system of choice is Raspbian (Wheezy), available from here.  Expand the root partition to fill up the SD card, otherwise you''ll run out of space.

--------------

so lets begin....

sudo apt-get update

sudo apt-get install libcurl4-openssl-dev git

Set up the NOVA-TD DVB-T TV tuner

Plug in the tv-tuner then run
dmesg

You will see something along the lines of:

[ 1138.657011] usb 1-1.3.2: new high-speed USB device number 7 using dwc_otg
[ 1138.758550] usb 1-1.3.2: New USB device found, idVendor=2040, idProduct=9580
[ 1138.758581] usb 1-1.3.2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 1138.758599] usb 1-1.3.2: Product: NovaT 500Stick
[ 1138.758611] usb 1-1.3.2: Manufacturer: Hauppauge
[ 1138.758624] usb 1-1.3.2: SerialNumber: 4027809413
[ 1138.865068] IR NEC protocol handler initialized
[ 1138.896509] IR RC5(x) protocol handler initialized
[ 1138.929688] IR RC6 protocol handler initialized
[ 1138.957795] IR JVC protocol handler initialized
[ 1138.968073] dib0700: loaded with support for 21 different device-types
[ 1138.968456] dvb-usb: found a 'Hauppauge Nova-TD Stick/Elgato Eye-TV Diversity' in cold state, will try to load a firmware
[ 1138.991871] IR Sony protocol handler initialized
[ 1139.025138] IR MCE Keyboard/mouse protocol handler initialized
[ 1139.026512] dvb-usb: did not find the firmware file. (dvb-usb-dib0700-1.20.fw) Please see linux/Documentation/dvb/ for more details on firmware-problems. (-2)
[ 1139.026723] usbcore: registered new interface driver dvb_usb_dib0700
[ 1139.057895] lirc_dev: IR Remote Control driver registered, major 251
[ 1139.062582] IR LIRC bridge handler initialized


The problem here is that the dvb-usb module failed to find the nova-td firmware file (dvb-usb-dib0700-1.20.fw)  A quick google reveals that the best place to find tv tuner firmware for linux is here

cd ~
wget http://linuxtv.org/downloads/firmware/dvb-usb-dib0700-1.20.fw
sudo cp dvb-usb-dib0700-1.20.fw /lib/firmware

or more elegantly ...

sudo apt-get install firmware-linux-nonfree

Pull out the tv-tuner, and push back in.

demesg


[ 1660.721801] usb 1-1.3.2: USB disconnect, device number 7

[ 1663.006468] usb 1-1.3.2: new high-speed USB device number 8 using dwc_otg

[ 1663.107981] usb 1-1.3.2: New USB device found, idVendor=2040, idProduct=9580
[ 1663.108013] usb 1-1.3.2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 1663.108029] usb 1-1.3.2: Product: NovaT 500Stick
[ 1663.108042] usb 1-1.3.2: Manufacturer: Hauppauge
[ 1663.108054] usb 1-1.3.2: SerialNumber: 4027809413
[ 1663.118347] dvb-usb: found a 'Hauppauge Nova-TD Stick/Elgato Eye-TV Diversity' in cold state, will try to load a firmware
[ 1663.173435] dvb-usb: downloading firmware from file 'dvb-usb-dib0700-1.20.fw'
[ 1663.378063] dib0700: firmware started successfully.
[ 1663.886683] dvb-usb: found a 'Hauppauge Nova-TD Stick/Elgato Eye-TV Diversity' in warm state.
[ 1663.889868] dvb-usb: will pass the complete MPEG2 transport stream to the software demuxer.
[ 1663.890139] DVB: registering new adapter (Hauppauge Nova-TD Stick/Elgato Eye-TV Diversity)
[ 1664.159908] DVB: registering adapter 0 frontend 0 (DiBcom 7000PC)...
[ 1664.201931] MT2266: successfully identified
[ 1664.373180] dvb-usb: will pass the complete MPEG2 transport stream to the software demuxer.
[ 1664.373438] DVB: registering new adapter (Hauppauge Nova-TD Stick/Elgato Eye-TV Diversity)
[ 1664.552728] DVB: registering adapter 1 frontend 0 (DiBcom 7000PC)...
[ 1664.557289] MT2266: successfully identified
[ 1664.766474] Registered IR keymap rc-dib0700-rc5
[ 1664.767378] input: IR-receiver inside an USB DVB receiver as /devices/platform/bcm2708_usb/usb1/1-1/1-1.3/1-1.3.2/rc/rc0/input3
[ 1664.767916] rc0: IR-receiver inside an USB DVB receiver as /devices/platform/bcm2708_usb/usb1/1-1/1-1.3/1-1.3.2/rc/rc0
[ 1664.769460] dvb-usb: schedule remote query interval to 50 msecs.
[ 1664.769497] dvb-usb: Hauppauge Nova-TD Stick/Elgato Eye-TV Diversity successfully initialized and connected.

That was easy enough!  This could easily have been a nightmare, but by the looks of it Raspbian has the correct kernel modules loaded.   Now that the tv tuner has been detected and initialised we can proceed with building tvheadend. 

Building tvheadend from source


cd ~
git clone -b release/3.2 https://github.com/andoma/tvheadend.git
cd tvheadend
./configure
make
sudo make install


This will install the tvheadend binary: /usr/local/bin/tvheadend
This targets release 3.2, but by leaving off -b release/3.2 you can target the bleeding edge!

Make it auto start at boot

Create tvheadend user home and group

sudo groupadd tvheadend
sudo useradd -g tvheadend -G video -m tvheadend


Create file named /etc/init.d/tvheadend



#!/bin/bash

TVHNAME="tvheadend"
TVHBIN="/usr/local/bin/tvheadend"
TVHUSER="tvheadend"
TVHGROUP="tvheadend"

case "$1" in
  start)
    echo "Starting tvheadend"
    start-stop-daemon --start --user ${TVHUSER} --exec ${TVHBIN} -- \
                -u ${TVHUSER} -g ${TVHGROUP} -f -C
  ;;
  stop)
    echo "Stopping tvheadend"
    start-stop-daemon --stop --quiet --name ${TVHNAME} --signal 2
  ;;
  restart)
    echo "Restarting tvheadend"

    start-stop-daemon --stop --quiet --name ${TVHNAME} --signal 2


    start-stop-daemon --start --user ${TVHUSER} --exec ${TVHBIN} -- \
                -u ${TVHUSER} -g ${TVHGROUP} -f -C

  ;;
  *)
    echo "Usage: tvheadend {start|stop|restart}"
    exit 1
esac
exit 0

Set the permissions to make it runnable:
sudo chmod 755 /etc/init.d/tvheadend

Enable start during boot process
sudo update-rc.d tvheadend defaults

Finally start it manually so we can configure it:
sudo /etc/init.d/tvheadend start 

Configuration


Navigate to http://<rasppi_ip>:9981

Configuration -> General : Set the language (and hit 'Save Changes')



Configuration->TV Adapters: Select the Adapter 



Hit Add DVB Network by location and select Generic->auto_Default 


Now wait while the Muxes have been scanned for services.  This will take a long time.  The right hand panel will update as channels (services) are discovered.



Hit Map DVB services to channels (once it becomes enable) and wait.  In the meantime hit the services tab, to see what's been found.  If everything is working as it should, there should be a list of TV and radio channels.


Select Configuration->Channels to view mapped channels


XBMC On the Nexus 7

All that's left to do is to install XBMC on the nexus 7.  The easiest way to do this, is by installing the 'nightlies' from the XDA forums (for the nexus 7 it's the non-neon build you need).  

To connect to TVHeadend, add the "Tvheadend HTSP Client" Addon:
System -> Settings -> Add-ons -> Search  
Type: Tvheadend

Enable and Configureas follows:

Tvheadend hostname or IP address :  <RaspPI+IP>
HTTP port:  9981
HTSP port: 9982

System -> Settings -> Live TV -> General    : set to Enabled

That's it... Go to Live-TV -> TV Channels




The XBMC tvheadend client is not the finished article.   The EPG looks a bit screwed up, as do some of the channel names.  Also not all the channels appear to be present.  Apart from the above niggles, the XBMC client looks amazing and seems to fit the bill.

TVHGuide

Available from here.  It's not in the google play store, so you need to  download it and install it by hand. In comparison to XBMC, this seems to be the more finished product although the interface could do with some polish - its fairly basic!  The EPG works, although is a bit industrial.  The configuration is intuitive, and as a bonus it can use an external player like the amazing MXPlayer

Until the bugs in XBMC can be resolved, I'll be sticking with TVHGuide.

Thursday, 27 September 2012

Apache SSL reverse proxy tutorial

In this tutorial, we will take a high level requirement, work out a design, then prototype a solution.

High level Requirement

  • All communication  must be encrypted and trusted

Analysis

In terms of our java application which calls a remote web service, this means:
  1. We need to authenticate the remote web service - they are who they say they are
  2. The remote web service needs to authenticate us -  we are who we say we are
  3. http calls to be encrypted

A good technology fit for this problem is  SSL with mutual authentication.  We will use apache as an SSL  reverse proxy (this will forward our plain http requests to the remote web service, applying SSL).  Will use certificate based authentication to prove the authenticity of the server and client.

What is a Certificate trust chain?

Certificate chains are all about trust.  In the diagram below the certificate authority ca.zaltek has signed the certificates for "Bob Ltd" and "Fred Ltd".  The certificate authority is saying "I trust Bob Ltd and Fred Ltd".



Now the leap - If I trust that certificate authority (ca.zaltek) will only sign certificates for legitimate organizations, then I can trust any organization who presents me with a certificate signed by ca.zaltek.  

That's SSL authentication in a nutshell

Target Architecture

   



In the next section we will create 2 certificates and get them signed by a certificate authority.  We are going to use the same certificate authority for both, but equally we could used different ones - as long as we trust them.  We will then set up an apache ssl reverse proxy and emulate a remote SSL web servers.

Generate certificates

Will generate private keys and certificates for both the client and the server using openssl and tinyca:

mkdir c:/ssl
mkdir c:/ssl/client
mkdir c:/ssl/server
mkdir c:/ssl/cacerts
mkdir c:/ssl/apache

# Generate private keys and remove password protection
cd c:/ssl/client
openssl genrsa -des3 -out key_.pem 512      (use password)
openssl rsa -in key_.pem -out key.pem      
rm key_.pem
# Generate CSR (Certificate Signing Request)
openssl req -out csr.pem -key key.pem -new  (cn=client.coder36)

cd ../server
openssl genrsa -des3 -out key_.pem 512      (use password)
openssl rsa -in key_.pem -out key.pem
rm key_.pem
# Generate CSR (Certificate Signing Request)
openssl req -out csr.pem -key key.pem -new  (cn=server.coder36)


The next step is to get the CSR's signed.  Openssl can do this, however I prefer tinyca  See here for a quick guide to using tinyca.  Copy the signed certificates into their respective folders ssl/client/cert.pem and ssl/server/cert.pem.  Copy the ca certificate into ssl/cacerts/ca.pem.

Certificates can be viewed using:
openssl x509 -in cert.pem -noout -text

Checking certificates and keys are all working:

The easiest way to confirm that the keys and certificates are playing nicely is to use openssl as a server.
cd c:/ssl/server
openssl s_server -accept 443 -www -key key.pem -cert cert.pem -CAfile ../cacerts/ca.pem -Verify 1



This will set the server listening on port 443 (the SSL port)

In another session, start the client:

openssl s_client -connect localhost:443 -verify 1 -CAfile ../cacerts/ca.pem -cert cert.pem -key key.pem
get /



If everything is as it should be, the client will perform the SSL handshake without any errors as per the above  picture.

Okay, so this is the base line.  We know the certificates work.  The next step is to get the certificates working with apache.

Setting up apache

Download the latest version of apache (2.4) from here.  Extract to c:/

Combine server certificate and key into single file
cd c:/ssl/server
cat cert.pem > certandkey.pem
cat key.pem >> certandkey.pem

Create apache config c:/ssl/apache/proxy.conf :

LoadModule authz_core_module modules/mod_authz_core.so
LoadModule logio_module modules/mod_logio.so
LoadModule log_config_module modules/mod_log_config.so
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_http_module modules/mod_proxy_http.so
LoadModule ssl_module modules/mod_ssl.so

LogFormat "%h %l %u %t \"%r\" %>s %b" common
CustomLog "c:/ssl/apache/access.log" common
ErrorLog "c:/ssl/apache/error.log"
LogLevel warn

Listen 80

<VirtualHost localhost:80>
   SSLProxyEngine on
   SSLProxyCACertificateFile "c:/ssl/cacerts/ca.pem"
   SSLProxyVerify require
   SSLProxyVerifyDepth  2
   SSLProxyMachineCertificateFile "c:/ssl/server/certandkey.pem"
   <Proxy *>
   </Proxy>

   <Location />
     ProxyPass          https://server.coder36:443/
     ProxyPassReverse   https://server.coder36:443/
   </Location>
</VirtualHost>


Update  hosts file:
127.0.0.1 server.coder36
127.0.0.1 client.coder36


Testing

Start up apache:
cd c:/apache24/bin
httpd -f c:/ssl/apache/proxy.conf

In another session, start the openssl server:

cd c:/ssl/server
openssl s_server -accept 443 -www -key key.pem -cert cert.pem -CAfile ../cacerts/ca.pem -Verify 1

Test 1

        http            https
Browser -------> Apache -------> Openssl Server


Fire up a browser and go to: http://server.coder36.  If  everything is working you should be presented with the openssl server diagnostic page.

Test 2
         http            https
Java App -------> Apache -------> Openssl Server


To test from a java app use the following code:

import java.io.*;
import java.net.*;

public class Client {
  public static void main( String [] args ) {
    try {
      URL url = new URL( args[0] );
      URLConnection con = url.openConnection();
      InputStream is = con.getInputStream();
      BufferedReader br = new  BufferedReader( new InputStreamReader( is ));
      String s = "";
      while ( (s = br.readLine()) != null ) {
     System.out.println( s );
      }
      is.close();
    }
    catch( Exception e ) {
      e.printStackTrace();
    }
  }
}

To run::
java Client http://server.coder36


Resources

CA Certificate:
-----BEGIN CERTIFICATE-----
MIIEGzCCA4SgAwIBAgIJALf/i8hTdAG8MA0GCSqGSIb3DQEBBQUAMIGPMQswCQYD
VQQGEwJVSzEQMA4GA1UECBMHRW5nbGFuZDEcMBoGA1UEBxMTTmV3Y2FzdGxlLXVw
b24tdHluZTETMBEGA1UEChMKWmFsdGVrIEx0ZDESMBAGA1UEAxMJY2EuemFsdGVr
MScwJQYJKoZIhvcNAQkBFhhtYXJreW1pZGRsZXRvbkBnbWFpbC5jb20wHhcNMTIw
NDA1MDI0MzQ2WhcNMjIwNDAzMDI0MzQ2WjCBjzELMAkGA1UEBhMCVUsxEDAOBgNV
BAgTB0VuZ2xhbmQxHDAaBgNVBAcTE05ld2Nhc3RsZS11cG9uLXR5bmUxEzARBgNV
BAoTClphbHRlayBMdGQxEjAQBgNVBAMTCWNhLnphbHRlazEnMCUGCSqGSIb3DQEJ
ARYYbWFya3ltaWRkbGV0b25AZ21haWwuY29tMIGfMA0GCSqGSIb3DQEBAQUAA4GN
ADCBiQKBgQCv/X87xEwkmpoKMQE+KA5BHhCnF70OOcvn9sO/FONvNIX7DA7HUe6Y
WrrE8nKHLWt8XTDvbJKzbq2DyxOhPJZ759IbDBlmUpHQDfPexBL5Rg7AJR41TMAw
SQAqjGk03hPckptWxK1mEzib9X8XO2dkUe9KUuSqoYiht6ScxBXsNwIDAQABo4IB
ezCCAXcwHQYDVR0OBBYEFEpA7/Lo5wBFi7t725ViT6+8hp5SMIHEBgNVHSMEgbww
gbmAFEpA7/Lo5wBFi7t725ViT6+8hp5SoYGVpIGSMIGPMQswCQYDVQQGEwJVSzEQ
MA4GA1UECBMHRW5nbGFuZDEcMBoGA1UEBxMTTmV3Y2FzdGxlLXVwb24tdHluZTET
MBEGA1UEChMKWmFsdGVrIEx0ZDESMBAGA1UEAxMJY2EuemFsdGVrMScwJQYJKoZI
hvcNAQkBFhhtYXJreW1pZGRsZXRvbkBnbWFpbC5jb22CCQC3/4vIU3QBvDAPBgNV
HRMBAf8EBTADAQH/MBEGCWCGSAGG+EIBAQQEAwIBBjAJBgNVHRIEAjAAMCsGCWCG
SAGG+EIBDQQeFhxUaW55Q0EgR2VuZXJhdGVkIENlcnRpZmljYXRlMCMGA1UdEQQc
MBqBGG1hcmt5bWlkZGxldG9uQGdtYWlsLmNvbTAOBgNVHQ8BAf8EBAMCAQYwDQYJ
KoZIhvcNAQEFBQADgYEACVt3BMuusRAOW3G6w4T11FZFbJnKn5/GY5KFZQtn5hFV
AheWyWMrDPcHsxTW8EiGakjqbIjepOuHHFS21NkTWnwaLcQ2WtxwZyuEbee4kUWr
yRxjzsPWfShkAPE+ky6rvPmpORXXjq7ryEJdtdVdW9oobpHqA51j/K1r3wsmwpM=
-----END CERTIFICATE-----

Client Certificate
-----BEGIN CERTIFICATE-----
MIID5TCCA06gAwIBAgIBATANBgkqhkiG9w0BAQUFADCBjzELMAkGA1UEBhMCVUsx
EDAOBgNVBAgTB0VuZ2xhbmQxHDAaBgNVBAcTE05ld2Nhc3RsZS11cG9uLXR5bmUx
EzARBgNVBAoTClphbHRlayBMdGQxEjAQBgNVBAMTCWNhLnphbHRlazEnMCUGCSqG
SIb3DQEJARYYbWFya3ltaWRkbGV0b25AZ21haWwuY29tMB4XDTEyMDQwNTAyNTIy
NVoXDTE3MDQwNDAyNTIyNVowgZQxCzAJBgNVBAYTAlVLMRAwDgYDVQQIEwdFbmds
YW5kMRwwGgYDVQQHExNOZXdjYXN0bGUtdXBvbi10eW5lMRMwEQYDVQQKEwpaYWx0
ZWsgTHRkMRcwFQYDVQQDEw5jbGllbnQuY29kZXIzNjEnMCUGCSqGSIb3DQEJARYY
bWFya3ltaWRkbGV0b25AZ21haWwuY29tMFwwDQYJKoZIhvcNAQEBBQADSwAwSAJB
AKmb4tqyOly9t934sK7MNytO1rmJiNi/VOC0DoNImzxArVV78LznE5oqCjVxcZ5T
8Z44HMt9a5SfshJ4YH2961cCAwEAAaOCAYwwggGIMAkGA1UdEwQCMAAwEQYJYIZI
AYb4QgEBBAQDAgSwMCsGCWCGSAGG+EIBDQQeFhxUaW55Q0EgR2VuZXJhdGVkIENl
cnRpZmljYXRlMB0GA1UdDgQWBBTpnreeFat6HsREismDmymwrnGgpzCBxAYDVR0j
BIG8MIG5gBRKQO/y6OcARYu7e9uVYk+vvIaeUqGBlaSBkjCBjzELMAkGA1UEBhMC
VUsxEDAOBgNVBAgTB0VuZ2xhbmQxHDAaBgNVBAcTE05ld2Nhc3RsZS11cG9uLXR5
bmUxEzARBgNVBAoTClphbHRlayBMdGQxEjAQBgNVBAMTCWNhLnphbHRlazEnMCUG
CSqGSIb3DQEJARYYbWFya3ltaWRkbGV0b25AZ21haWwuY29tggkAt/+LyFN0Abww
IwYDVR0SBBwwGoEYbWFya3ltaWRkbGV0b25AZ21haWwuY29tMCMGA1UdEQQcMBqB
GG1hcmt5bWlkZGxldG9uQGdtYWlsLmNvbTALBgNVHQ8EBAMCBaAwDQYJKoZIhvcN
AQEFBQADgYEAIgOaDKqrgdw3CEojzb6ZRWGI7x/W8pMy9VtSlYzpoDOeQl9GMTZ/
uyLKCA6Xm6rLP81G0aZ/u1KndL9Y5aTFQjPhd1ackNGymT/LuOK4pJlq0DJJuH4q
9JD9b2NwwjTtZ4UHx/W5Q5uq03kFIKIPwgqFnitPJ9xx18kyhDcZx9c=
-----END CERTIFICATE-----


Client Private Key
-----BEGIN RSA PRIVATE KEY-----
MIIBOwIBAAJBAKmb4tqyOly9t934sK7MNytO1rmJiNi/VOC0DoNImzxArVV78Lzn
E5oqCjVxcZ5T8Z44HMt9a5SfshJ4YH2961cCAwEAAQJACz4B820M4UjR/yyUyYFP
xtca9IFhSLyl658Dt0AAyiWDqb0oOiEN6KA4nNjxe2bFmii10y37V6b8FrknWhie
AQIhAN4N7ubIgwOKsE24o4c4zz36ocgNkh8x2hTkfdzY7vhhAiEAw4mBm60gK8b7
UDnEHQ7BddLVCRRxR1bZMdKtx9AvHrcCIQC2oQUE4JMIfa+FLL+qPmoMkq8uhyLP
70Nq/YYa9nyM4QIgA8PsGQFWc0FbqYnTOvQDya8Lpycbn1IO0KTL7+MUZTUCIQDa
AsruwTONhb1A96ljtFci82oER9tkwTdvKsBBvBvJjA==
-----END RSA PRIVATE KEY-----


Server Certificate
-----BEGIN CERTIFICATE-----
MIID2DCCA0GgAwIBAgIBAjANBgkqhkiG9w0BAQUFADCBjzELMAkGA1UEBhMCVUsx
EDAOBgNVBAgTB0VuZ2xhbmQxHDAaBgNVBAcTE05ld2Nhc3RsZS11cG9uLXR5bmUx
EzARBgNVBAoTClphbHRlayBMdGQxEjAQBgNVBAMTCWNhLnphbHRlazEnMCUGCSqG
SIb3DQEJARYYbWFya3ltaWRkbGV0b25AZ21haWwuY29tMB4XDTEyMDQwNTAyNTgy
NloXDTE3MDQwNDAyNTgyNlowgZQxCzAJBgNVBAYTAlVLMRAwDgYDVQQIEwdFbmds
YW5kMRwwGgYDVQQHExNOZXdjYXN0bGUtdXBvbi1UeW5lMRMwEQYDVQQKEwpaYWx0
ZWsgTHRkMRcwFQYDVQQDEw5zZXJ2ZXIuY29kZXIzNjEnMCUGCSqGSIb3DQEJARYY
bWFya3ltaWRkbGV0b25AZ21haWwuY29tMFwwDQYJKoZIhvcNAQEBBQADSwAwSAJB
AN+SXjkPPUjpdglZ1VeoTGjmW+YgWzkrr3a0ZzLRiV1WKi/UzbtTv37PWWDBuRLB
KYxLyfbGLOiBwwurJXMtFtUCAwEAAaOCAX8wggF7MAkGA1UdEwQCMAAwEQYJYIZI
AYb4QgEBBAQDAgZAMCsGCWCGSAGG+EIBDQQeFhxUaW55Q0EgR2VuZXJhdGVkIENl
cnRpZmljYXRlMB0GA1UdDgQWBBTZgIKxgIeIxlvxIovV1fKyvTL8ejCBxAYDVR0j
BIG8MIG5gBRKQO/y6OcARYu7e9uVYk+vvIaeUqGBlaSBkjCBjzELMAkGA1UEBhMC
VUsxEDAOBgNVBAgTB0VuZ2xhbmQxHDAaBgNVBAcTE05ld2Nhc3RsZS11cG9uLXR5
bmUxEzARBgNVBAoTClphbHRlayBMdGQxEjAQBgNVBAMTCWNhLnphbHRlazEnMCUG
CSqGSIb3DQEJARYYbWFya3ltaWRkbGV0b25AZ21haWwuY29tggkAt/+LyFN0Abww
IwYDVR0SBBwwGoEYbWFya3ltaWRkbGV0b25AZ21haWwuY29tMCMGA1UdEQQcMBqB
GG1hcmt5bWlkZGxldG9uQGdtYWlsLmNvbTANBgkqhkiG9w0BAQUFAAOBgQAYjpjF
W4S40VJtqDJ5jciXlzrCs2VmIC3MG22iRKdUzrASH3kphpqFSsV1KXnaZvrRCxZu
+2ADrTFGAzVkS80Zqwhk7+Snq1MANEtt4qCKaBfaKMvaVKDH6JD3BF55M1Rh0fiM
Ey2Qq/X86qjJ4zRlfa8EBEDCLAeg+XoTcOXNBw==
-----END CERTIFICATE-----


Server Key
-----BEGIN RSA PRIVATE KEY-----
MIIBOgIBAAJBAN+SXjkPPUjpdglZ1VeoTGjmW+YgWzkrr3a0ZzLRiV1WKi/UzbtT
v37PWWDBuRLBKYxLyfbGLOiBwwurJXMtFtUCAwEAAQJAH+EeIGLE6V2o4CJ4Ily0
MI0Qc5sPhvJsva9xu7RXMwCvAzpN/7bijBXzanSuveKuVoNQyrv5rQgfP86/s5lg
YQIhAP0lhGFxXepBEZmGEChyQyI3a7i1HtOUXyYQo+/21IEtAiEA4heCcFe+KRiU
svC+TEMz+cQbEQIv+wmEwmc1WruD5UkCIBK17y70nihuXsqmKhOTPnsM9Z1VZc3l
rZK2sCdQayyRAiBGPReA30Fscv5/ONtlxdr31TyzZAKT8rkTzAx/S5IGQQIhAOi+
6gm1IHJagF7n3BJj4K5Sqx2ca4Bo/sn+5ogBuoOp
-----END RSA PRIVATE KEY-----




Certificate Trust chain





Wednesday, 26 September 2012

Spring batch with Hibernate Part 1


Hibernate is a great ORM for web applications (it's traditional usage pattern), but it can also be very good at batch.  A word of warning, though, you need to be very careful as your HQL query can result in multiple SQL calls,  which would be a disaster for batch.

This post describes how hibernate can be configured to operate in a batch mode to minimizes the number of SQL calls to the underlying database.

Enabling batch mode

Configure hibernate with the following properties (see improving hibernate performacne):

hibernate.jdbc.batch_size=1000
hibernate.default_batch_fetch_size=1000

eg.

<bean id="sessionFactory" class="org.springframework.orm.hibernate4.LocalSessionFactoryBean">
  <property name="dataSource" ref="dataSource"/>
  <property name="packagesToScan">
    <list>
      <value>coder36.sbent.sample.domain</value>
      <value>coder36.sbent.sample.domain.stage</value>
    </list>
  </property>
  <property name="hibernateProperties">
    <props>
      <prop key="hibernate.dialect">org.hibernate.dialect.MySQLDialect</prop>
      <prop key="hibernate.jdbc.batch_size">1000</prop>
      <prop key="hibernate.default_batch_fetch_size">1000</prop>   
    </props>
  </property>
</bean

A good tip, is to also enable hibernate SQL logging, so you can see exactly what hibernate is doing behind the scenes (see here for the types of logging output) :

<prop key="hibernate.show_sql">true</prop>
<prop key="hibernate.format_sql">true</prop>
<prop key="hibernate.use_sql_comments">true</prop>


How does hibernate batch mode work?  

The best way to describe it is via an example taken from sbent.  Have the following entities:

@Entity
public class Transaction {
  ...

  @ManyToOne(fetch=FetchType.LAZY) 
  Customer customer;

  public Customer getCustomer() {
    return customer;
  }
}

and

@Entity
public class Customer {
...

@Column( nullable=false, length=60 )
private String name;

public String getName() {
return name;
}

}

To demonstrate what happens under the covers, load some data, then retrieve it with the following code:

List<Transaction> trans = 
    session.createQuery( "select t from Transaction t" ).list();
for ( Transaction t: trans ) {
  System.out.println( "getting customer");
  Customer c = t.getCustomer();
  System.out.println( "name: " + c.getName() );
}

With hibernate SQL logging turmed on, the following is sent to stdout:

getting list of Transaction's
Hibernate: 
    /* select t from Transaction t */ 
    select
        transactio0_.id as id6_,
        transactio0_.amount as amount6_,
        transactio0_.bank_id as bank4_6_,
        transactio0_.customer_id as customer5_6_,
        transactio0_.version as version6_ 
    from
        Transaction transactio0_
getting customer
Hibernate: 
    /* load coder36.sbent.sample.domain.Customer */ 
    select
        customer0_.id as id1_0_,
        customer0_.name as name1_0_,
        customer0_.nino as nino1_0_,
        customer0_.version as version1_0_ 
    from
        Customer customer0_ 
    where
        customer0_.id in ( ?, ?, ? )
name: Mark Middleton
getting customer
name: John Smith
getting customer
name: Fred Dibnah

What's happening here?

The Transaction entity is configured to lazily load its Customer.  So when the customer is first accessed, (c.getName()), the lazy load is triggered.  As batch mode is enabled, hibernate also triggers the lazy load of all Transaction.customer's for all Transaction entities in the cache - hence the customer0_.id in ( ?,?,? ) bit.

Summarize

  • Enable hibernate batch mode by setting hibernate.jdbc.batch_size and  hibernate.default_batch_fetch_size
  • Use lazy loading - ensure hibernate entities are correctly configured with  fetch=FetchType.LAZY
  • Enable SQL logging during unit testing, to prove hibernate is behaving as expected.