Friday, October 29, 2010

XML Parsing with AXIOM

Recently a friend of mine inquired about what the best way is to parse XML. I have to say im not an expert in this subject but from what i have read i instantly remembered that AXIOM is a pull parser which means that when you request for a particular element within your XML document push parsers will give you that exact element where as other pull parsers will build the whole XML document before handing over the relevant document element to you. Hence AXIOM will leave the least memory foot print.

The problem was i could not find a clear and concise tutorial explaining how to deal with XML using AXIOM. After playing around with it i figured out how to manipulate XML with AXIOM which i should say is so much better that the cumbersome code you have to deal with when manipulating with DOM or JDOM.

So following i show a simple example of how to manipulate XML with AXIOM;

First off i present the XML i will be parsing

<?xml version="1.0" encoding="utf-8" ?>
<my_servers>
 <server>
  <server-name>PROD</server-name>
  <server-ip>xx.xx.xx.xx</server-ip>
  <server-port>80</server-port>
  <server-desc>Running A/S</server-desc>
 </server>

 <server>
  <server-name>PROD2</server-name>
  <server-ip>xx1.xx1.xx1.xx1</server-ip>
  <server-port>80</server-port>
  <server-desc>Running A/S</server-desc>
 </server>
</my_servers>

Next i wrote a factory method to handout StaxBuilder instances depending on the XML file you pass. I have done as such so as to minimize the task of creating new StaxBuilder instances everytime.


import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;

import javax.xml.stream.XMLStreamException;

import org.apache.axiom.om.impl.builder.StAXOMBuilder;

public class AxiomStaxBuilderFactory {

    private static Map<String, StAXOMBuilder> staxBuilderMap = new ConcurrentHashMap<String, StAXOMBuilder>();

    /**
     * The factory method stores the {@link StAXOMBuilder} instance created for each XML file<br>
     * passed in so that we do not need to create unnecessary objects every time.<br>
     * An instance of {@linkplain ConcurrentHashMap} is used so as to make the<br>
     * instances thread safe.
     * 
     * @param xmlFilePath the path of the XML file
     * @return an instance of the {@link StAXOMBuilder} from the cache or newly created
     */
    public static StAXOMBuilder getAxiomBuilderForFile(String xmlFilePath) {
        StAXOMBuilder staxBuilder = null;
        if (staxBuilderMap.containsKey(xmlFilePath)) {
            staxBuilder = staxBuilderMap.get(xmlFilePath);
        } else {
            try {
                staxBuilder = new StAXOMBuilder(new FileInputStream(xmlFilePath));
                staxBuilderMap.put(xmlFilePath, staxBuilder);
            } catch (FileNotFoundException e) {
                throw new AxiomBuilderException(e);
            } catch (XMLStreamException e) {
                throw new AxiomBuilderException(e);
            }
        }

        return staxBuilder;

    }
}



I have used a Concurrent Hash map so that this wil work well in a multi threaded application. If your not bothered with that you might as well use a normal HashMap for better performance which in this case would be negligible. I have also used a custom exception as i did not want the user to have to handle exceptions so i wrapped the exceptions thrown to my custom run time exception. Following is that code. Nothing major just a normal extension of the RuntimeException class;

/**
 * This exception class wraps all exceptions thrown from the Axiom API
 * as the user does not need to be bound by such checked exceptions.
 * @author dinuka
 *
 */
public class AxiomBuilderException extends RuntimeException {

    /**
     * 
     */
    private static final long serialVersionUID = -7853903625725204661L;

    public AxiomBuilderException(Throwable ex) {
        super(ex);
    }

    public AxiomBuilderException(String msg) {
        super(msg);
    }
}



Next off i have written a utility class to deal with the XML parsig. Ofcourse this is not needed but i just had it so that client calls will be much cleaner without having to deal with XML releated coding which would be abstracted by the utility class. Note -The current method only reads the root level elements passed in.



import java.util.ArrayList;
import java.util.HashMap;
import java.util.Iterator;
import java.util.List;
import java.util.Map;

import javax.xml.namespace.QName;

import org.apache.axiom.om.OMElement;
import org.apache.axiom.om.impl.builder.StAXOMBuilder;

/**
 * The utility class provides abstraction to users so that <br>
 * the user can just pass in the xml file name and the node he/she<br>
 * wants to access and get the values without having to bother with<br>
 * boilerplate xml handling info.
 * 
 * @author dinuka
 */
public class AxiomUtil {

    /**
     * This method is used if you have for example a node with multiple children<br>
     * Note that this method assumes the node in query is within the root element
     * 
     * @param xmlFilePath the path of the xml file
     * @param nodeName the node name from which you want to retrieve values
     * @return the list containing key value pairs containing the values of the sub elements within<br>
     *         the nodeName passed in.
     */
    public static List<Map<String, String>> getNodeWithChildrenValues(String xmlFilePath, String nodeName) {
        List<Map<String, String>> valueList = new ArrayList<Map<String, String>>();

        StAXOMBuilder staxBuilder = AxiomStaxBuilderFactory.getAxiomBuilderForFile(xmlFilePath);

        OMElement documentElement = staxBuilder.getDocumentElement();
        Iterator nodeElement = documentElement.getChildrenWithName(new QName(nodeName));

        while (nodeElement.hasNext()) {
            OMElement om = (OMElement) nodeElement.next();

            Iterator it = om.getChildElements();
            Map<String, String> valueMap = new HashMap<String, String>();
            while (it.hasNext()) {
                OMElement el = (OMElement) it.next();

                valueMap.put(el.getLocalName(), el.getText());

            }

            valueList.add(valueMap);
        }
        return valueList;
    }

}



And finally i give to you a sample class to test out the XML parsing example i have presented to you here.



import java.util.List;
import java.util.Map;

/**
 * Test class depicting the use of Axiom parsing XML
 * 
 * @author dinuka
 */
public class testServerConfigXML {

    public static void main(String argv[]) {

        List<Map<String, String>> values = AxiomUtil.getNodeWithChildrenValues("/home/dinuka/serverInfo.xml", "server");

        for (Map<String, String> mapVals : values) {
            for (String keys : mapVals.keySet()) {
                System.out.println(keys + "=" + mapVals.get(keys));
            }
        }

    }

}


Thats it. If you have any queries or any improvement points you see pls do leave a comment which would be highly appreciated. Hope this helps anyone out there looking for a similar basic tutorial on AXIOM XML parsing.


Cheers

Thursday, October 14, 2010

My First Experience with MongoDB

I know im pretty much slow to this concept of NoSQL. But nevertheless better late than never right ? :) ... So this is my first post on getting my feet wet in the world NoSQL. I have to say coming from a RDMS background it was not much hard to get my self familiar with MongoDB. Ofcourse there are a number of NoSQL implementations out there. But going through each of them MongoDB was the one i felt inclined to go with as the learning curve for me using it was pretty low.

So i was reading through multiple posts, articles to get a feel of what MongoDB can really do. After going through that i wanted to try out an example to get a feel of it. So i first downloded Mongo DB from here.  Im on Ubuntu 8.04 so i downloaded the Linux 32-bit one. They do say about a limitation of using the 32-bit version. But i was not much concerned as this is my stage of getting used to it.

So coming from a J2EE development background mostly using annotations i was inclined to search for a solution using annotations to deal with MongoDB so that the transition from usual JPA/Hibernate mappings to MongoDB will be minimal. I know that the way of thinking in terms of MongoDB is different to the way we map things in a traditional RDMS as there is no concept as foreign keys etc. But i wanted the transition to be minimal. So going on those steps i found this project done by google supporting annotation based mapping facility for the MongoDB called Morphia. This was exactly what i needed.

So i did a quich write up to test it. You need to download Morphia  and also get the Java driver for MongoDB. Next start your mongodb. Go to your MongoDB installation path's bin directory and do as follows;




So i specified the parameters as 


./mongod --dbpath<path_to_your_data_store>;
This starts Mongo DB in its default port. Afterwards i present to you a simple program using the above mentioned libraries.


I have a simple class called MyInfo which has an embedded object called Address. Code is as follows;

import org.bson.types.ObjectId;

import com.google.code.morphia.annotations.Entity;
import com.google.code.morphia.annotations.Id;



@Entity("myInfo")
public class MyInfo {

    @Id ObjectId id;
    private String name;
    private int age;
    
    private Address address;
    
    public ObjectId getId() {
        return id;
    }
    public void setId(ObjectId id) {
        this.id = id;
    }
    public String getName() {
        return name;
    }
    public void setName(String name) {
        this.name = name;
    }
    public int getAge() {
        return age;
    }
    public void setAge(int age) {
        this.age = age;
    }
    public Address getAddress() {
        return address;
    }
    public void setAddress(Address address) {
        this.address = address;
    }
    
}


import com.google.code.morphia.annotations.Embedded;

@Embedded
public class Address {
    
    
    private String adderss1;
    
    private String address2;

    public String getAdderss1() {
        return adderss1;
    }

    public void setAdderss1(String adderss1) {
        this.adderss1 = adderss1;
    }

    public String getAddress2() {
        return address2;
    }

    public void setAddress2(String address2) {
        this.address2 = address2;
    }
    
    
}


Those were my domain classes. Next i present the main class which does the storing of my objects within the MongoDB.

import java.net.UnknownHostException;

import com.google.code.morphia.Datastore;
import com.google.code.morphia.Morphia;
import com.mongodb.Mongo;
import com.mongodb.MongoException;

/**
 * This class Creates a test Monog Connection and persists an instance of {@linkplain MyInfo}
 * @author dinuka
 *
 */
public class MainMongoPersist {

    public static void main(String[] args) throws UnknownHostException, MongoException {
        
        /**
         * Creates a connection on the local Mongo DB server
         */
        Mongo mon = new Mongo("localhost",27017);
        
        /**
         * Add the classes annotated with @Entity.
         * Note that you do not need to add the Address as we Embedded it within 
         * the MyInfo instance. 
         */
        Morphia morphia = new Morphia();
        morphia.map(MyInfo.class);
        
        /**
         * Create a data source by giving the DB you want to connect to.
         */
        Datastore ds = morphia.createDatastore(mon, "mydb");
        
        MyInfo inf = new MyInfo();
        inf.setName("Roshan123");
        inf.setAge(1);
        Address ad = new Address();
        ad.setAdderss1("No 42");
        ad.setAddress2("3424234234");
        inf.setAddress(ad);
        
        /**
         * Persist the object
         */
        ds.save(inf);
    }
}


That is it. Now i start my mongo process which allows me to query the DB to check whether the data really did get stored within the DB. So i go to my mongo installation path's bin directory and start a mongo client. Following snippet shows how to do that;


dinuka@dinuka:~/software/mongodb-linux-i686-1.6.3/bin$ ./mongo 
MongoDB shell version: 1.6.3
connecting to: test
> use mydb
switched to db mydb
> db.myInfo.find();
{ "_id" : ObjectId("4cb6d81d166cacce8ab4e4e8"), "className" : "MyInfo", "name" : "Dinuka", "stars" : 1 }
{ "_id" : ObjectId("4cb6d8f71757accef6a9784d"), "className" : "MyInfo", "name" : "Roshan", "stars" : 1 }
{ "_id" : ObjectId("4cb6d910f9d0accee936df12"), "className" : "MyInfo", "name" : "Roshan123", "stars" : 1, "address" : { "adderss1" : "fsdfsdfsd", "address2" : "3424234234" } }
> 



I switched the database as i used the database called mydb which is the one i specified when i created the data source. This is just a basic write up on Mongo DB. There is so much more in it which i am on the verge of learning. As it is a very interesting topic to explore on for me. Up to now the potential i see of using Mongo DB is;

  • It allows you to have a dynamic schema so that adding a column will be no hassle. 
  • You can scale the database with ease using the auto sharding facility provided by MongoDB.
  • I love the fact that you can think in terms of how you map your json data in the front end to the database as it is.
Many more is there which im not familiar with so i rather not comment on anything i do not know :) .. If anyone out there can mention any experience using this in a production system with regards to performance and reliability that would be a great help.

Would love to hear your thoughts/comments on this regards.


And the journey of learning Mongo continues............