Technical: Hadoop – Hbase – Programming – “Hello World”

Technical: Hadoop – Hbase – Programming – “Hello World”

Introduction

So here I am trying to develop a simple HelloWorld program in Hadoop-Hbase.

List of Files

List of Files – Source Files

  • hbaseDemoMedia.mediaNode ( a simple plain object that encapsulates the properties our media object has)
  • hbaseDemoMedia.hbaseDemoDB ( contains most of the methods that directly interacts with Hadoop/HBase)
  • hbaseDemoMedia.hbaseDemoGUI (contains the User Interace; unfortunately this time we are using a very old fashion terminal GUI)

List of Files – Compilation & packaging

  • copyEnv.sh (Copy Libraries [Java jar files] and Configuration files [.xml])
  • compile.sh (Compile and package into jar file)
  • initiate.sh (Start application while referencing necessary jar and xml file)

Classes 

Class – hbaseDemoMedia.mediaNode

  • As said in the summary the mediaNode class is a basic object that captures the various attributes we are trying to capture
  • As of this iteration, the list of attributes are mediaName, mediaType, dateAcquired, and mediaComment

Primary Key

  • Currently, HBase does not support composite primary keys.  Composite keys refers to primary keys that have more that one attribute.
  • An Example of single attribute table is an Order Table; of which the Order Number (OrderNumber) is fully representative and unique
  • On the other hand, an example of a multi-valued keyed table is OrderItem; of which the OrderNumber and ItemID are needed to fully cover uniqueness.

Primary Key (for us)

  • In our case, we use MediaName and MediaType to justify uniqueness
  • Though, not needed we used the ASCII Character (ASCII: Dagger = †) as a delimiter between the two values (Media Name and Media Type)
  • The merging of these two attributes is used to compute the MediaKey

Class – hbaseDemoMedia.hbaseDemoDB

Configuration – Data Sources (XML)

  • There are two XML files (hbase-site.xml and hbase-default.xml) that one needs to copy from the Hadoop\HBase HMaster Node
  • The hbase-default.xml contains all configurable entries
  • The hbase-site.xml is the one that is required; as it contains site-specific attributes
  • The hbase-site.xml contains very important attributes.  Those attributes are listed below:
  • Attribute – hbase.rootdir  – Example — hdfs://node.engineering.mydomain.org:8020/hbase
  • Attribute – hbase.security.authentication – Example — true
  • Attribute – hbase.zookeeper.quorum – Example – zooKeeper.engineering.mydomain.com
  • Attribute – hbase.zookeeper.property.clientPort – Example – 2181

Configuration – Get DB Connection

The following statements reads the aforementioned XML files and connects to the listed Hbase & Zookeeper manager.



   org.apache.hadoop.conf.Configuration objHadoopConf = null;

   objHadoopConf = HBaseConfiguration.create();

Configuration – Quick Points

  • Note that if the XML files are not present or not reference-able via the ClassPath one will get the error pasted below


13/04/25 20:51:28 INFO zookeeper.ClientCnxn: Opening socket connection to server fe80:0:0:0:0:0:0:1%1/fe80:0:0:0:0:0:0:1%1:2181. Will not attempt to authenticate using SASL (Unable to locate a login configuration)
13/04/25 20:51:28 WARN zookeeper.ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)

MissingXMLFile

Connects to Table

  • Once we have a connection to the Hadoop/Hbase cluster, we can instantiate the HTable object 


String TABLE_NAME = "media";

org.apache.hadoop.hbase.client.HTable objHTable = null;
// This instantiates an HTable object that connects you to the table
objHTable = new org.apache.hadoop.hbase.client.HTable(
				                          objHadoopConf
							, TABLE_NAME
						      );

Store Data



   String COLUMN_FAMILY_CF = "cf";
   String COLUMN_NAME_MEDIA_NAME = "MediaName";
   String COLUMN_NAME_MEDIA_TYPE = "MediaType";
   String COLUMN_NAME_DATE_ACQUIRED = "DateAcquired";
   String COLUMN_NAME_MEDIA_COMMENT = "MediaComment";

   byte[] bytesColumnFamily = Bytes.toBytes(COLUMN_FAMILY_CF);
   byte[] bytesColumnMediaName = Bytes.toBytes(COLUMN_NAME_MEDIA_NAME);
   byte[] bytesColumnMediaType = Bytes.toBytes(COLUMN_NAME_MEDIA_TYPE);
   byte[] bytesColumnDateAcquired = Bytes.toBytes(
                                      COLUMN_NAME_DATE_ACQUIRED);
   byte[] bytesColumnMediaComment  = Bytes.toBytes(
                                      COLUMN_NAME_MEDIA_COMMENT);

   Put objPut = null;
   objPut = new Put(Bytes.toBytes(strRowKey));

   strRowKey = objMediaNode.getMediaKey();

   objPut.add(bytesColumnFamily
               , bytesColumnMediaName
               , Bytes.toBytes(objMediaNode.getMediaName()));

   objPut.add(bytesColumnFamily
                , bytesColumnMediaType
                , Bytes.toBytes(objMediaNode.getMediaType()));

   objPut.add(bytesColumnFamily
                 , bytesColumnDateAcquired
                 , Bytes.toBytes(objMediaNode.getDateAcquired()));

   objPut.add(bytesColumnFamily
                , bytesColumnMediaComment
                , Bytes.toBytes(objMediaNode.getMediaComment()));

   objHTable.put(objPut);

Retrieve Data


String strKey = "Data being sought";
char ENDING_CHAR = (char) 254;
String ENCODING = "UTF-8";
Scan objScan = null;
ResultScanner objResultScanner = null;  
mediaNode objMediaNode = null; 
byte[] bytesKeyStart = null; 
byte[] bytesKeyEnd = null; 
byte[] byteRowKey = null; 
byte[] byteMediaName = null; 
byte[] byteMediaType = null; 
byte[] byteMediaComment = null; 

ArrayList mediaNodeList = new ArrayList(); 
int lNumberofMatches = 0;

//Clear List 
mediaNodeList.clear(); 
bytesKeyStart = Bytes.toBytes(strKey); 
bytesKeyEnd = Bytes.toBytes(strKey + ENDING_CHAR ); 
objScan = new Scan(bytesKeyStart, bytesKeyEnd); 
objResultScanner = objHTable.getScanner(objScan); 
lNumberofMatches = 0;

for ( 
       Result result = objResultScanner.next(); 
       result != null; 
       result = objResultScanner.next() 
     ) 
     { 

         lNumberofMatches = lNumberofMatches + 1; 
         objMediaNode = new mediaNode(); 

         mediaNodeList.add(objMediaNode); 
         byteRowKey = result.getRow();

        //Use the result object 
        objMediaNode.setDBMediaKey(byteRowKey, ENCODING); 

        byteMediaName = result.getValue(  bytesColumnFamily 
                                        , bytesColumnMediaName ); 

        objMediaNode.setMediaName( new String(byteMediaName)); 

        byteMediaType = result.getValue( bytesColumnFamily 
                                         , bytesColumnMediaType ); 
        objMediaNode.setMediaType( new String(byteMediaType));

        byteMediaComment = result.getValue ( bytesColumnFamily
                                            , bytesColumnMediaComment );
        objMediaNode.setMediaComment( new String(byteMediaComment));

      } //for

Remove Data


//in the example below vMediaNodeList contains the objects
//we will like to remove
ArrayList vMediaNodeList;
mediaNode objMediaNode = null;
String strKey = null;
byte[] byteRowKey = null;

List objListDelete = new ArrayList();
Delete objDel = null;

for (mediaNode objNode: vMediaNodeList)
{

     /* get key as string */
     strKey = objNode.getDBMediaKey();

     /* convert string as bytes */   
     byteRowKey = strKey.getBytes();

     /* convert byte array to Hbase Delete Object */ 
     objDel = new Delete(byteRowKey);

     //add Hbase Delete Object to Hbase Delete List */
     objListDelete.add(objDel); 

  }

  // delete elements from HBase         
  objHTable.delete(objListDelete);
  

Class – hbaseDemoMedia.hbaseDemoUI

Getting User Input

  • Courtesy of http://www.homeandlearn.co.uk/java/user_input.html, we used the relatively new java.util.Scanner class
  • The default Scanner delimiter tokenizes based on spaces among other things.  This default path obviously does not work for us when we try to bring in movie titles (as they often contain 3 to 4 words).Here is one that works as it changes the delimiter to the “new line”
    
       // http://stackoverflow.com/questions/3493230/scanner-usedelimiter 
       Scanner userInput = new Scanner( System.in );
       userInput.useDelimiter( "\n" );
    
    

Libraries

Libraries – Java – Jar Files

There are quite a few Java Jar files that are needed:

Vendor Category Jar File
Apache.Org Apache Common
commons-configuration-1.6.jar
commons-lang-2.5.jar
commons-logging-1.1.1.jarC
Cloudera Hadoop
hadoop-common-2.0.0-cdh4.2.0.jar
hadoop-core-2.0.0-mr1-cdh4.2.0.jar
hadoop-auth-2.0.0-cdh4.2.0.jar
Cloudera HBase
hbase-0.94.2-cdh4.2.0.jar
Cloudera Zookeeper
zookeeper-3.4.5-cdh4.2.0.jar
Google Protocol Buffer
protobuf-java-2.4.0a.jar
Apache Log4j
log4j-1.2.16.jar
slf4j.org slf4j
slf4j-api-1.6.1.jar
slf4j-log4j12-1.6.1.jar

As these files are available on the Hadoop\Hbase nodes, probably your best available path to gaining access to them is to copy (scp) them over to you development environment from our HBase node.

Later in the write-up, I will present a copy of a bash file that we used.

Environment

XML Configuration files

To allow us to connect to Hadoop\Hbase\ZooKeeper we need the hbase-site.xml file.

The location of the hbase-site.xml file depends on the Hadoop distribution we are using.

Cloudera – CDH4

File-name Hadoop\Hbase Folder
hbase-site.xml /etc/hbase/conf.cloudera.hbase1/

Java Jar Files

The Java jar files needs to be copied from the Hadoop\Hbase node.

Cloudera – CDH4

Category Hadoop\Hbase Folder
Apache Common /usr/lib/hbase/lib/
Apache Hadoop / Hbase / ZooKeeper / Google / log4j / slf4j /usr/share/cmf/lib/

Code

Class – hbaseDemoMedia.mediaNode



package hbaseDemoMedia;

public class mediaNode
{

	/*
          (no) Properties in Java?
	  http://stackoverflow.com/questions/70471/no-properties-in-java
	*/
	private String mediaName = null;
	private String mediaType = null;
	private String dateAcquired = null;

	private String strMediaKey = null; 
	private String strDBMediaKey = null;
	private String strMediaComment = null;

	//(char) 254;
	//Double dagger
	private static int DOUBLE_DAGGER = 135;
	private static char MEDIA_KEY_SEPARATOR = (char)DOUBLE_DAGGER;

	public String getMediaKey()
	{
		return (strMediaKey);
	}

	public void setMediaKey(String value)
	{
		strMediaKey = value;
	}

	public String getDBMediaKey()
	{
		return (strDBMediaKey);
	}

	public String getDBMediaKey(String encoding)
	{
		return (strDBMediaKey);
	}

	public void setDBMediaKey(String value)
	{
		strDBMediaKey = value;
	}

	public void setDBMediaKey(byte[] value, String encoding)
	throws java.io.UnsupportedEncodingException
	{

//http://balusc.blogspot.com/2009/05/unicode-how-to-get-characters-right.html
	    strDBMediaKey = new String(value, encoding); // Correct.

	}	

	public String getMediaName()
	{
		return (mediaName);
	}

	public String getMediaNameAsString()
	{

		if (mediaName == null)
		{
			return "";
		}
		else
		{
			return (mediaName);
		}

	}

	private void computeMediaKey()
	{
		strMediaKey = mediaName 
                                 + MEDIA_KEY_SEPARATOR 
                                 + mediaType;

	}

	public void setMediaName(String value)
	{

		mediaName = value;

		computeMediaKey();

	}

	public String getMediaType()
	{
		return (mediaType);
	}

	public String getMediaTypeAsString()
	{

		if (mediaType == null)
		{
			return "";
		}
		else
		{
			return (mediaType);
		}

	}

	public void setMediaType(String value)
	{

		mediaType = value;

		computeMediaKey();	

	}

	public String getDateAcquired()
	{
		return (dateAcquired);
	}

	public String getDateAcquiredAsString()
	{

		if (dateAcquired == null)
		{
			return "";
		}
		else
		{
			return (dateAcquired);
		}

	}

	public void setDateAcquired(String value)
	{
		dateAcquired = value;
	}

	public String getMediaComment()
	{
		return (strMediaComment);
	}

	public String getMediaCommentAsString()
	{

		if (strMediaComment == null)
		{
			return "";
		}
		else
		{
			return (strMediaComment);
		}

	}

	public void setMediaComment(String value)
	{
		strMediaComment = value;
	}

}

hbaseDemoMedia.hbaseDemoDB (DB Interface)


package hbaseDemoMedia;

import java.io.IOException;
import java.util.ArrayList;
import java.util.List;

//import org.apache.hadoop.conf.Configuration;
//import org.apache.hadoop.conf.Configuration;

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.*;

import org.apache.hadoop.hbase.client.HTable;
import org.apache.hadoop.hbase.client.Put;
import org.apache.hadoop.hbase.util.Bytes;

import org.apache.hadoop.hbase.client.Result;
import org.apache.hadoop.hbase.client.ResultScanner;
import org.apache.hadoop.hbase.client.Scan;

import org.apache.hadoop.hbase.client.Get;

import org.apache.hadoop.hbase.client.Delete;

/*
import org.apache.hadoop.hbase.HColumnDescriptor;
import org.apache.hadoop.hbase.HTableDescriptor;
import org.apache.hadoop.hbase.KeyValue;
import org.apache.hadoop.hbase.MasterNotRunningException;
import org.apache.hadoop.hbase.ZooKeeperConnectionException;
import org.apache.hadoop.hbase.client.Delete;
import org.apache.hadoop.hbase.client.Get;
import org.apache.hadoop.hbase.client.HBaseAdmin;
import org.apache.hadoop.hbase.client.HTable;
import org.apache.hadoop.hbase.client.Result;
import org.apache.hadoop.hbase.client.ResultScanner;
import org.apache.hadoop.hbase.client.Scan;
import org.apache.hadoop.hbase.client.Put;
import org.apache.hadoop.hbase.util.Bytes;
*/

public class hbaseDemoDB
{

    private static org.apache.hadoop.conf.Configuration objHadoopConf 
    					= null;
    private org.apache.hadoop.hbase.client.HTable objHTable = null;
    private ResultScanner objResultScanner = null; 

    private static String TABLE_NAME = "media";
    private static String COLUMN_FAMILY_CF = "cf";

    private static String COLUMN_NAME_MEDIA_NAME = "MediaName";
    private static String COLUMN_NAME_MEDIA_TYPE = "MediaType";
    private static String COLUMN_NAME_DATE_ACQUIRED = "DateAcquired";
    private static String COLUMN_NAME_MEDIA_COMMENT = "MediaComment";

    private static byte[] bytesColumnFamily = 
                Bytes.toBytes(COLUMN_FAMILY_CF);

    private static byte[] bytesColumnMediaName = 
                Bytes.toBytes(COLUMN_NAME_MEDIA_NAME);

    private static byte[] bytesColumnMediaType = 
                Bytes.toBytes(COLUMN_NAME_MEDIA_TYPE);

    private static byte[] bytesColumnDateAcquired = 
                Bytes.toBytes(COLUMN_NAME_DATE_ACQUIRED);

    private static byte[] bytesColumnMediaComment  = 
                Bytes.toBytes(COLUMN_NAME_MEDIA_COMMENT);

    private static char ENDING_STRING = (char) 254;

    private static String ENCODING = "UTF-8";

    private static String MUST_NOT_BE_NULL = " must not be null";
    private String strFunctionName ="";

    private mediaNode objMediaNode = null;

    /*

        hbase-site.xml
        hbase-default.xml

        You need a configuration object to tell the client 
        where to connect.
        When you create a HBaseConfiguration, it reads in 
        whatever you've set into your hbase-site.xml and in 
        hbase-default.xml, as long as these
        can be found on the CLASSPATH

    */

    String strRowKey = null;

    //Generic ArrayList to Store only mediaNode objects
    private ArrayList mediaNodeList = 
                new ArrayList(); 

    private void clearList(ArrayList objList)
    {

        for (Object item: objList)
        {
            item = null;

        }

        objList.clear();

    }

    private void initHBaseConfiguration()
    {

        objHadoopConf = HBaseConfiguration.create();

    }

    private Boolean prepareTable()
    //throws java.io.IOException
    {

        Boolean bAction = false;
        strFunctionName = "In hbaseDemoDB::prepareTable ";

        try
        {

             System.out.println(strFunctionName
                                 + TABLE_NAME + " ....");

             objHTable = new HTable(
                                          objHadoopConf
                                        , TABLE_NAME
                                   );

              System.out.println(strFunctionName
                                     + TABLE_NAME);

             bAction = true;

        }
        catch (Exception ex)
        {

            bAction = false;

            System.out.println(strFunctionName
                                + ex.getMessage());

            for (StackTraceElement ste :
                     Thread.currentThread().getStackTrace()) 
            {
               System.out.println(ste);
            }

        }

        return bAction;

    }

    public Boolean store(mediaNode objMediaNode)
    //throws java.io.IOException
    {

        Boolean bAction = false;
        strFunctionName = "In hbaseDemoDB::store";

        if (objHTable == null)
        {
            throw new NullPointerException("HTable "
                                 + MUST_NOT_BE_NULL);  
        }

        try
        {

            strRowKey = objMediaNode.getMediaKey();

            if (strRowKey == null) 
            {
                throw new NullPointerException("Row Key" 
                            + MUST_NOT_BE_NULL);  
            }

            System.out.println(strFunctionName + strRowKey);

            Put objPut = new Put(Bytes.toBytes(strRowKey));

            if (objMediaNode.getMediaName() == null) 
            {
                throw new NullPointerException("Media name "
                         + MUST_NOT_BE_NULL);  
            }

            if (objMediaNode.getMediaType() == null) 
            {
                throw new NullPointerException("Media type " 
                        + MUST_NOT_BE_NULL);  
            }

            if (objMediaNode.getDateAcquired() == null) 
            {
                throw new NullPointerException("Date Acquired "
                     + MUST_NOT_BE_NULL);  
            }

            objPut.add(bytesColumnFamily
                        , bytesColumnMediaName
                        , Bytes.toBytes(objMediaNode.getMediaName()));

            objPut.add(bytesColumnFamily
                        , bytesColumnMediaType
                        , Bytes.toBytes(objMediaNode.getMediaType()));

            objPut.add(bytesColumnFamily
                        , bytesColumnDateAcquired
                        , Bytes.toBytes(objMediaNode.getDateAcquired()));

            objPut.add(bytesColumnFamily
                         , bytesColumnMediaComment
                         , Bytes.toBytes(objMediaNode.getMediaComment()));

            objHTable.put(objPut);

            bAction = true;

        }
        catch (Exception ex)
        {

            System.out.println(strFunctionName
                                + ex.getMessage());

            for (StackTraceElement ste : 
                        Thread.currentThread().getStackTrace()) 
            {
               System.out.println(ste);
            }

            bAction = false;
        }   

        return (bAction);

    }

    public ArrayList getRecords(String strKey)
    //throws java.io.IOException
    {

        Boolean bAction = false;
        strFunctionName = "In hbaseDemoDB::getRecords";

        mediaNode objMediaNode = null;

        byte[] bytesKeyStart = null;
        byte[] bytesKeyEnd = null;
        Scan objScan = null;

        byte[] byteRowKey = null;
        byte[] byteMediaName = null;
        byte[] byteMediaType = null;
        byte[] byteMediaComment = null;

        if (objHTable == null)
        {
            throw new NullPointerException("HTable " + MUST_NOT_BE_NULL);  
        }

        try
        {

            clearList(mediaNodeList);

            bytesKeyStart = Bytes.toBytes(strKey);

            //bytesKeyEnd =  Bytes.toBytes(strKey +  new byte[] {0});
            bytesKeyEnd =  Bytes.toBytes(strKey + ENDING_STRING );

            objScan = new Scan(bytesKeyStart, bytesKeyEnd);

            //objScan = new Scan(bytesKeyStart);

            objResultScanner = objHTable.getScanner(objScan);

            int lNumberofMatches = 0;

            for (
                    Result result = objResultScanner.next(); 
                    result != null; 
                    result = objResultScanner.next()
                )
               {
                    lNumberofMatches = lNumberofMatches + 1;

                    objMediaNode = new mediaNode();

                    mediaNodeList.add(objMediaNode);

                    byteRowKey = result.getRow();

                    //Use the result object
                    objMediaNode.setDBMediaKey(byteRowKey, ENCODING);

                    byteMediaName = result.getValue( 
                                            bytesColumnFamily
                                          , bytesColumnMediaName
                                         );

                    if  (byteMediaName != null)
                    {

                        objMediaNode.setMediaName(
                        	new String(byteMediaName));

                    }

                    byteMediaType = result.getValue( 
                                          bytesColumnFamily
                                        , bytesColumnMediaType
                                                   );

                    if  (byteMediaType != null)
                    {

                        objMediaNode.setMediaType(
                        	new String(byteMediaType));

                    }

                    byteMediaComment = result.getValue
                                             ( 
                                                  bytesColumnFamily
                                                , bytesColumnMediaComment
                                              );

                    if  (byteMediaComment != null)
                    {

                        objMediaNode.setMediaComment(
                        	new String(byteMediaComment));

                    }

               } //for

               System.out.println("# of Matches " + lNumberofMatches);

            bAction = true;

        }
        catch (Exception ex)
        {

            System.out.println("Exception in " 
                                    + strFunctionName
                                    + ex.getMessage());

            for (StackTraceElement ste 
                        : Thread.currentThread().getStackTrace()) 
            {
               System.out.println(ste);
            }

            bAction = false;
        }   

        return (mediaNodeList);

    }

    public boolean deleteRecords(ArrayList vMediaNodeList)
    //throws java.io.IOException
    {

        Boolean bAction = false;
        strFunctionName = "In hbaseDemoDB::deleteRecords";

        mediaNode objMediaNode = null;
        String strKey = null;
        byte[] byteRowKey = null;

        List objListDelete = new ArrayList();
        Delete objDel = null;

        if (objHTable == null)
        {
            throw new NullPointerException("HTable "
            				 + MUST_NOT_BE_NULL);  
        }

        try
        {

            if (vMediaNodeList != null)
            {

                for (mediaNode objNode: vMediaNodeList)
                {

                    /* get key as string */
                    strKey = objNode.getDBMediaKey();

                    /* convert string as bytes */   
                    byteRowKey = strKey.getBytes();

                    /* convert byte array to Hbase Delete Object */ 
                    objDel = new Delete(byteRowKey);

                    //add Hbase Delete Object to Hbase Delete List */
                    objListDelete.add(objDel); 

                }

                objHTable.delete(objListDelete);

            }    

            bAction = true;

        }
        catch (Exception ex)
        {

            System.out.println("Exception in "
                                 + strFunctionName
                                 + ex.getMessage());

            for (StackTraceElement ste : 
                    Thread.currentThread().getStackTrace()) 
            {
               System.out.println(ste);
            }

            bAction = false;
        }   

        return (bAction);

    }

    public boolean init()
    {

        Boolean bAction = false;
        strFunctionName = "In hbaseDemoDB::init";

        try
        {
            initHBaseConfiguration();
            prepareTable();
        }
        catch (Exception ex)
        {

            System.out.println("Exception in " 
                                 + strFunctionName
                                 + ex.getMessage());

        }

        return bAction; 
    }

}

hbaseDemoUI



package hbaseDemoMedia;

import java.util.Scanner;

import java.io.IOException;
import java.util.ArrayList;
import java.util.List;

import java.io.PrintStream;

public class hbaseDemoUI
{

    /*
        Accepting Input from a User
        http://www.homeandlearn.co.uk/java/user_input.html
    */
    Scanner userInput = new Scanner( System.in );

    String strChoice = null;

    String strMediaName = null;
    String strMediaType = null;
    String strDateAcquired = null;
    String strMediaComment = null;

    mediaNode objMediaNode =  new mediaNode();

    ArrayList mediaNodeList = null;

    /*
            hbaseDemoUI.java:27:  expected
            objMediaNode = new mediaNode();
    */
    //mediaNode objMediaNode;
    //objMediaNode = new mediaNode();

    static hbaseDemoUI objHBaseDemoUI = null;

    static hbaseDemoDB objHBaseDemoDB = null;

    public hbaseDemoUI()
    {

        initVars();

    }

    private void initVars()
    {

        //System.out.println("InitVars..");

        //http://stackoverflow.com/questions/3493230/scanner-usedelimiter
        //userInput.useDelimiter( "-|\n" );

        userInput.useDelimiter( "\n" );

    }

    private void mainMenuDisplay()
    {

        System.out.println("******************************");
        System.out.println("*                            *");
        System.out.println("*    Hbase Demo              *");
        System.out.println("*                            *");
        System.out.println("*  S) Store record           *");
        System.out.println("*  D) Display record(s)      *"); 
        System.out.println("*  R) Remove record(s)       *"); 
        System.out.println("*  Q) Quit                   *");                         
        System.out.println("*                            *");
        System.out.println("******************************");

    }

    private void mainMenuPrompt()
    {

        Boolean bMore = true;

        do
        {

            System.out.print("Choice ? ");

            strChoice = userInput.next();

            strChoice = strChoice.toUpperCase();

            //System.out.print("Choice is " + strChoice);

           if (strChoice.compareTo("S") == 0)
            {
                do
                {
                    dataAcceptance();
                    dataStore();
                    bMore = dataAcceptanceMore();
                }
                while (bMore);
            }
            else if (strChoice.compareTo("D") == 0)
            {
                do
                {
                    getDataToQuery();
                    dataQuery();
                    bMore = dataQueryMore();
                }
                while (bMore);
            }
            else if (strChoice.compareTo("R") == 0)
            {
                do
                {

                    getDataToRemove();

                    Boolean bRemoveData;

                    bRemoveData = (strMediaName.compareTo("") != 0);

                    if (bRemoveData)
                    {
                        dataRemove();
                    }
                    else
                    {
                        break;
                    }

                    //bMore = dataQueryMore();
                }
                while (bMore );
            }           
            else
            {
                break;
            }

            bMore = true;

            mainMenuDisplay();

        }

        while (bMore);
    }

    private void dataAcceptance()
    {

        System.out.print("Media Name:       ");
        strMediaName = userInput.next();
        objMediaNode.setMediaName(strMediaName);

        System.out.print("Media Type:       ");
        strMediaType = userInput.next();
        objMediaNode.setMediaType(strMediaType);

        System.out.print("Date Acquired:    ");
        strDateAcquired = userInput.next();
        objMediaNode.setDateAcquired(strDateAcquired);

        System.out.print("Media Comment:    ");
        strMediaComment = userInput.next();
        objMediaNode.setMediaComment(strMediaComment);

    }   

    private Boolean dataAcceptanceMore()
    {

        String strPromptMore =  null;
        Boolean bMore = false;

        System.out.print("Add More (Y/N) ");
        strPromptMore = userInput.next();

        bMore = (strPromptMore.compareTo("Y") == 0);

        return (bMore);

    }   

    private void dataStore()
    {

        objHBaseDemoDB.store(objMediaNode);

    }   

    private void getDataToQuery()
    {

        System.out.print("Media Name: ");
        strMediaName = userInput.next();
        objMediaNode.setMediaName(strMediaName);

    }   

    private void dataQuery()
    {

        System.out.println("Searching for :" + strMediaName);

        mediaNodeList = objHBaseDemoDB.getRecords(strMediaName);

        if (mediaNodeList != null)
        {

            for (mediaNode objNode: mediaNodeList)
            {

                try
                {
                    java.io.PrintStream out = new java.io.PrintStream
                                                 (
                                                      System.out
                                                    , true
                                                    , "UTF-8"
                                                  );

                    out.println("Key:" + objNode.getDBMediaKey());
                }
                catch (java.io.UnsupportedEncodingException ex)
                {

                    System.out.println("UnsupportedEncodingException:" 
                        + objNode.getDBMediaKey());
                }

                //System.out.println("Key:" + objNode.getDBMediaKey());

                System.out.println("\tName:" 
                                    + objNode.getMediaName());
                System.out.println("\tType:" 
                                    + objNode.getMediaTypeAsString());
                System.out.println("\tDate Acquired:" 
                                    + objNode.getDateAcquiredAsString());
                System.out.println("\tMedia Comment:" 
                                    + objNode.getMediaCommentAsString());                       

            }

        }

    }   

    private Boolean dataQueryMore()
    {

        String strPromptMore =  null;
        Boolean bMore = false;

        System.out.print("Query for More (Y/N) ");
        strPromptMore = userInput.next();

        bMore = (strPromptMore.compareTo("Y") == 0);

        return (bMore);

    }   

    private void getDataToRemove()
    {

        strMediaName = "";

        {
            System.out.print("Media Name (to removed): ");
            strMediaName = userInput.next();

        }

    }   

    private void dataRemove()
    {

        Boolean bConfirm;
        int     iNumberofMatches = 0;

        System.out.println("Searching for :" + strMediaName);

        mediaNodeList = objHBaseDemoDB.getRecords(strMediaName);

        if (mediaNodeList != null)
        {

            for (mediaNode objNode: mediaNodeList)
            {

                System.out.println("\tKey:" 
                        + objNode.getDBMediaKey());

            }

        }

        iNumberofMatches = mediaNodeList.size();

        if (iNumberofMatches > 0)
        {

            bConfirm = dataRemoveConfirm();

            if (bConfirm)
            {

                objHBaseDemoDB.deleteRecords(mediaNodeList);

            }

        }

    }   

    private Boolean dataRemoveConfirm()
    {

        String strPromptMore =  null;
        Boolean bMore = false;

        System.out.print("Confirm Removal (Y/N) ");
        strPromptMore = userInput.next();

        bMore = (strPromptMore.compareTo("Y") == 0);

        return (bMore);

    }   

    private Boolean dataRemoveMore()
    {

        String strPromptMore =  null;
        Boolean bMore = false;

        System.out.print("Query for More (Y/N) ");
        strPromptMore = userInput.next();

        bMore = (strPromptMore.compareTo("Y") == 0);

        return (bMore);

    }   

    public static void main(String[] agrs) 
    {

        objHBaseDemoDB = new hbaseDemoDB();

        objHBaseDemoDB.init();

        objHBaseDemoUI = new hbaseDemoUI();

            objHBaseDemoUI.mainMenuDisplay();

            objHBaseDemoUI.mainMenuPrompt();

        objHBaseDemoUI = null;

    }

}

Copy XML Configuration & Jar files (copyEnv.sh)



# Copy hbase-site.xml
scp  dadeniji@hbaseNode:/etc/hbase/conf.cloudera.hbase1/hbase-site.xml .

# Copy hbase-default.xml
scp  dadeniji@hbaseNode:/etc/hbase/conf.cloudera.hbase1/hbase-default.xml .

# Apache - Common *jar
scp  dadeniji@hbaseNode:/usr/lib/hbase/lib/commons-configuration-1.6.jar .
scp  dadeniji@hbaseNode:/usr/lib/hbase/lib/commons-lang-2.5.jar .
scp  dadeniji@hbaseNode:/usr/lib/hbase/lib/commons-logging-1.1.1.jar .

# hadoop
scp  dadeniji@hbaseNode:/usr/share/cmf/lib/cdh4/hadoop-common-2.0.0-cdh4.2.0.jar  .
scp  dadeniji@hbaseNode:/usr/share/cmf/lib/cdh4/hadoop-core-2.0.0-mr1-cdh4.2.0.jar  .
scp  dadeniji@hbaseNode:/usr/share/cmf/lib/cdh4/hadoop-auth-2.0.0-cdh4.2.0.jar .

# hbase
scp  dadeniji@hbaseNode:/usr/share/cmf/lib/cdh4/hbase-0.94.2-cdh4.2.0.jar .

# zookeeper
scp  dadeniji@hbaseNode:/usr/share/cmf/lib/cdh4/zookeeper-3.4.5-cdh4.2.0.jar  .

#Google
scp  dadeniji@hbaseNode:/usr/share/cmf/lib/cdh4/protobuf-java-2.4.0a.jar .

#log4j
scp  dadeniji@hbaseNode:/usr/share/cmf/lib/log4j-1.2.16.jar .

#slf4j
scp  dadeniji@hbaseNode:/usr/share/cmf/lib/slf4j-api-1.6.1.jar  .
scp  dadeniji@hbaseNode:/usr/share/cmf/lib/slf4j-log4j12-1.6.1.jar  .

Compile



# Compile
# mkdir javaClasses
# http://stackoverflow.com/questions/8790515/how-to-check-directory-exist-or-not-in-linux
if [ ! -d "javaClasses" ]; then
   mkdir javaClasses
fi
# Forums Archive Linux/BSD How to rm files recursively?
# http://forums.whirlpool.net.au/archive/350903
#rm javaClasses/*.class
find . -name '*.class*' -print0 | xargs -0 rm
rm hbaseDemoMedia.jar

# http://kevinboone.net/classpath.html
javac -d javaClasses mediaNode.java 

# compile hbaseDemoDB.java (reference java jar files)
javac -cp  hadoop-common-2.0.0-cdh4.2.0.jar:hadoop-core-2.0.0-mr1-cdh4.2.0.jar:hbase-0.94.2-cdh4.2.0.jar:zookeeper-3.4.5-cdh4.2.0.jar:javaClasses -d javaClasses hbaseDemoDB.java 

# compile hbaseDemoUI.java 
javac -cp  hadoop-common-2.0.0-cdh4.2.0.jar:hadoop-core-2.0.0-mr1-cdh4.2.0.jar:hbase-0.94.2-cdh4.2.0.jar:zookeeper-3.4.5-cdh4.2.0.jar:javaClasses -d javaClasses hbaseDemoUI.java

# package java class compiled files into hbaseDemoMedia.jar 
jar -cfv hbaseDemoMedia.jar -C javaClasses .

Invoke

# Initiate



# Setting multiple jars in java classpath
# http://stackoverflow.com/questions/219585/setting-multiple-jars-in-java-classpath

java -cp hadoop-common-2.0.0-cdh4.2.0.jar:hadoop-core-2.0.0-mr1-cdh4.2.0.jar:hbase-0.94.2-cdh4.2.0.jar:zookeeper-3.4.5-cdh4.2.0.jar:hbaseDemoMedia.jar:commons-configuration-1.6.jar:commons-lang-2.5.jar:commons-logging-1.1.1.jar:guava-14.0.1.jar:hadoop-auth-2.0.0-cdh4.2.0.jar:slf4j-api-1.6.1.jar:slf4j-log4j12-1.6.1.jar:log4j-1.2.16.jar:protobuf-java-2.4.0a.jar:. hbaseDemoMedia.hbaseDemoUI

Repository

The code has being packed and placed into GitHub

Repository – GitHub

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s