Technical: Hadoop – Hive – Review Settings (Using the set command)

Technical: Hadoop – Hive – Review Settings (Using the set command)

Background

Fresh off another install Hadoop \ Hive, I googled on some items and found Apache’s own Hive documentation.
As per the install, I have found Cloudera’s install to be a bit clearer; as I can just install via RPMs.

For Apache’s install, I need to do my own build.

BTW, Apache has a fairly comprehensive doc @ https://cwiki.apache.org/Hive/gettingstarted.html

As Life is a contact sport and IT by nature is same, I do not mind reading through documentations and understanding some of the design thoughts.

One of the areas mentioned in Apache’s documentation is the role of the mapred.job.tracker.

Unlike our traditionally databases, Hive SQL Queries are actually translated to mapreduce code and then processed.

Thus mapreduce configurations are important.

Review Hive Settings

So how do we review Hive Settings.  By issuing set; of course.

Review Hive Settings – All Configuration Items


set;

 

Review Hive Settings – Hive Settings that are different from Hadoop settings


set -v;

 

Review Hive Settings – Specific Items

To review values for specific items, issue set followed by the item name.

Syntax:
   hive -e 'set <item>'

Sample:
    hive -e 'set hive.metastore.uris'

Output:

Hadoop -- Hive -- set

Review Hive Settings – Wild Card Items

To review values for specific items, we end up getting all items and then greping; use the cut command to restrict line length to 80 characters.

Syntax:
   hive -e 'set -v;' | cut 1-80 | grep <item>

Sample:
    hive -e 'set -v;' | cut -c 1-80 | grep mapred

Explanation:

  • In the example above, we are interested only in mapred items.

Output:

Hadoop -- Hive -- set -- wildcard (mapred)

References

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s