Javatech » Uncategorized

Hadoop Explained

deevis — Sun, 21 Jun 2009 07:10:21 +0000

Cloudera Hadoop Training: Programming with Hadoop from Cloudera on Vimeo.

Amazon Cloud

deevis — Fri, 19 Jun 2009 05:41:03 +0000

I’ve been playing around with a Fedora system out in EC2 of late. It’s pretty darn cool and I’ve got my system almost ready to start playing around seriously. What this means is that I’ve got enough stuff installed on it to stop playing the install game.
Being the type to optimize everything, I’m trying to get my system configured in the least amount of uptime. So, I’ll have it up for an hour or two when I have a chance to work on it, and then if I make any good progress I’ll snap a new AMI and startup from this new AMI when I pick up the next time.
It’s my first time using Fedora and I’m getting used to things being different from my comfort zone – Ubuntu.
The first big difference is that ‘apt-get’ doesn’t exist in Fedora. But it does have an analog called ‘yum’.
To install mysql on Fedora I simply ran:
yum install mysql-server mysql

To get apache web server with php/mysql support installed I just ran:
yum install httpd php-mysql
This is every bit as easy as apt-get. Making some really good progess. So, the first time around I terminate the instance and figure I’ll pick up where I’ve left off.
The next time when I come back to the AWS console – my instance isn’t there for me to restart. Now, here’s the thing with the EC2 server instances :

WHEN YOU TERMINATE AN EC2 INSTANCE, IT’S LIKE YOU STUFFED THE CASE WITH TNT AND THREW IT INTO A CAMPFIRE.

That’s right, terminate doesn’t mean power-down. It means obliterate. Anything on the drives is G-O-N-E. It is no more – it’s an ex-instance.

That was rather eye-opening. So, back to the drawing board and this time, to be safe, I made sure to pick an AMI with EC2 utilities installed on it already. I installed my software on the instance and this time even registered the services to startup automatically.
service mysql start
service httpd start
I restarted the instance and tested that it came back up with apache and mysql running – it did. Good enough – it was time to save this instance as an AMI into one of my S3 buckets. After a bit of finagling I found the correct couple of incantations.
ec2-bundle-vol –cert publicCert –privatekey privatePem –user awsAccountNumber
The above command will create an imageManifestXml which you’ll use in the following statement:
ec2-upload-bundle –bucket s3BucketName –accessKey awsAccessKey –secretKey awsSecretKey –manifest imageManifestXml

This took maybe ten minutes to run both of the above, but afterwards I had new starting point AMI to use – in theory at least…

I double checked everything by starting another instance ( before shutting down the good one ) with the new AMI and making sure that it passed snuff – which it did. I shut down both instances and retired for the night – all of this to save 10 cents an hour. But more importantly it is to understand the process. No, it’s the money
This virtualization stuff is really powerful – I’ll write some more on mounting Volumes and using existing Snapshots of open-sourced datasets soon.

Prototype, JQuery and JQueryUI

deevis — Wed, 18 Feb 2009 07:51:06 +0000

We’ve switched over to using JQuery from Prototype at my current employer. We’d previously used Prototype-1.5 for a long time and had a lot of dependencies to it. When we first started using JQuery it was version 1.1.3.1. There was no JQueryUI at that time.

So, we currently use JQuery-1.2.6 and JQueryUI-1.6rc2 and I really like them a lot. Oh yeah, we still have Prototype in the mix and it’s version 1.6 ( which is lots better than 1.5 ).

I recently, however, had issues with attempting to port new code back to a previous install. The new code was using JQueryUI, and the old code was on Prototype 1.5 and JQuery 1.1.3.1. Well – it took me a while to figure this out ( and I mentioned it already above ), but there is NO JQueryUI version compatible with JQuery-1.1.3.1. It wasn’t created until the 1.2.x branches of JQuery.

So – here’s the compatibility matrix for Prototype, JQuery and JQueryUI for any of you who care:

Prototype	JQuery	JQueryUI
1.5	1.1.3.1	—-
1.6	1.2.6	1.5.3
1.6	1.3+	1.6rc6
1.6.0.2	1.3.2	1.7

Ciao.

ArrayList vs Vector

deevis — Tue, 18 Mar 2008 14:15:06 +0000

I’ve done some more work on the ArrayList vs Vector performance issue. The post below originally stated that Vector would/could outperform ArrayList even though it is synchronized while ArrayList is not. Well, in digging deeper into the issue, I”ve determined that there is a problem with the benchmarking framework being used. I ran a very simple test with ArrayList and Vector to look at performance of add() and contains() methods and have determined that:

ArrayList.add() is 13% faster than Vector.add()

ArrayList.contains() is 3% faster than Vector.contains()

Now I need to identify the problem with the benchmark framework.

I’ve been surprised of late in my Unique Collection Building benchmarks that TreeSet was outperformed by ArrayList. One reason this could happen would be due to the nature of the dataset being processed and the data set I had contained many items with the first 20 characters or so being the same. You know, lots of entries starting with “/WEB-INF/javascript” perhaps. This would have made the TreeSets comparisons in deciding ordering more costly.

So, now I run the test again but with Vector thrown into the mix and the Strings are all 40 characters long and very, very random. The odds of differing after 2 or 3 characters are now very high.

TreeSet should do much better.

But how will Vector fair?

You probably know as well as I do that ArrayList is the faster, unsynchronized option to Vector ( much like using StringBuilder instead of StringBuffer ). But how much faster is ArrayList? Here are the results:

More on Autoboxing

deevis — Mon, 17 Mar 2008 05:31:31 +0000

I recently did a benchmark on Autoboxing where I determined that Autoboxing cost about 15 nanoseconds. Well, then I got to thinking that I’d written that test to work with int/Integer datatypes. But what about the other types? Today I add Boolean, Float, Double and Long into the mix.

To cut to the chase, I’ll first tell you the cost of Autoboxing these types.

Boolean: 1/1 nanoseconds

Integer: 15/17 nanoseconds

Float: 19/19 nanoseconds

Double: 22/24 nanoseconds

Long: 30/40 nanoseconds

The two numbers are the results for Half and Full Boxing. Half Boxing is going one way from either primitive to Object or Object to primitive. Full Boxing is a full round trip going primitive to Object and back to primitive. So, for the Long results it takes 30 nanoseconds to Half Box (one way) while it takes 40 nanoseconds to Full Box ( round trip ).

So, the underlying datatype’s size seems to directly impact the cost of the Autoboxing. I’ll be doing more tests with other types shortly. Here are the charts from the Boolean and Long trials.

Boolean Autoboxing Results

Integer Autoboxing Results

Float Autoboxing Results

Double Autoboxing Results

Long Autoboxing Results

The cost of Autoboxing

deevis — Sat, 15 Mar 2008 06:36:52 +0000

Simple question: How expensive is autoboxing of int/Integer types?
Simple answer: 15 nanoseconds per boxing.

No autoboxing ( ints only )
[source:java]
public void runInternalTrial() throws Exception {
int i = getRandomWithoutAutobox();
}

public int getRandomWithoutAutobox() {
return rnd.nextInt(1000);
}
[/source]
Autoboxing
[source:java]
public void runInternalTrial() throws Exception {
int i = getRandom();
}

// Will Autobox an int to an Integer…
public Integer getRandom() {
return rnd.nextInt(1000);
}
[/source]
Note that the Autoboxing example has to Autobox the value returned by getRandom to an Integer and then the caller has to then unbox this back to an int primitive. The two examples for HalfBoxing are simply Autoboxing and then not unboxing back to an int. Here are the results:

JVM Comparison and Java Optimization Myths

deevis — Sat, 15 Mar 2008 04:18:19 +0000

Here’s a fairly comprehensive overview of 6 performance related myths as they relate to 7 different JVMs. I now feel safer running with try/catch blocks and with synchronization. We’ll be running our own benchmarks on these and the results will be up pretty soon. Here’s the link to the paper.

JBoss Seam Presentation

deevis — Thu, 15 Nov 2007 23:08:41 +0000

Seam is a must learn technology – check it out.

clipped from www.theserverside.com

In this presentation, recorded at the recent Grails Exchange event in London, organized by Skills Matter, Christian Bauer discusses how JBoss Seam simplifies the handling of stateful conversations, multi-window operations and concurrent, fine-grained Ajax requests. He also shows how Seam unifies and integrates popular open source technologies like Facelets, Hibernate, jBPM, Drools, Groovy, iText and Lucene.

Using Patterns to Suppliment your Spring Configuration

corbix — Thu, 15 Nov 2007 17:24:58 +0000

Your Spring configuration probably consists of one or more well know XML files which are loaded by name. Have you ever wanted to override something without having to touch your base configuration? Here’s what we do. After we load our base configuration we use a pattern to load any XML files which exist in a specific “configuration-override” directory. In most cases the directory is empty and just the base configuration is loaded. When we want to override a particular bean we just place a file in the override directory.Here’s a clip from the Spring API docs:

clipped from static.springframework.org

4.7.2.1. Ant-style Patterns

When the path location contains an Ant-style pattern, for example:

     /WEB-INF/*-context.xml
     com/mycompany/**/applicationContext.xml
     file:C:/some/path/*-context.xml
     classpath:com/mycompany/**/applicationContext.xml

Integrating Google Earth with a Java webapp

deevis — Sun, 11 Nov 2007 07:51:17 +0000

I did this at work the other day and thought I’d give the broad sweeping overview here should else want to do it. It’s not too bad after all’s said and done ( and sifted through… ). Here’s the quick and easy:

1) Add these mimetypes in web.xml:
application/vnd.google-earth.kml+xml kml application/vnd.google-earth.kmz kmz
2) You’ll want to write back KML ( a flavor of XML ) with a content-type of “application/vnd.google-earth.kml+xml”:

response.setContentType(”application/vnd.google-earth.kml+xml”);

3) Make sure you give the stream a default filename that ends in .kml or else GoogleEarth won’t actually process the file.

response.setHeader(”Content-Disposition”, “attachment; filename=” + filename + “.kml”);

That’s really a 40,000 foot view of the process. Here’s Google’s KML Tutorial so you can learn more.