Home > Java, JavaOne > JavaOne day five

JavaOne day five

The keynote

The keynote started with a dj playing some house music. Quite good house music I’d say since San Francisco has a great house scene (although the dj was from L.A.)
The session started with James Gosling presenting several people who have implemented projects in Java. It started by showing a vide about John Gage and giving him a gold duke and a painting of the new Duke design by James Gosling.

Then a demo of visualvm followed. visualvm is a profiler (it reminds me a lot of the NetBeans profiler). The visualvm is not for testing, it is meant to be used in production to evaluate the application running.

Tor Norbye went on the stage and talked about JavaScript. He then built a simple diary by using JavaScript and the yahoo libraries. To me it seemed like he tried to show the JavaScript editor in NetBeans, which was quite good to be honest.

Ken Russel and Sven Gothel talked about java on nVidia APX 2500 on a cell phone. The also showed a 3D demo model of a city on a mobile phone. If we take in mind the mobile device’s capabilities this demo was quite impressive I’d say.

Chris Melissinos and Joshua Slack talked about jMonkeyEngine and project darkstar, and also showed us a demo (unfortunatelly I didn’t take any video of the demo).

Next Laurent Logasanto went on stage and talked about extreme Java Card innovation and also presented a Java Card demo, a robocode game.

Joe Polastre talked about pervasive Java where they have some of the worlds smallest computers and they run Sentilla which fits in 10Kb or ram and 48 of ROM memory.

Jim Marggraff did an excellent demo of livescribe which is a new product of a mobile platform. Basically livescribe is a pen and a specially constructed notepad that contains micro cells and can record everything you write on this notepad as well as voice. It also provides musical capabilities and translation. And all this is written in Java. The only drawback currently is that if you run out of notebooks you will need to order them from pulse (the company behind livescribe) but they don’t ship outside USA. The good news is that in a few months you will be able to print your own notebooks on certified laserjet printers

Dr. Greg Bollella spoke about industrial strength java and showed the Blue Wonder project

Paul Perrone from Perrone Robotics presented an automatic car demo. If you are born in the ’70s it’s more than likely that you remember a TV series called “The Knight Rider” where Kit, a super car was following orders and was doing things.

This demo was doing exactly the same (although not so sophisticatedly). A car was full of sensors and a navigation system. All car’s intelligence, the AI and the decision making, is done on a microprocessor that runs Solaris and Java. Paul was giving verbal instructions to the car and this was moving! He was commanding it to go forth and it was going forth, he was commanding it to turn left and it was turning left! Impressive.

Dr Phil Christensen talked and showed a demo about Java rocks on Mars. He presented the demo JMars (which targets, distributes and analyses data from Mars) and it’s open source.

He talked about the “opportunity rover” which is the rover that was sent to Mars by NASA to gather information. In about two weeks from now the next NASA lander is using JMars to define a safe place to go. In two weeks they will find out if they did the right job.

The a video was shown about “the grid” of CERN and Derek Mathieson talked about accelerating java and how CERN is building huge projects and have hundreds of Java engineers working on them and also about the TIM (Technical Infrastructure Monitoring) project.

Then he presented a few more demos, namely

  • 3D model of Atlas with GraXML
  • EGEE
  • GridPP

Using Java technology at the world’s largest web site by Joshua Blatt and Dean Yu.

yahoo! has more than 500 million registered users and all the traffic generated every day is more than 20TB of data. To cope with this volume of data it has hundreds of thousands of servers and dozens of data centres around the world.

Although the core technology of yahoo! is C and C++ Java comes into play through yahoo! acquisitions. They also use apache and LAMP applications but over the years yahoo! has made several Java acquisitions

All these are run by using Tomcat and JBoss application server. They also use JNI and IPC bridge for C/C++ and Java communication. Although JNI was a few times slower than direct Java access they found that this was mainly due to character conversion and they could work on that.

In order to have proper web security they use

  • JSCV (Apache common daemon)
  • Cross-site scripting (XSS) for input validation
  • HTML input validation (all entities are encoded, or stripped)

For scalability and reliability they have

  • load balancers
  • DNS round robin algorithm
  • HTTP redirectors

They also use maven to do the builds and get each version and dependencies of the libraries they are using.

In order to deploy all the Java applications they face several challenges

  • all the software and hardware configurations have to be consistently reproducible
  • all the changed have to be auditable
  • J2Ee containers need to be configured differently for different products
  • manual deployment and configuration is impossible for thousands of machines

The package management systems helped

  • Dependency resolution and conflict detection
  • Detect modifications made after installation
  • Repair manual, uncontrolled modifications
  • Additional Y! requirements
  • Clearly defined package lifecycle
  • Consistent start/stop/restart directives
  • Settings as part of package state
  • Automatic journaling of system’s package state
  • easy to change system to match any point in journal
  • state can be cloned to other machines
  • Centralized deployment console

In summary

  • Bulk of JNI API performance overhead is charset conversion
  • Synchronizing JNI API calls insufficient when dealing with many native libraries
  • Use process privilege levels to protect sensitive files
  • Employ opt-out input validation strategies
  • Multi-process Tomcat improves performance and availability
  • Transitive dependency conflicts need to be treated like source merge conflicts
  • Finer grained package of Java EE platform containers enable easier reconfiguration

Advanced Java NIO technology based application using the Grizzly framework by Jean-Francois Arcand and Oleksiy Stashok

This talk was about how easy it can be to write scalable client- server applications with the Grizzly Framework.

Grizzly is an open source framework that

  • Uses Java NIO primitives and hides the complexity programming with Java NIO.
  • Easy-to-use high performance APIs for TCP, UDP and SSL communications.
  • Brings non-blocking sockets to the protocol processing layer.
  • Utilizes high performance buffers and buffer management.
  • Choice of several different high performance thread pools.
  • Ship with an HTTP module which is really easy to extend.
  • Use the Framework, but STILL have a way to control the IO layer.

Grizzly use SelectorHandlers for each protocol implementation. Each handler is added to a controller (a controller represents the main point of the grizzly framework. A controller has several handlers and all these interfaces have ready to use implementations.). There are TCPSelectorHandler, UDPSelectorHandler, TLSSelectorHandler and of course you can write your own selector handlers since everything in Grizzly is customizable.

A controller by default is composed of

  • SelectorHandler
  • ConnectorHandler
  • SelectionKeyHandler
  • ProtocolChainInstanceHandler
  • ProtocolChain
  • Pipeline

(all these interfaces have ready to use implementations by default)

Grizzly also provides other classes you need

A ProtocolFilter encapsulates a unit of processing work to be performed, whose purpose is to examine and/or modify the state of a transaction that is represented by a Context. Most Grizzly based application only have to write an implementation of one ProtocolFilter and you can assembled many ProtocolFilters into a ProtocolChain.

A ProtocolParser is a specialized ProtocolFilter that knows how to parse bytes into protocol unit of data. Grizzly implements it by parsing state into steps:

  • start processing a buffer
  • enumerate the message in the buffer
  • and end processing the buffer.

The buffer can contain 0 or more complete messages.

A ProtocolChain implements the “Chain of Responsibility” pattern, The ProtocolChain API models a computation as a series of “protocol filters”, that can be combined into a “protocol chain”.

A ProtocolChainInstanceHandler is where one or several ProtocolChains are created and cached and it decides if a stateless (one ProtocolChain instance shared amongst Threads) or stateful (one ProtocolChain instance per Thread) ProtocolChain needs to be created:

PipeLine is an interface is used as a wrapper around any kind of thread pool. Can have one Pipeline per SelectorHandler or a shared one amongst them.

On the client Grizzly provides the following components

A ConnectorHandler which is a connection unit that implements basic I/O functionality (connect, read, write, close). It provides possibility of working in both synchronous and asynchronous modes (e.g. blocking or non blocking read/write) and you can control the asynchronous either by a CallbackHandler with user custom code, or automatically by framework, using asynchronous read and write queues.

 
Controller ctrl = new Controller(); 
ctrl.addSelectorHandler(new TCPSelectorHandler(true)); 
startGrizzly(ctrl); 
ConnectorHandler connection = ctrl.acquireConnectorHandler(Protocol.TCP); 
connection.connect(new InetSocketAddress(host, port)); 
connection.write(outputByteBuffer, true); 
connection.read(inputByteBuffer, true); 
connection.close(); 
ctrl.releaseConnectorHandler(connection); 

A CallbackHandler handles client side asynchronous I/O operations (connect, read, write). When NIO channel is ready to perform the operation the corresponding CallbackHandler method will be called. All asynchronous events could be either processed inside CallbackHandler or propagated to a ProtocolChain. When processing asynchronous event inside CallbackHandler, be careful with SelectionKey interests registration.

 
Controller ctrl = new Controller(); 
ctrl.addSelectorHandler(new TCPSelectorHandler(true)); 
startGrizzly(ctrl); 
ConnectorHandler connection = ctrl.acquireConnectorHandler(Protocol.TCP); 
connection.connect(new InetSocketAddress(host, port), new CustomCallbackHandler(connection)); 
connection.write(giantByteBuffer, false); 

public class CustomCallbackHandler implements CallbackHandler<Context> {
    private ConnectorHandler connection;
    CustomCallbackHandler(ConnectorHandler connection)
    {
        this.connection = connection;
    }    

     public void onWrite(IOEvent<Context> ioEvent) {
        connection.write(giantByteBuffer, false);
        if (!giantByteBuffer.hasRemaining()) {
           notifyAsyncWriteCompleted();
        }
    }
 }
 

ThreadAttachment

  • Between OP_READ and OP_WRITE, some protocol needs to keep some states (remaining bytes inside a byte buffer, attributes, etc.).
  • In Grizzly, the default is to associate one Byte Buffer per Thread. That means a byte buffer cannot be cached as its Thread can always be re-used for another transaction.
  • To persist the byte buffer among transactions, a Thread Attachment can be used
  • What happens is internally, all attributes associated with the WorkerThread are ‘detached’, and new instance recreated (Warning: A new ByteBuffer will be created!!)
  • The ThreadAttachment can be attached to a SelectionKey, and next time an OP_READ/OP_WRITE happens, you are guarantee the value will be re-used.

HTTP modules

  • The Grizzly Framework also have an HTTP framework that can be used to build Web Server
  • This is what GlassFishTM v1|2|3 build on top of.
  • More specialized modules are also available like Comet (Async HTTP).
  • Simple interface to allow customization of the HTTP Protocol
  • GrizzlyRequest: A utility class to manipulate the HTTP protocol request.
  • GrizzlyResponse: A utility class to manipulate the HTTP protocol response.
  • GrizzlyAdapter: A utility class to manipulate the HTTP request/response object.

NIO.2: Asynchronous I/O (JSR-203) JDK version 7.0

  • An API for asynchronous (as opposed to polled, non-blocking) I/O operations on both sockets and files.
  • The completion of the socket-channel functionality defined in JSR-51, including the addition of support for binding, option configuration, and multicast datagrams.
  • Starting in Grizzly 1.8.0, Asynchronous I/O is supported if you are running JDK version 7.
  • You can easily mix non blocking with asynchronous I/O.
  • Grizzly supports NIO.2. Switching an application from using NIO.1 to NIO.2 is quite simple
  • Bytes are delivered the same way, independently of NIO.1 or NIO.2
Advertisements
Categories: Java, JavaOne
  1. No comments yet.
  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: