So, JavaOne is finally here, and it starts big! This blog will talk some about the different sessions I’ve been to during the day, but first a few impressions. There are very many people here. John Gage said at the first general session that this is the largest JavaOne ever, and I believe him. I’m sad to say that the WiFi network is very spotty at best.

JavaOne has a new way of getting into sessions; you have to use their schedule builder to reserve places for a session, and then use your RFID chip to register when entering a specific session. I thought this was a mad idea, but it actually works really well, and I’m glad to say that the JavaOne team seems to have solved most of the overflow troubles from last year.

Another small reflection is that the focus on compatibility in the core platform seem to be a major focus this year. I’ve seen and heard the word spoken more times than I can speak.

General session
The first general session is always one of the more interesting times during JavaOne. Both Sun and other company leaders are just brimming with announcements. One of the more interesting quotes of the day came fairly early, from Rich Green: “It’s not a question of whether, it’s a question of how“. As you may guess, this was an answer to the question of open sourcing Java. In plain language the situation is that Java will be open source, as soon as Sun finds a good way to do it in.

Java Enterprise Edition 5 is now finished, released and of production quality, and these are some highlights from the release:
It’s very focused on ease of development.
Contains much Web 2.0 support.
Interoperability with .NET have been greatly enhanced. (Except for compatilibity, interoperatibility is the major illy today).
SOA has been simplified.
They have simplified the programming model, mostly by using annotations.
New EJB version, using plain old Java objects.
A new (annotation-based) persistence API.

During the general session, many, many libraries and applications were open sourced, among these JMS. It’s nice to see that Sun really wants open source to work.

Session on EJB 3.0
The idea for EJB 3 was to make it easier for the developer, by making the container harder to implement. This tradeoff seems reasonable in retrospect, but until now most of the EJB specification made it easy for the container.

The new model is based on letting the container provde requested services to the bean, and also using reasonable defaults for most operations.

Once again, the presentation pressed very hard on compatibility. Existing applications had to continue working. Clients using the new API should have no trouble connecting to servers using the old libraries and the other way around.

Most of EJB 3.0 is based on POJOs and declarations with either annotations or XML information.

Environment access is made easier with dependency injection or simple lookup.
Client code also uses dependency injection, which means the new model removes the need for home interfaces, PortableRemoteObject.narrow, RemoteException and other checked exceptions.

The EJB lifecycle doesn’t use explicit callbacks anymore. Instead you can annotate the method that should get notifications about a lifecycle change and this will be taken care of by the container. You can also allow a separate interceptor as the notification and callback manager. Very neat.

All in all, EJB seems to be heading the right way, at last. Ease of development really matters, and configuration by reasonable defaults have already been shown to be a viable solution.

Technical general session
The technical session had some interesting information too. Most of the talk was about Mustang (Java SE 6) and Dolphin (Java SE 7), and what we could expect from these releases.

The projections for Mustang looks good, and it is scheduled for release in October. They have been using a new, very open process for Mustang development, which have worked extremely well. The same system will most likely be used even more in Dolphin development.

So, some nuggets of good stuff in Mustang:
Many performance improvements. They showed some pretty convincing performance graphs, and I was duly impressed.
They’ve fixed the so called gray rect problem in Swing, which results in heightened perceived performance dramatically.
There is many improvements in the monitoring and management areas.
Scripting support will come, with JSR 223. Also, check out http://scripting.dev.java.net.
There are many desktop fixes, and Vista is the desktop focus for Mustang.

The talk went on to the future for Standard Edition:
Probably direct support for XML in the language.
Super packages, and new module system for packaging with versioning.
They’re thinking about adding BeanShell to the scripting languages provided by the core language.
And more, more,more desktop stuff.

Hamilton went on to talk about scripting- and dynamic languages, and more or less recommended using them in the Web tier and in situations where a fast cycle of development is required. This layer can then use Java for the business logic. (And this is really what KI is heading for right now; using Ruby on Rails for web, and using SOAP to get at Java business logic exposed with Web Services.)

Anothing thing that’s coming is JSR 292: the new bytecode for dynamic invocation.

An interesting demo of a Visual Basic to Java compiler. The system is not a total clone, but will enable people used to Visual Basic to program for the JVM instead. It will not enable translation of existing applications, though.

Mustang and Dolphin session
This session started out with Mark commanding us to upgrade to Tiger now, there is really no reason not to do it.

So, what’s new in Mustang?
Some class file changes.
A new Compiler API.
New annotation processors.
JDBC4.
Scripting support.
Streaming API for XML.
Common Annotations.
WS-Metadata
and JAX-WS 2.0.

Mark top ten list of new things in Mustang:
10. Attach-on-demand monitoring.
9. Plugin API for JConsole.
8. jhat OQL (a query language to explore heap dumps).
7. Solaris d-trace.
6. javac will now do annotation processing interleaved with compilation.
5. Classpath wildcards!
4. API for finding free Disk-space.
3. API for password prompting.
2. New Grouplayout for Swing.
1. That JAX-WS can do RESTful web services.

Mustang will also bundle Apache Derby, a small inmemory database.

There are many ideas for what Dolphin will contain. There are some interesting things that can be really cool.
These are mostly core language changes:
Properties, improving getters and setters.
Real method references.
Block closures.
Native XML support.
The new bytecode for dynamic invocation.
Bundling BeanShell.
And beans binding for making Swing more easy to use.

The session also had some very fun information about how the testing of JDK is done. It’s really quite amazing. The big trouble with creating new versions is disconcerting fact that running a full test cycle takes 10 weeks.

Session on the new Concurrency features in Java 5
The session began with some talk about the rationale behind the new concurrency features in Java. The easy answer is that real concurrency is hard to do right, and the builtin primitives for threads and locks are, well, primitive.

The new concurrency packages have something for everyone. There is both easy-to-use utilities for mostly anyone, and also some primitives for hard core programmers, that enable some things that just can’t be done in Java right now.

Java has always had thread-safe collections, but these are not conccurent. They also had a bad performance structure. Therefore a few new conccurent collections has been added. Most of them allow unlimited reads and up to 16 simultanous writes. Mostly, the semantics are the same, but they differ in a few areas. The most glaring difference is iterators. With the old collections you got a ConcurrentModificationException if someone updated the collection while you were iterating over it. This will not happen anymore. Tiger has a new Map called ConccurentHashMap and Mustang adds a SkipListMap.

A new collection interface has been added to java.util, which is called Queue. This is a subset of the List functionality, that can be used for implementing high performance versions with just this restricted functionality.
The most interesting Queue is the BlockingQueue, which makes explicit a producer-consumer relationship. Mostly all code already uses something like it, but this implementation is industry strength and very easy to use.

They have hadded ThreadPools with a system of Executors and ExecutorServices. They are very easy to use and configure with factory methods.

Another interesting addition is Future and Callable, which is used to put an execution inside another thread and then get the value when it’s finished. But it is not
call-by-need, which was what I first thought. The value will always be calculated, even if it’s not needed.

Scheduling primitives have been added, to replace Timer and TimerTask.

There are also some new advanced features for locking. There are some things you just can’t do with the synchronized keyword in Java. Hand-over-hand locking is one example. The concurrency library adds Locks, Conditions and Semaphores for more advanced use. They are very complicated so they shouldn’t be used if you don’t really need them. One good reason for this is that you have to release locks manually.

JUnit next generation
JUnit has many warts and problems. How do you run one test in a test case, for example?
It’s not really good for anything but unit testing. It has been very few updates and the
protocol is intrusive. It also uses a very static programming model. And it doesn’t use the latest Java features.

TestNG is the new JUnit. It uses annotations for most configuration. There are test groups which can be dynamic depending on your needs.
You also have the possibility to have dependent tests, parallell testing, load testing and partial failures. It has also got a very nice plugin API.

All in all, it looks really good. My personal opinion is that there is no reason not to use TestNG instead of JUnit on all
Java 5 projects.

Restructuring a web application with Hibernate and Spring
This was the first BOF for me this JavaOne and I was very disappointed. The presenters use case was a web application that had been badly written from the beginning. They then decided to rewrite it with Hibernate and Spring, but there really didn’t seem to be much better code written for this application. I guess it’s a testament to how good Spring and Hibernate are, that they got a really good performance improvement anyway.

Testing a persistence layer
Testing a persistence layer is really hard, for a few different reasons. You want it to be fast, and easy to write. There are many different strategies to testing persistence, and the presentation talked about most of them.

There are a few different kinds of persistence layers. They can be SQL-based, Object/Relational-based or using the ActiveRecord pattern (which the presenter viewed as a special case of SQL-based persistence).

As noted above, there are many strategies available.
You can mock the DAO’s for testing the business logic. This is easy if you use a dependency injection framework like Spring to initialize the DAO objects. Otherwise this is the main problem. It’s very fast but the scaffolding can get hairy to write.

You can also mock the ORM-framework, to test the DAO’s. The presenter have written a utility called ORMUnit to faciliate this testing.
Regarding mock frameworks, the presenter prefers JMock, but is planning on migrating to EasyMock instead.

Another strategy is to test the metadata mapping and schema. This checks for stupid errors like forgetting to map a field. It can also checks that all referenced tables and columns actually exist.

Of course, the standard way of testing persistence, by doing CRUD operations is also available, but it’s complex, and very slow. You have to write much code to drop and add data for each test.

Another simple way is to check the generated queries directly, to see that they return correct data.

Using an inprocess database can be very fast, but may also lead to trouble with incomplete SQL implementations.

A final strategy for testing slightly things faster is doing operations but never committing transactions. This has the advantage of being fast and also avoids actually changing the DB.

Summary
All in all, it was a busy first day. Much of it was very interesting, and I’ve learned many new things. I just regret missing the Scripting Languages BOF at the end of the night, but I was just to beat to manage going to it.

No Comments | By Ola Bini | In: Uncategorized | tags: Uncategorized. | #

The Fireside Chat is usually an interesting conversation between JavaOne alumni and some of the more Java founders. This year it was James Gosling, Graham Hamilton, Jeff Jackson and a few others. These are more disjointed notes of what I am interested in, than a running commentary on everything that’s said.

The format for this chat is that someone in the audience asks one ore more questions, and the panel tries to answer as much as possible. Nothing advanced.

Questions about Java Applets downloading time, is there any plans on having a Java Web Edition? Short answer: no.

Performance, specifically the startup time of the Java engine; Sun have tried, but right now it seems hard to get it any faster without removing significant functionality.

I gathered from some comments that one of Sun’s primary priorities right now is good interoperability with Microsoft and .NET.

Regarding deprecated API’s, it seems likely they will never disappear, at least not until there is enough good tool support to actually remove and refactor all dependencies on such code.

AJAX: Sun is handling this in a few ways; with toolkits for generating JavaScript with servlets and stuff like that. JSF should create AJAX-aware components without us needing to
have to care about AJAX.

Then the panel got the question what one thing they’d like to remove from Java if they could. Goslings immediate answer was java.awt. He got off on a tangent and talked a little about how the basic feature set of Java was decided, and that he only put in stuff that people really, really needed. The result of this is that he doesn’t regret anything with the language. He seemed particularly glad that he didn’t try to put in generics or enumerations from the beginning, because he would probably have failed doing it “right”.

If he could add something to the language, that would be really lightweight objects. Some kind of structs, for implementing the canonical example of an object oriented number class, for example. Right now the most lightweight classes are still way to heavy. Also, Hamilton would like to improve getters and setters, but he doesn’t know how yet.

There seems to be some interesting improvements to JavaBeans in Dolphin.

Hamilton also said that he would like to undeprecate java.util.Date, since it is very much more usable than java.util.Calendar most of the time.

Sun is also working on really good refactoring tools. Gosling has a vision of “lint on steroids”. They have a few good prototypes, but nothing close to being released yet.

Another point in planning is better performance for JNI. Make it possible to remove checks and stuff like that. It seemed that arrays were the big problem. You can get quite good speed already, by not using arrays, and having as much NIO as possible.

These were my opinions on the more interesting stuff from the Fireside Chat.

No Comments | By Ola Bini | In: Uncategorized | tags: Uncategorized. | #

. sI am not an experienced Python programmer. When I decided to port PyYAML3000 to Ruby I wasn’t sure exactly what kind of problems I would run into. The stuff I found out when trying to get more or less the same semantics in Ruby as in Python was actually quite interesting and I have the notion that some of it may be of interest. At least I will get my thoughts on the translation straight.

The first phase of RbYAML was supposed to be a straight forward port of PyYAML from Python to Ruby. This ideal was of course impossible since some things really didn’t work the same way in the two languages, but mostly the semantics translated cleanly. The biggest change in architecture between the implementations was the Unicode support – which I had to remove for now – and the inheritance structure. PyYAML uses the Python version of multiple inheritance to provide basic combination of functionality. Since Ruby doesn’t have MI I implemented this using mixins instead. This is probably the first thing that I will change for phase two of RbYAML since mixins (and modules) just doesn’t work the same way as inheritance. Right now I’m thinking that I’ll probably use the Composite pattern where Loader and Dumper export the interface that’s available, and all other operations will have to be done explicitly with the separate components.

The next thing that caused some trouble was Python’s notion of truth and false. In Ruby, only the values nil and false is considered false, everything else is true. In Python a whole range of values are considered false: False, None, 0, the empty string, the empty list and the empty dict.

Another big stumbling block for implementing the same architecture in Ruby as in Python was the heavy use of coroutines
in a few parts of PyYAML. Right now I implemented this by just doing a really inefficient version that collects all nodes
before sending them away. I could of course have implemented a coroutine engine with call/cc, but since JRuby doesn’t support
continuations yet, this would have defeated the purpose of the project. My plan for the next phase is to reorganize the code so that I
can use Ruby’s standard idioms for iterators, which should improve both performance and memory usage drastically.

Some of the regular expressions wasn’t really compatible with each other, but this was pretty easy to fix.

The final trouble was Python’s use of requiring () for invocation of a method, and if a method is referenced without parenthesis, this means
that the symbol for the method is referenced instead. This was mostly a problem because of my unfamiliarity with Python.

So, in conclusion, porting Python to Ruby is quite easy, except for a few small areas. I would try it again if there existed an application that’s needed for
Ruby.

This is the first part of a series of blogs about RbYAML. The next entry will talk more about Rubyfying the code quite much.

1 Comment | By Ola Bini | In: Uncategorized | tags: Uncategorized. | #

For a few years I’ve been involved with an open source project called SpocP, through work. SpocP is a framework and protocol for handling mostly any authorization tasks any application can have. The base distribution is a server written in C, which when started with rules definitions answers TCP (or socket) requests with YES or NO, and attaching a blob if the rule says so. If you want to, you can write all your authorization information directly in SpocP rules. This is usually not that interesting since the common case is that you have a data source somewhere that you’d like to use as basis for authorization. Maybe a LDAP server with role information, maybe an RDBMS, or something entirely different.

The interesting thing about SpocP is the format used for Rules and Queries. (Rules are what SpocP uses to provide an answer, a Query is a specific authorization request translated into the SpocP Query language. In human terms a rule might be “All persons who have OU=itc in our LDAP server have access to all pages of the web application named ‘itdoc'”, and a typical query that might answer YES for this rule is “Can the person with uid ‘olagus’ access the page ‘showLinks.do’ for application ‘itdoc’?”). The format for queries and rules in SpocP is based on S-expressions (which is a fancy word for more-or-less-Lisp syntax). The reason for using S-expressions is the interesting fact that S-expressions have the mathematical property that one S-expression can be compared to another, and judge less-than-or-equal or not. This is all that’s needed to implement generic authorization facilities, and also some other fascinating possibilities. As an example, take a rule file looking like this:

(spocp (action view) (resource (app “itdoc”) (page “admin.do”)) (subject (uid)))
(spocp (action view) (resource (app “itdoc”) (page)) (subject (uid)))
(spocp (action admin) (resource (app “kimkat”)) (subject (uid) (role)))

and a query like this:

(spocp (action view) (resource (app “itdoc” “http://itdoc.it.ki.se”) (page “showLinks.do”)) (subject (uid “olagus”) (role “user”) (loggedIn 200505260115)))

When receiving this request, SpocP will walk through all rules until it finds anyone where the RULE is <= QUERY. If it does find one matching it will return "YES" otherwise "NO". In this case rule number 2 would make the answer "YES". You've now seen the first part of SpocP matching. But how do we involve external data sources? With something called boundary conditions. The idea is that for each rule matching, check the attached boundary conditions, and if these return positive answers too, the whole expression is deemed true. A typical boundary condition may look something like this:

urn:spocp:ldapset:ldap.ki.se;ou=people,o=ki.se;{uid & ${uid}}/ou & “itc”

which more or less is the same as the LDAP query (&(uid=$UID)(ou=itc)).

This boundary conditions checks if the second element of the list with tag “uid” exists in our LDAP server, and if that entry also has an ou with value “itc”. Boundary conditions can be chained and linked with and, or and not, and you can also predefine boundary conditions so you don’t have to define the same one more than once.

I recommend using SpocP if you have a need to decouple your authorization from the application. As soon as you have defined what kind of queries a typical application may do, you can implement this on a central server, and when your business rules change, you can change the SpocP definitions without having to change the application. There is not much extra complexity involved and you buy yourself some very nice flexibility. There are SpocP client libraries available for most of the standard languages.

The fun thing with SpocP is that the fundamental operation of checking less-than-or-equal between two S-expressions have always been defined mathematically, and in some highly optimized C-code. When fooling around with this, I found a simple executable definition of the SpocP core in Common Lisp. The whole thing look like this:

(defun starform-p (list)
(eql (first list) ‘*))

(defun starform-match (query rule)
(let ((form (second rule))
(data (cddr rule)))
(case form
(any (member query data))
(prefix (string-equal (car data) (string query) :end2 (length (car data))))
(suffix (string-equal (car data) (string query) :start2 (- (length (string query)) (length (car data)))))
(range (member query data)) ;;not implemented
(set (member query data))
(t t))))

(defun matches-p (query rules))
(some #'(lambda (rule) (match-p query rule)) rules))

(defun match-p (query rule)
(if (and (atom query) (atom rule))
(eql query rule)
(if (starform-p rule)
(starform-match query rule)
(every #’match-p query rule))))

The only thing missing here is one of the so called star-forms, range. Otherwise this is a complete definition of the SpocP core evaluation of rules. To use it, do something like this:

(matches-p ‘(spocp (action view) (resource “foo”)) ‘((spocp (action admin) (resource) (subject (uid) (role “admin”)))(spocp (action view) (resource “foo”))))

2 Comments | By Ola Bini | In: Uncategorized | tags: Uncategorized. | #

A few weeks ago, Craig Larman visited KI and had a one day presentation about agile development to some 30 people from different professions at KI. I was there and some parts were very interesting.

Of course, most of it is not new for the developers, but the we got some good firepower for convincing management about some more realistic practices. KI haven’t been very good at embracing the new and hot in development methodology the latest years, but hopefully this is about to change.

So, what about the presentation? The first part, ’til lunch, were mostly statistics and “management-convincing” arguments why agile is good practice. The only really interesting part were Larman’s dissecting of Royce’s original paper on the waterfall method. It’s really totally obvious to anyone that actually reads the paper instead of just looking at the graphs, that Royce really is against the waterfall method except for very small projects when all requirements are known upfront and cannot be changed.

The afternoon session contained much more of interest, at least for me. He talked some about good tools to practice good Agile, naming continuous integration, test driven development, good version control, self organizing teams and good dissemination of information through wikis as key points. IT Center – that is, me and me 4 colleagues – at KI are pretty good at this. The only weak point is our use of Wiki. I have some plans for this, since I totally agree with Craig about this. Wikis are unique in the way they combine simplicity with expressability and information processing.

After this I finally got something really worthwhile out of the presentation; Craig presented the first good argument for pair programming that I’ve ever heard. Regarding Pair Programming I’m probably like most programmers: very skeptical. I have tried it a few times and I don’t like it. But anyway, Craig’s premise was a situation where you had programmers working on several projects simultaneously, and also having maintenance work on applications they had the responsibility for.

In this case it’s very easy that people get unproductive, due to insistent interruptions from other projects or immediate maintenance work. Usually this most of the maintenance work is also person dependent. (This situation describes exactly my groups current work environment.) When dealing with issues like this it makes sense to create a rotating schedule where programmers work on one project at a time, or completely on maintenance. When something that is dependent on someone in a project shows up, one of the maintenance programmers pair program with this person to fix the problem, which ensures that the next time this problem appears, there will be at least two persons capable of handling it.

All in all, it was an interesting session and I felt I got out of there knowing more than I knew before. The most important thing is still having someone to have as reference when talking with management about adopting more agile methods.

Maybe I’m imagining this, but it feels like since Craig was here, we’ve been allowed much more slack in using technology that’s not “industry standard”, but more agile to solve tasks suited for problems suited for it. For example using Ruby on Rails to create internal test and development tools for bigger projects seems to be totally accepted now.

No Comments | By Ola Bini | In: Uncategorized | tags: Uncategorized. | #

This is an announcement of the availability of RbYAML, a pure Ruby YAML parser, based on the Python project PyYAML3000. The last two weeks I’ve converted this to Ruby, with more or less the same functionality. The glaring hole is Unicode, which isn’t there right now.

The project will be updated regularly, and will also soon move to RubyForge.

Right now the homepage is here: http://rbyaml.ologix.com

In a few days I’ll post a longer blog with some information about the Python-to-Ruby translation, which had some interesting surprises, at least for me.

No Comments | By Ola Bini | In: Uncategorized | tags: Uncategorized. | #

Due to my long time away from such matters, the proposed YAML engine for Java that I intended to create have not got so far. Right it seems as if YAML is a major bottleneck in JRuby, so I started looking around. Loo and behold, at http://jyaml.sourceforge.net/ there seems to exists something that’s almost exactly what we need. So I’ve impromptu decided to discontinue my JAML project and instead tackle the task of making this engine work together with JRuby.

RubyGems is really close right now. Each step takes us closer and closer, and right now it feels as if we have 99.99% of everything needed. If just the SourceForge CVS-system could start working we could actually really get this of the ground now.

1 Comment | By Ola Bini | In: Uncategorized | tags: Uncategorized. | #

Alright, San Francisco: here we come! Our development group is going to JavaOne together this year. Actually, this is the first time there has been a development group to talk about, since I was it’s only member until this summer. And there seems to be much fun going on. I’m particularly looking forward to the sessions with a focus on other languages. Sometimes I find it strange to have lots of Java Certifications and still loathe the language.

Things I’m not going to miss: The JRuby session, Best practices with Generics and other Tiger features, What’s new and exciting with AspectJ, Rapid web application development with Grails, Portlet Best Practices, Trails in depth, Server-side scripting, Scripting in Mustang, Effective Java reloaded, Mustang and Dolphin. And these are just a few. Wow, it’s going to be a busy week, and trying to meet up with lots of friends and contacts in SF is going to make a complete experience of this.

By the way, it’s interesting to see that there are many sessions on Spring, Dependency Injection and other stuff that make Java lightweight. One title especially caught my eye: “DI: Configuration Files Must Die”. My take on the configuration file-issue is to have it as executable as possible. A project I’m working with on and off in purview of Ologix will use Common Lisp and judicious use of macros to create the configuration information. The nifty thing about this is that I haven’t yet decided what the main language will be. It could be Lisp, it could be Ruby or it could be Java. But whatever I choose it will still be fairly easy for me to handle the configuration issue. In Lisp I can just read-eval it, in Ruby it’s very easy to parse and in Java I could use Jatha and expand the macros to calls into the Java structure. Not very efficient, maybe, but configuration is not performance. Flexibility is king.

Hope to see you in San Francisco.

1 Comment | By Ola Bini | In: Uncategorized | tags: Uncategorized. | #

Due to massive loads of work I have been unable to write here for a few months. That will probably change as of right now, since the current projects are finally soon closing down and the workload returns to more or less normal.

Oh, and if you for some reason haven’t read Steve Yegge’s stuff yet, go do it here and on his blogspot here. This guy know what he’s talking about.

2 Comments | By Ola Bini | In: Uncategorized | tags: Uncategorized. | #

A very interesting article from IBM Developerworks: Ruby off the Rails.

On a similar note, I propose that Jakarta Struts should be renamed to Java in Jail.

No Comments | By Ola Bini | In: Uncategorized | tags: Uncategorized. | #