New projects

OK, I’ve hinted and talked about in several contexts that I have some projects going on that haven’t been announced.  So today I’m going to just mention two of them (so that they will be released at some point). (And no, ioke won’t be on this list).


This one I have real hopes for. It’s already functioning to a degree. So what does it do? Well, I call it example-driven DSLs. Instead of writing a parser/regexps for handling external DSLs, Xample will take some examples and then derive code to do something based on those examples. It’s one of those things that will not do everything, but it might solve 80% of all the simple DSLs. Those cases where the overhead of creating the DSL framework would not be worth it. Xample will help in those cases in making it really easy to create quite sophisticated DSLs. It’s in Ruby, and I have plans that include a generic work bench for handling these DSLs. Since you have examples of how they should look, you can use the information in all kinds of cool ways.

I have just made the Xample repository public on github.


This is probably one of the worst cases of yak shaving I’ve ever started on. But it’s a good and worthwhile project. And now that I’ve mentioned it to Terence I guess there’s no way of getting away from it… =)

So, simple, Antlr-ELisp is a backend for Antlr, that allows you to create parsers for Emacs Lisp. This should make it quite easy to get Emacs modes written for a language, without having to resort to all the awful hacks Emacs generally does to handle different languages. This one is really in the infant stage right now, but it looks like it shouldn’t be too hard. Of course I’m not sure what performance will be like, but I’m trying to use as much macros as possible so the low level operations can still be efficient without looking ugly.

This project is also on github right now.

So, now I’m stuck – talking about these things on my blog means I will actually have to get both of them to a release, really soon now!

Short term adoption and Ruby DSLs

I feel like I’ve been picking a bit on DataMapper in my recent posts, so I first want to say that I like most of what I see in DataMapper, and is generally just nitpicking. I’m also what I’ve noticed there to give examples instead of making up something. And actually, one of the reasons is that it was some time since I read much Ruby code, and DataMapper happens to be one of the things I’m looking a lot at right now.

Which brings me to something Sam Smoot said in the discussion about adding operator methods to Symbol. (In the blog post Simplified finders in DataMapper). What he said was this:

That might satisfy some, but it ranks pretty low on the “beauty” gauge. And that really does matter for adoption…

Now, the reason I’m bringing it up here is not to pick on Sam or DataMapper (Really!). Rather, it’s because an attitude that I’ve noticed all over the Ruby world lately. In most cases it’s not explicitly said like this, but I like Sam’s quote because it verbalizes exactly what it’s about. You really want your library to look nice. Beauty is really important in a Ruby framework. It really is important, and lots of focus is put on it. And one of the reasons for this is to drive adoption.

Since it’s been verbalized like this, I’ve realized that this is one of the things that really disturbs me with the Ruby communities fascination with making everything, absolutely everything into a DSL.

Now, I really do like thinking about DSLs and working with them. Being a language implementor brings that out in me. But a DSL really has it’s place. Why shouldn’t you make everything into a DSL? Why shouldn’t everything look nice? First of all, the initial development might just not be worth it. A DSL-like approach generally involves more effort, so maybe a regular API works fine? A DSL introduces cost, both in development and in maintenance. Everything you do needs to be balanced. My coworker Marcus Ahnve gave a very good example of this today. He wanted to use ctags with an RSpec file, but since ctags doesn’t understand “describe” and “it”, there was no way for it to extract any useful information from it. Now, this is a trade of the framework designer has to make, and in the case of RSpec I think that the DSL is totally justified, but in other cases it’s not.

So take the issue at hand. Adding a few methods to Symbol, so that finding information in a database will look a bit nicer. In real terms this involves polluting the Symbol namespace with methods that have a very narrow scope. The usage (that looks more or less like this: Exhibition.all( => 3)) uses something that reads easily from an English/SQL perspective, but not necessarily from a Ruby perspective. The main question I have when seeing this is what the eventual cost down the line will be for having it. Is this approach expandable? Is there any possibility of clashes with other libraries? Will someone else easily maintain it?

Focusing on beauty to generate adoption seems like the wrong choice for me. You add things that might not be so good long term, to get a larger initial code base. I would prefer to make the right choices for the right reasons first, and then let adoption be driven by that.

In summary, be responsible when designing APIs. I know those are very boring words. But it’s a reality, and your users will thank you for it.

PS: Sam just wrote a new comment, saying that the Symbol operators will be optional in the next DataMapper release. Good choice.

Java Annotations

I feel that a few rants are about to burst in me. Let me take this first; this is not a rant, though. Just a very large exclamation:

Java Annotations are NOT a way to implement DSLs, and will never be.

Java annotations are really nice, but seriously, they are NOT manna from heaven.

Concurrent DSLs

With all the current talk of DSLs and concurrency, what I find lacking is discussions about how to combine the two. Of course, domain specific languages are incredibly important – they create a logical separation between the implementors of the business logic, and the people implementing the actual implementation of the DSL. Does it seem like a strange idea to want many DSLs to be able to run parallel to each other? I would imagine that in most cases a DSL that describes business rules and business logic is sequential in the particulars, but that there are also larger concurrency possibilities. This should be totally invisible for the business rule implementor in most cases – the runtime system should be able to run everything as efficient as possible.

A natural way of looking at DSLs is as declarative languages. In many cases that’s the way you write them (just look at the ActiveRecord API. it looks very declarative – it just happens to be implemented using imperative primitives). Now, if the language is truly declarative it should be side effect free. In the end, that isn’t a real goal, but if it would be possible to clearly understand which parts of the rules are using side effects, the rest of the implementation should be able to run totally concurrently.

These kinds of things should be possible to implement in any language with sane multi threading/multi processing. That said, I wouldn’t want the task of doing it with Java’s concurrency primitives if I can help it. So what kind of tools would be helpful? Possibly Erlang of course, since it’s already functional and that makes the identification of side effects much easier. Another possible alternative seems to be Gambit Scheme and Termite.

Anyone else thinking about these issues? Is there any research going on that would shed some light on it? And further, what’s the next step? Why haven’t this question already been discussed? It seems to be well time for it now.