JAOO was interesting this year. A collection of very diverse subjects, and many focusing on programming languages – we had presentations about functional programming, JavaScript, Fortress and JRuby. Guy Steele and Richard Gabriel did their 50 in 50 presentation, which was amazing. I’ve also managed to get quite a lot of work done on Ioke. The result of all this is that my head has been swimming with thoughts about programming languages. I’ve also had the good fortune of spending time talking about languages with such people as Bill Venners, Lars Bak, Neal Ford, Martin Fowler, Guy Steele, Richard Gabriel, Dave Thomas, Erik Meijer, Jim des Rivieres, Josh Holmes and many others.

It is obvious that we live in interesting times for programming languages. But are they interesting enough? What are the current trends in cutting edge programming languages? I can see these:

• Better implementation techniques. V8 is an example of this, and so is Hotspot. V8 employs new techniques to drive innovation further, while Hotspot’s engineers continuously adds both old and new techniques to their tool box.
• DSLs. The focus by some people on domain specific languages seem to be part of the larger focus on languages as an important tool.
• Functional semantics. Erik Meijers keynote was the largest push in this direction, although many languages keep adding features that make it easier to work in a functional style. Clojure is one of the new languages that come from this point, and so is Scala. The focus on concurrency generally lead people to the conclusion that a more functional style is necessary. From the concurrency aspect we get the recent focus on Erlang. Fortress aslo seems to be mostly in this category.
• Static typing. Scala and Haskell are probably the most representative of this approach, in trying to stretch static typing as far as possible to improve both the programmer experience, semantics and performance.

Is this really it? You can quibble about the specific categories and where the borders are. I’m not entirely satisfied with where I put Fortress, for example, but all in all it feels like this is what’s going on.

Seeing 50 in 50 reminded me about how many languages we have seen, and how different these all are. It feels like most of the innovation happened in the past. So why is the current state of programming languages so poor? Is it because other things overshadow the language itself? I really don’t believe that. I think a good enough language would enable better tools, more productivity and more successful projects. So why isn’t it happening? We seem to be stuck in a rut. Anders Hejlsberg said in his opening keynote that the last 10-15 years have been an anomaly. I really do hope so.

What is apparent from the list compiled above is that everything that currently happens is very much evolutionary in approach. Innovation is happening, but it’s mostly small innovation.

We need a language revolution. We need totally new ways at looking at programming languages. We need new innovation, unfettered by the failures and successes of times past. We need more language implementors. We need more people thinking about these things.

I don’t know what the new approaches need to be, but the way I see it the last 10 years have been quite disappointing. If programming languages really are important tools, why haven’t we seen the same kind of innovation in that field as we have in IDEs and tools? Why haven’t we seen totally new ideas crop up? Is it because language development is always evolutionary? Does it have to be? Or is everyone interested in the field already convinced that we are at the peak right now? Or that Lisp or Smalltalk was the peak?

What needs to be rethought? I’ve read Jonathan Edwards recently, and he writes a lot about revisiting basic ideas and conclusions. I don’t agree with everything he says, but in this matter he’s totally right. We need to revisit all assumptions. We need to figure out better ways of doing things. Programming languages are just too important. We shouldn’t be satisfied with the current approaches just because we don’t know anything better.

We need a revolution.

12 Comments | By Ola Bini | In: blogging | tags: future of programming, programming language design, programming languages. | #

I’m sitting here at JAOO, waiting for the second day to start. The first presentation will be a keynote by Lars Bak about V8. It was a quite language heavy event yesterday too, with both Anders Hejlsberg and Erik Meijer keynoting about languages – plus there were introductions to both Fortress and Scala going on. And after the JVM language summit last week, I feel like the world is finally starting to notice the importance of programming languages.

So it seems only fitting that I’ve decided to go public with Ioke, the source code, what it is and where it’s going.

Ioke is a strongly typed, extremely dynamic, prototype based object oriented language. It’s homoiconic and got built in support for several kinds of macros. The languages that most closely influence Ioke is Io, Smalltalk, Self, Ruby and Lisp (Specifically Common Lisp).

The language is currently built on top of the JVM, but I’m currently considering compiling it down to JavaScript and run it on V8.

I have several goals with the language but the most specific one is to create a language that combines the things I like about Ruby and Lisp together. It turns out that Io already has many of the features I’m looking for, but in some cases doesn’t go far enough. I also wanted to have a language that is very well suited to express internal DSLs. I want to have a language that doesn’t get in my way, but also gives me loads of power to accomplish what I want. To that event I’ve designed a macro system that some people will probably find insane.

The current status of the implementation is that there isn’t any. I’m starting from scratch. I’ve already created two partial implementations to find the right way to implement the language, so with this blog post I’m starting the implementation from scratch. I know quite well what I want the language to look like and how it should work.

I’ve used Scala for the other two implementations but have decided to not do that for this implementation. The reason being one that Charles Nutter often talks about – that having to include the Scala runtime in the runtime for Ioke seems very inconvenient. So the implementation will initially use Java, but I’m aiming for the language to be self hosting as quickly as possible. That includes creating an Ioke Antlr backend, so it will take some time.

I’m going to post about Ioke quite regularly while I’m working on it, talking about design decisions and other things related to it. I will try to base me decisions in Ioke on what seems right, and not necessarily on the words chosen for representation in other language. I’ll try to talk about my reasoning behind choices like this.

And what about performance? Well, I know already that it will be atrocious. If you want to do scientific computing, maybe Ioke won’t be for you. The current design of the language will make it fairly hard to do any kinds of performance tunings, but I do have a plan for how to compile it down to bytecode at least. This still doesn’t mean it will perform extremely well, but my goals for Ioke specifically doesn’t include performance. I care about performance sometimes, but sometimes I don’t and Ioke is a tool I want to have for those cases where raw expressiveness power is what is most important.

You can follow the development in my git repository at http://github.com/olabini/ioke.

15 Comments | By Ola Bini | In: ioke | tags: future of programming, ioke, metaprogramming, programming languages. | #

I blogged about looking at languages a while back. At that point I didn’t know what my next language to explore would be. I got lots of excellent suggestions. In the end I decided to try OCaML, but gave that up quickly when I found out that half of the type system exists to cover up deficiencies in the other half of it. So I went back and decided to learn Scala. I haven’t really had time to start with it though. Until now, that is.

So let’s get back to the motivation here? Why do I want to learn another language? Aren’t I happy with Ruby? Well, yes and no. But that’s not really the point. You can always point to the Prags one-language-a-year, but that’s not it either. I mean, it’s really good advice, but there is a more urgent reason for me to learn Scala.

I know many people have said this before, but it bears repeating. Everyone doesn’t share this opinion, but have a firm belief that the end of big languages is very close. There won’t be a next big language. There might be some that are more popular than others, but the way development will happen will be much more divided into using different languages in the same project, where the different languages are suited for different things. This is the whole Polyglot idea. And my take on it is this: the JVM is the best platform there is for Polyglot platform, and I think we will see three language layers emerge in larger applications. Now, the languages won’t necessarily be built on top of each other, but they will all run on the JVM.

The first layer is what I called the stable layer. It’s not a very large part of the application in terms of functionality. But it’s the part that everything else builds on top off, and is as such a very important part of it. This layer is the layer where static type safety will really help. Currently, Java is really the only choice for this layer. More about that later, though.

The second layer is the dynamic layer. This is where maybe half the application code resides. The language types here are predominantly dynamic, strongly typed languages running on the JVM, like JRuby, Rhino and Jython. This is also the layer where I have spent most of my time lately, with JRuby and so on. It’s a nice and productive place to be, and obviously, with my fascination for JVM languages, I believe that it’s the interplay between this layer and the stable layer that is really powerful.

The third layer is the domain layer. It should be implemented in DSL’s, one or many depending on the needs of the system. In most cases it’s probably enough to implement it as an internal DSL within the dynamic layer, and in those cases the second and third layer are not as easily distinguishable. But in some cases it’s warranted to have an external DSL that can be interacted with. A typical example might be something like a rules engine (like Drools).

I think I realized a long time ago that Java is not a good enough language to implement applications. So I came up with the idea that a dynamic language on top of Java might be enough. But I’m starting to see that Java is not good enough for the stable layer either. In fact, I’m not sure if Java the language is good enough for anything, anymore. So that’s what my language exploration is about. I have a suspicion that Scala might be a good language at the stable layer, but at this point the problem is there aren’t any other potential languages for that layer. So what I’m doing is trying to investigate if Scala is good enough for that.

But I need to make one thing clear – I don’t believe there will be a winner at any of these layers. In fact, I think it would be a clearly bad thing if any one language won at any layer. That means, I’m seeing a future where we have Jython and JRuby and Rhino and several other languages coexisting at the same layer. There doesn’t need to be any rivalry or language wars. Similarly, I see even less point in Scala and Ruby being viewed as competing. In my point of view they aren’t even on the same continent. And even if they were, I see no point in competing.

I got accused of being “religious” about languages yesterday. That was an interesting way of putting it, since I have always been incredibly motivated to see lots of languages coexisting, but coexisting on the JVM in a productive way.

20 Comments | By Ola Bini | In: Uncategorized | tags: future of programming, jruby, jvm, jython, programming languages, rhino, scala. | #

We seem to be living in the dark ages of programming languages. I’m not saying this to bash everything; I’m actually being totally objective right now. Obviously, our situation right now is much better than it was 10 years ago. Or even 5 years ago. I would actually say that it’s really much better now, than 1 year ago. But programming is still way too painful in almost all cases. We are doing so much stuff by hand that obviously should be done be computer.

I spend quite much time learning new languages now and then, to try to find something that’s really good for me. So far, the best contestants are Ruby, Erlang, OCaml and Lisp, but all of those have their share of problems too. They just suck less than the alternatives.

• Ruby… I really like Ruby. Ruby is such an improvement that I really want to do almost everything in it nowadays. I think in Ruby half the time and in Lisp the other half. But it’s not enough. It is still clunky. I want tail calls. I want real macros. I want blazing speed and complete integration with good libraries for everything and more. I’m just a sucker for power, and I want more of it in Ruby.
• Erlang and OCaml. These languages are really great. For specific applications. Specifically, Erlang is totally superior for concurrent programming. And OCaml is incredibly fast, very typesafe and has great GUI libraries. So, if I was asked to do something massively concurrent I would probably choose Erlang, and OCaml if it was GUI programming. But otherwise… Well, Erlang does have some neat functional properties, but not any nice macro support. It doesn’t have a central code repository and many other things you expect from a general purpose language. OCaml suffers from the same things.
• Lisp is the love of my life. But as so many people before me has noted, all the implementations are bad in some way or another. Scheme is lovely; for research. Common Lisp is so powerful, but it needs users. Lots of them, creating libraries for every little data format there can be, creating competing implementations of particularly important API’s; like databases.

Conclusion. Nothing is good enough, right now. I see two two paths ahead. Two ways that could actually end in the “100-year language”.

The first path is one new language. This language will be based on all the best features of all current languages, plus a good amount of research output. I have a small list what this language would need to be successful as the next big one:

• It needs to be multiparadigm. I’m not saying it can’t choose one paradigm as the base, but it should be possible to program in it functionally, OOP, AOP, imperative. It should be possible to build a declarative library so you can do logic programming without leaving the language.
• It should have static type inference where possible. It should also allow optional type hints. This is so important for creating great implementations. It can also increase readability in some cases.
• It needs all the trappings of functional languages; closures, first-order functions and lambdas. This is essential, to avoid locking the language into an evolutionary corner.
• It needs garbage collection. Possibly several competing implementations of GC’s, running evolutionary algorithms to find out which one is best suited for long running processes of the program in question.
• A JIT VM. It seems almost a given right now that Virtual Machines are a big win. They can also be made incredibly fast.
• Another JIT VM.
• A non-VM implementation. Several competing implementations for different purposes is important to allow competition and experimentation with new features of implementation.
• Great integration with legacy languages (Java, Ruby (note, I’m counting on all Rubyists moving to this new language when it gets out, making Ruby legacy), Cobol). This is obvious. There are to many things lying around, bitrotting, that we will never get rid of.
• The language and at least one production quality implementation needs to be totally open-source. No lock-in of the language should be possible.
• Likewise, good company support is essential. A language needs money to be developed.
• A centralized code/library repository. This is one of Java’s biggest failings. Installing a new library in Java is painful. We need something like CPAN, ASDF, RubyGems.
• The language needs great, small and very orthogonal libraries. The libraries included with the language needs to be great, since they have to be small but still pack all the most needed punch.
• Concurrency must be a breeze. There should be facilities in the language itself for making this obvious. (Like Erlang or Gambit Scheme).
• It should be natural to do meta-programming in it (in the manner of Ruby).
• It should be natural to solve problems bottom-up, by implementing DSL’s inside or outside the language.
• The languages needs a powerful macro facility that isn’t to hard to use.
• Importantly, for the macro facility, the language needs to have a well-defined syntax tree of the simplest possible kind, but it also needs to have optional syntax.

So, that’s what I deem necessary (but maybe not sufficient) for a really useful, good, long term programming language. When I read this list, it doesn’t seem that probables that this language will show up any time soon, though. Actually, it seems kinda unrealistic.

So maybe the other way ahead is the right one? The other way I envision is that languages become easier and easier to create, and languages have their strength in different places. Along this path I envision the descendants of Ruby and Erlang exploiting what they’re good at and eschewing everything else. But for this strategy to work, the first thing implemented in each language needs to be a seamless way to integrate to other languages. Maybe there will come an extremely good glue-language (not like Perl or Ruby, but a language that only will serve as glue between programming languages), and all languages will implement good support for that language. For example you could code a base Erlang concurrent framework, which uses G (the glue language) to implement some enterprise functionality in Java sandboxes, and some places where Ruby through G will implement a DSL, which have subparts where Ruby uses G to run Prolog knowledge engines.

If you had to choose among the two futures, I am frankly more inclined towards the one-language one. But the multi-language way seems much more probable. And since I’m trying to choose way now, I’m placing my bets on the second option. We are not ready to implement G yet, but I do think that as many p-language techs as possible should do their best to learn how languages can cooperate in different ways, to prepare this project.

19 Comments | By Ola Bini | In: Uncategorized | tags: future of programming, lisp, metaprogramming, programming languages, ruby. | #