Static type thinking in dynamically typed languages


A few days back I said something on Twitter that caused some discussion. I thought I’d spend some time explaining a bit more what I meant here. The originating tweet came from Debasish Ghosh, who wrote this:

“greenspunning typechecking into ruby code” .. isn’t that what u expect when u implement a big project in a dynamically typed language ?

My answer was:

@debasishg really don’t agree about that. if you handle a dynamically typed project correctly, you will not end up greenspunning types.

Lets take this from the beginning. The whole point of duck typing as a philosophy when writing Ruby code is that you shouldn’t care about the actual class of an object you get as argument. You should expect the operations you need, to actually be there, and assume they are. That means anyone can send in any kind of object as long as it fulfills the expected protocol.

One of the problems in these kind of discussions is that people conflate classes with types in dynamic languages. In well written Ruby code you will usually end up with a type for every argument – that type is a number of constraints and protocols that will wary depending on what you do with the objects. But the point is that it generally will make things more complicated to equate classes with these types, and you will design classes without any real purpose. Since you don’t have static checking, you don’t need to have overarching classes that act as type specifiers. The types will instead be implied in the contract of a method.

So far so good. Should you keep this in mind when designing a method? In most cases, no. I tend to believe that you will end up conflating classes and types. That’s what I’ve seen on several projects at least. The first warning sign is generally kind_of? checks. Of course, you can do things this way, but you will restrict quite a lot of the power of dynamic typing by doing this. One of the key benefits of a dynamically typed language is the added flexibility. If you end up greenspunning a type system, you have just negated a large part of the benefit of your language.

The types in a well defined system will be implicitly defined by what the method actually does – and specified by the tests for that method. But if you try to design explicit types, you will end up writing a static type system into your tests, which is not the best way to develop in these languages.


26 Comments, Comment or Ping

  1. Duck typing falls down when you need to do foo.respond_to?(:delete), because lots of objects could respond to delete, but they could have very different semantic meaning, so you can’t just blindly send it the method and cross your fingers.

    April 6th, 2009

  2. Paul:

    I don’t agree. Doing respond_to? should mostly be avoided too. And it’s the callers responsibility to send in something that does the right thing on the call to delete.

    April 6th, 2009

  3. Dynamic typing does not equal duck typing So there is no sense in which “kind_of?” is greenspunning a static type system – it’s dynamic typing without duck typing. See Scheme. Dynamic typing != duck typing.

    Static typing does not equal nominative typing, for that matter. See structural typing in O’Caml and Scala. Static typing != nominative typing.

    Finally, class doesn’t equal type in any of today’s popular statically typed OO languages either. In lowly Java ArrayList a different type from ArrayList is a different type from ArrayList – but they all come from the same class. Class != type.

    April 6th, 2009

  4. brackets got eaten in my last comment. Should be

    In lowly Java ArrayList[String] a different type from ArrayList[Integer] is a different type from ArrayList[? extends Object] – but they all come from the same class. Class != type.

    April 6th, 2009

  5. James:

    Well, I don’t really agree with you. First of all, I never say dynamic typing == duck typing. Duck typing is a philosophy in how you can write code in a dynamically typed language.

    Using lots of kind_of? checks still ends up greenspunning a type system in many circumstances. I don’t know what you mean by looking at Scheme – I have never seen that kind of kind_of? checks in Scheme.

    I assume you mean nominal typing. I know that static typing isn’t necessarily nominal. The way you use structural typing is generally dissimilar to both nominal static typing and dynamic typing. Scala is mostly nominally typed, though.

    Also, you are right in that with generics in Java, different ArrayList reifications isn’t the same type.

    I’m not exactly sure what you’re trying to prove with these examples.

    April 6th, 2009

  6. If used sparingly, I don’t see use of respond_to? as a particularly odious code smell. You could consider it as an “optional message” in a duck-typed protocol. The addition of Object#try in ActiveSupport points in this direction.

    class Object
    def try(message)
    send(message) if respond_to?(message)
    end
    end

    foo.try(:delete)

    April 6th, 2009

  7. Nick:

    You are totally right. There are valid uses for respond_to? but loads of uses of it is generally a smell. I was being a bit polemic in my post. =)

    April 6th, 2009

  8. Hi Ola –

    Thanks for the post. I fully agree that for most of the cases people coming from the static typing world fail to distinguish between types and classes. As you mention in the post, structural typing is the most natural way to think when you are using dynamic languages. I agree you should not think in terms of classes, think in terms of types, which are abstractions that respond to a bunch of messages.

    Consider the following example in Ruby ..


    # Accepts an object which responds to the year, month and day
    # methods.
    attr_accessor :date

    Here the documentation is the contract that mandates the set of messages to which the type of :date should correspond to. There is nothing in the live code that actually enforces this. And there is every chance that an incorrect value passed will result in a NoMethodError during runtime.

    The common recommended solution will be to use unit tests, where you will be doing the type checking tests for :date. Instead of in live code, we move the type checking to test code. Don’t you think it is greenspunning typechecking into Ruby code ? Unit tests are also Ruby code, after all ..

    Have a look at http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/100511. Towards the end it says “the moral of the story being that when Duck Typing, do your checking in your unit tests, rather than in the live code.”

    Thanks.
    – Debasish

    April 6th, 2009

  9. Debasish:

    Well, this is where we get into a discussion on what actually is a type checking system. I posit that you shouldn’t create your unit tests to test type invariants, but instead actually just test what the method should do. As a side effect of that, the unit test will specify the semantics of some type that matches this description, but I still think you’re doing yourself a disservice if think about it in either structural or nominal types.

    So, no, I don’t think that is actually type checking – at least not if done right.

    April 6th, 2009

  10. @Ola

    In Scheme predicates like “complex?” and “integer?” are used to create code that behaves differently based on runtime type. That’s not how it’s usually done in Ruby, granted, but it’s still dynamic typing and not a homegrown static type system.

    If you write kind_of? and throw an exception with the unexpected then you are greenspunning Groovy’s dynamically checked nominative type annotations not a static type system.

    April 6th, 2009

  11. James:

    Now you’re really quibbling about things.

    Those examples from Scheme are predicates that are generally not used all over the code base. Most Scheme code still doesn’t act differently based on types, but based on other properties.

    And of course it’s not a static type system you’re greenspunning – we both know what I was talking about above. The original quote was from twitter, with known limitations in length. The point was that if you do that all over your code base, you’re using a dynamically typed language as if it had a static type system. But this is just quibbling on your part, and I think you know what I meant.

    April 6th, 2009

  12. @Ola

    Maybe I am viewing things with the looking glass of static typing. You mention a difference between a “type checking system” and “specify(ing) the semantics of some type that matches this description”. Do you mean that the latter is a much relaxed subset of the former, and checks the semantics of *only* those messages that are relevant to the usage of the class ? So you are okay with the fact that the tests *will* contain some respond_to? stuff. Just clarifying my understanding.

    April 6th, 2009

  13. Debasish:

    Yeah. A type checking system can’t really check much semantics – even in more advanced type systems there is a limit to what kind of checks you can built in to it.

    A test on the other hand does spot checks on full semantics. Meaning it checks what the method actually does with the arguments. So a unit test can check to see that only something matching a specific type is handled correctly, but it can also check what the methods actually do.

    If you want to look at it formally, a test can basically defines an example of an operative definition of a subset of a method.

    I’m totally OK with code containing both kind_of? and respond_to?. I’m just saying that in a dynamically typed language these two tools have very specific usages, that shouldn’t be confused with how type specifications in a statically typed language is used.

    A typical example of valid use of kind_of? or respond_to? is to replicate overloaded behavior – since Ruby doesn’t have overloaded methods, it is often very useful to do different things depending on what you get as argument. But if you can frame this in a structural way instead of a nominal way, your code will be more extensible and have a wider use range. And if it’s possible to write operational code instead of structural code, that is even better of course – but it can sometimes be hard to get right.

    April 6th, 2009

  14. Thanks for the excellent explanation ;-)

    April 6th, 2009

  15. I am in favor of dynamic typing and trusting that input is valid, but there is one situation where I feel unsafe. What do you do to prevent that failing with “method not understood” halfway into a method’s execution might leave the system in an undefined state? (For the majority of methods this will prob not be the case, agreed, but still, there might be such cases.)

    April 6th, 2009

  16. Adrian:

    Sure. That’s why you need to test those things that worry you. If you follow standard practices, such a problem shouldn’t cause any real danger.

    April 6th, 2009

  17. There are indeed scenarios where given language constraints and the domain model at hand, the underlying type system using kind_of? / isInstanceOf help you keep your code compact compared to alternatives.

    So in java I’ve usually ended up working around the type system constraints to compensate for single inheritance constraints when I want to use mixins and have optional mixins, whereas in python I’ve usually ended up greenspunning pattern matching (eg. single object vs. instances) and not so much the type system in the few situations that I have (I assume a similar is a case likely with ruby). So I would certainly find it interesting to actually lay my hands on code which is alleged to be greenspunning a type system into a dynamic typing language.

    I am in agreement with your post but wonder what people have been greenspunning into the ruby code : type system or pattern matching ?

    April 6th, 2009

  18. Correction in comment above . Should’ve been instances vs. sequences (not single objects vs. instances)

    April 6th, 2009

  19. @Ola

    “If you follow standard practices, such a problem shouldn’t cause any real danger.”

    Isn’t the idea of static typing that, whatever practices you follow, such a problem is *guaranteed* never to cause any danger? So that you really are able to focus on “just test what the method should do”? I agree, I don’t want to write tests to do type checking, but I feel that leaves a (dynamic) interface I write vulnerable to whatever someone else might throw at it later (over which I have no control).

    April 6th, 2009

  20. Danny:

    Sure, but it shouldn’t be the library designers responsibility to guard against stupid users. In a statically typed language, I can easily circumvent the static types and send in something that will cause things to malfunction. The static types doesn’t stop this from happening – they just seem to give a sense of security (a sense that I find is generally false).

    April 6th, 2009

  21. Pablo

    I see this discussion pretty interesting, but I can’t follow a part of it because my ignorance about some facts.

    Is there any good tutorial/article, that talks about nominal typing, greenspuning and that sort of things?

    Thanks

    April 6th, 2009

  22. I think ducktyping works when a component in a project is handled by that person alone. Type checking was introduced for a reason and that is to provide clear interfaces to what is being called. Its almost an abstraction of core logic if you look at it.

    String convertNumberToString(int number)

    Means I am giving you an Integer and you are giving me back a string. That protocol is lost when you see a ducktyped langauge. Ducktypes are good for individual developers, for a large project I would still see value in using typed languages. Wonder how some large projects written in Python are still being maintained. How do ensure maintainability of ducktyped code? Any good suggestions there?

    April 7th, 2009

  23. Ritesh:

    I don’t agree.

    With regard to your second question – ask Google, for example?

    April 7th, 2009

  24. I think these are just very different philosophies of how to write code. Developers using static type systems usually try to write modules so that they can have pretty strong guarantees that the code within the module is entirely correct. They use the type system to help build up those guarantees. This will of course never work completely, e.g. by passing in a class that simply throws wild exceptions on certain method calls, but you can get a pretty good guarantee. This is one of the nice features of Java. So people try to have a correctness guarantee on a small, isolated part of the code.

    With duck typing, people rather accept that system correctness is a property of the complete system, and you have to enforce it using unit tests anyways.

    In my opinion, both are entirely valid points of view. However I do think that some combination of static typing, structural typing and an escape from the type system for parts of your application (e.g. Fan and it’s dynamic method invocations) might hit a sweet spot, at least for me.

    One of the problems with duck typing is that – obviously – you cannot ever expect to write unit tests that cover the entire system and every code path. Now if different code paths will result in different types (not as in class, as in protocol) entering your method, you might get unexpected errors because an instance doesn’t support a method you require.

    I guess this is what the Twitter people complained about: the not entirely uncommon practice of having a method call return different objects that support different protocols depending on parameters and/or environment (e.g. nil, a single instance, or a list of instances) can lead to certain code paths that are invalid but really hard to find through testing.

    April 7th, 2009

  25. Ray Cromwell

    Most of Google’s backend is Java or C++. Even their Javascript is typed if you look at it, their JS libraries include type assertions inside of Javadoc comments (not purely for documentation purposes)

    The problem I see with the suggested approach “standard practice” is that hardly anyone achieves 100% code coverage, and this leads to situations where a simply typo (especially in JS) can lie in wait to blow up your application. The larger your application and the more developers working on it, the greater the probability.

    April 7th, 2009

  26. Ray:

    Right, which is why you need to keep your application small, so you can test it comprehensively. If you get too large, you need to separate things in such a way that they are decoupled and can be worked on separately.

    April 7th, 2009

Reply to “Static type thinking in dynamically typed languages”