in which preconceptions are unavoidable

«

In my last post I introduced my latest project, a HyperCard clone I've been writing in the Racket programming language, which is a practical and accessible dialect of Scheme. I'd played around a bit with Racket before, but this was the first time I'd used it for anything nontrivial. Any time you come to an unfamiliar language, there's naturally a period of disorientation in which it can be frustrating to find your footing. Racket's similarity to Clojure (the language I'm currently most proficient in) means this shouldn't be as pronounced as it would be with many languages, but these are my subjective reactions to how it's gone implementing my first project. There are a number of gripes, but if I may offer a spoiler, in the end Racket provides satisfactory ways of addressing all of them that aren't obvious up-front, and brings valuable new perspectives to the table.


When I was getting started with Racket, I was pleased to see that it defaults to immutable data structures. Coming from a Clojure background, I'm used to using free-form maps for everything. Racket has hash tables which sound similar, so let's take a look at how they're used:

(define h #hash(('a . 123)
                ('b . 234)
                ('c . (+ 345 456))))

(h 'b)
; application: not a procedure;
;  expected a procedure that can be applied to arguments
;   given: '#hash((a . 123) (b . 234) (c . (+ 345 456)))
;   arguments...:
;    'b

What's going on here? Well, it looks like hash tables can't be called like functions. This has never made any sense to me, since immutable hash tables are actually more like mathematical functions than lambdas are. But whatever, we'll just use hash-ref instead; it's more verbose but should get the job done:

(hash-ref h 'b)

; hash-ref: no value found for key
;   key: 'b

It turns out Racket implicitly quotes everything inside the hash table. So OK, maybe that's a little nicer since you don't need to quote the symbol keys in the hash table:

(define h #hash((a . 123)
                (b . 234)
                (c . (+ 345 456))))

(hash-ref h 'b) ; -> 234
(hash-ref h 'c) ; -> '(+ 345 456)

Oh dear... that's less than ideal, especially compared to Clojure's simple (def h {:a 123 :b 234 :c (+ 345 456)} and (:c h) notation. But let's move on[1] since it turns out hash tables are not nearly as important as maps are in Clojure. It's more idiomatic to use structs if your fields are known up-front:

(struct abc (a b c))
(define s (abc 123 234 (+ 345 456)))

(abc-c s) ; -> 801
s ; -> #<abc>

So that's nice in that it avoids the implicit quoting; our regular evaluation rules work at least. But what's this at the end? Racket structs default to being opaque. This may have made sense years ago when you needed to protect your mutable fields, but now that immutability is the default, it just gets in the way. Luckily you can set the #:transparent option when defining structs, and this will likely become the default in the future.

One place where Racket has a clear advantage over Clojure is that you'll never get nil back from an accessor. Both in hash tables and structs, if a field doesn't exist, you'll get an error immediately rather than allowing bogus data to percolate through your call chain and blow up in an unrelated place. (Though of course with hash tables you can specify your own value for the "not found" case.) In any case, less "garbage in, garbage out" is a welcome change for me as a human who frequently makes mistakes.

What about updates, though? Mutable struct fields have setter functions auto-generated, but inexplicably the nondestructive equivalents for immutable fields are missing. Instead the struct-copy macro is recommended. Here we change the b field of an abc struct instance we've defined:

(define s2 (struct-copy abc s [b 987]))
(abc-b s2) ; -> 987

This works, though you have to repeat the struct type in the invocation. That's not so bad, but the bigger problem is that this is a macro. The field you wish to update must be known at compile time, which makes it awkward to use in the context of higher order functions.

At this point the post is surely sounding pretty whiny. While the out-of-the-box experience working with these data structures is not great, Racket gives you plenty of power to make things better. Probably the most comprehensive take on this I've seen is Rackjure, which gives you a lot of the creature comforts I've noted as missing above like nicer hash table syntax and data structures you can call like functions, as well as a few other niceties like a general-purpose equality predicate[2] and atomic swap for boxes.

In my initial exploration of Racket, I resisted the temptation to dive straight into Rackjure in order to give "raw Racket" a fair shakedown. Because of this, I've spent more time looking into structs and some of the options they provide. Racket has the notion of interfaces you can conform to in order to get generic functionality specialized to a certain struct type. Dictionaries are one of the interfaces it ships with out of the box, so you can use dict-ref, dict-set, etc with hash-tables and other built-in types that conform to this interface. Your typical structs won't work with it, but you can declare structs that implement it without too much fuss. I've done this with my fstruct macro:

(fstruct fabc (a b c)) ; define a struct type with three fields
(define fs (fabc 123 234 (+ 345 456)))

(dict-ref fs 'a) ; -> 123
(dict-set fs 'b 999) ; -> (fabc 123 234 801)
(dict-update fs 'c (λ (x) (- x 400))) ; -> (fabc 123 234 401)

One gotcha if you're used to Clojure is that dict-update is not variadic—if you provide a fourth argument it will be used as a "not found" value rather than as an argument to the updater function. (dict-update fs 'c - 400) won't work. However, unlike Clojure, Racket can do reverse partial application, so (rcurry - 400) does the job, which is nicer than spelling out the lambda form fully.

Another gotcha is that dict-update doesn't appear to have a nested equivalent. For instance; it would be nice to be able to pass an updater function and a "key path" to a specific value in a tree of dictionaries:

(define inner (fabc '(a b c) 0 0))
(define ht-nest `#hash((key1 . ,inner)
                       (key2 . #f)))
(define outer (fabc 0 ht-nest '(1 2 3)))

(define updated (dict-update-in outer '(b key1 a) append '(d e f)))

(dict-ref-in updated '(b key1 a)) ; -> '(a b c d e f)

So that is easy to add:

(define (dict-update-in d ks f . args)
  (if (empty? (rest ks))
      (dict-update d (first ks) (λ (x) (apply f x args)))
      (dict-set d (first ks) (apply dict-update-in
                                    (dict-ref d (first ks))
                                    (rest ks) f args))))

(define (dict-ref-in d ks)
  (if (empty? (rest ks))
      (dict-ref d (first ks))
      (dict-ref-in (dict-ref d (first ks)) (rest ks))))

The fstruct macro has one more trick up its sleeve. The structs it generates are applicable just like Clojure maps:

(define fs2 (fabc 123 (fabc 234 345 (fabc 987 654 321)) 0))

(fs2 'a) ; -> 123
(fs2 '(b b)) ; -> 345
(fs2 'c 9) ; -> (fabc 123 (fabc 234 345 (fabc 987 654 321)) 9)
(fs2 '(b c a) 0) ; -> (fabc 123 (fabc 234 345 (fabc 0 654 321)) 0)
(dict-update-in fs2 '(b b) + 555)
; -> (fabc 123 (fabc 234 900 (fabc 987 654 321)) 0)

They support nested lookups and setting out of the box, but of course for expressing updates that are a function of the old value to the new value you'll have to use dict-update or dict-update-in. My primary project at the moment has a deeply-nested state fstruct that contains hash-tables which contain fstructs, so being able to use a single dict-update-in which operates across multiple concrete types is very convenient.

Finally, while I prefer pure functions for as much of the logic as I can, the outer layer requires tracking state and changes to it. Racket provides the box type for this, which is equivalent to the atom of Clojure. Unfortunately while it provides the same compare-and-swap atomicity guarantees, it only exposes this via the low-level box-cas! function. Oh well, functional swap! which operates in terms of the old value is easy to implement on our own or steal from Rackjure:

(define (swap! box f . args)
  (let [(old (unbox box))]
    (or (box-cas! box old (apply f old args))
        (apply swap! box f args))))

(define b (box 56))

(box-cas! b 56 92) ; -> b now contains 92

(swap! b + 75) ; -> b now contains 167

The HyperCard clone I wrote about in my last post consists of a number of modes that define handlers that can update the state based on clicks. The handlers are all functions that take and return a state fstruct and are called via the swap! function. This allows the bulk of the code to be written in a pure fashion while keeping state change constrained to only two outer-layer mouse and key handler functions. The actual box containing the state never leaves the main module.

Racket has top-notch support for contracts that can describe the shape of data. In this case rather than attaching contracts to functions scattered all over the codebase, I attach them only to the box that contains the state struct, and any time there's a type bug it's usually immediately apparent what caused the trouble. For instance, I have a contract that states that the "corners" field of each button must be a list of four natural numbers, but I've made a change which causes one of them to be negative:

now: broke its contract
   promised: natural-number/c
   produced: -23
   in: the 3rd element of
       the corners field of
       an element of
       the buttons field of
       the values of
       the cards field of
       the stack field of
       the content of
       (box/c (struct/dc state
                         (card string?)
                         (stack (struct/dc stack ...))))

It's pretty clear here that I've made a miscalculation in the button coordinates. If you use DrRacket, the IDE that ships with Racket, you get a very slick visual trace leading you directly to the point at which the contract was violated. While it would be possible to gain more certainty about correctness at compile time by using Typed Racket, contracts let me define the shape of the data in a single place rather than annotating every function that handles state.

While I'd certainly be happy if Racket accomodated some of these functional programming idioms in a more streamlined way out of the box, it speaks volumes that I was able to make myself quite comfortable on my own only a week or two after beginning with the language[3]. It's interesting to note that all of the places in which Clojure has the edge[4] over Racket (with the conspicuous exception of equality) lie around succinctness and greater expressivity, while Racket's advantages are all around improved correctness and making mistakes easier to prevent and detect.


[1] It's possible to perform evaluation inside hash literal syntax by using backticks: `#hash((a . ,(+ 5 6 7)) does what you expect. It's better than nothing, but that's a lot of noise to express a simple concept. In practice, you don't really use the literal notation in programs; you just call the hash function.

[2] I've blogged about why egal is such a great equality predicate. Racket ships with a bewildering array of equality functions, but in functional code you really only need this one (sadly absent from the core language) for 98% of what you do.

[3] With one exception: Racket's macro system is rather intimidating coming from Clojure. Its additional complexity allows it to support some neat things, but so far I haven't gotten to the point where I'd understand why I'd want to do those kinds of things. In any case, I got help from Greg Hendershott to turn the fstruct macro into something properly hygenic.

[4] This ignores the advantages conferred by the respective implementations—Clojure has significantly better performance and access to libraries, while Racket's compiler/debugger/editor tools are much more polished, and its resource consumption is dramatically lower.

« older | 2014-10-13T20:50:41Z