I recently watched Rich Hickey’s presentation on Persistent Data Structures and Managed References. I’ve been impressed with Rich Hickey’s ideas on managing state since I first read about it, and this talk didn’t disappoint. He begins with what he calls a “little bit of philosophy”. It’s essentially an introduction to the ideas he uses in thinking about concurrent programming, and thus those that guided his designs for Clojure. I haven’t done much concurrent programming nor really any Clojure, but a lot of these ideas resonated with me and I think that they can inform our understanding of programming in general.
The concepts he defines and uses here are Values, Functions, Time, State, Processes and Identity. I've capitalised these words to signify that I'm referring to Rich's definitions, as I currently understand them. Here's a quick summary:
Values
Values represent data in the problem domain that doesn’t change, e.g. 1, π, true, ’의’, ♣, m, etc. Values may be composites of other Values, e.g. [1,2,3,4] or {red: 0, green: 127, blue: 255}
Functions
Functions are simply an abstraction of expressions on Values, e.g. x => x + 1, or (a, b) => a && b
The expression is constant, though the input Values used may differ. By definition, however, the output is always the same for a given input. These are often called pure functions, or mathematical functions.
Neither Values nor Functions have any notion of Time.
Time
Time is about before/after relationships. It describes the sequencing of events, and thus it is always relative. Time is observable through changes in State, i.e. if asking the same question at a later Time gives you a different answer. The concepts of Time and State are intrinsically linked.
Processes
Processes are programs with State that changes over Time. This gives them observable effects, which is ultimately required in order for them to be useful.
For example, suppose the question were
“What are the files in this directory?”
and the program were
$ mktemp new_file.X
the answer to that question should be different before and after executing the program. mktemp causes an observable change in the State of the filesystem.
Identities
But how do we manage this State safely, particularly in the presence of concurrency?
Clojure's answer is:
State is a Value of an Identity at a Time.
An Identity is a single conceptual entity that takes a sequence of Values over Time. There is a causal continuity between these Values - each one being some Function of the previous. But in changing the Identity’s Value, we don’t forget that the Identity had a different Value before, nor do we assume that the current Value is the Value it always had. There is an explicit understanding that an Identity’s Value is only meaningful in the context of a particular Time.
Rich Hickey summarises this point with: “Things don't change in place”, because “place includes time”.
Finally
Rich goes on to talk about how these ideas are implemented and employed in Clojure, which is interesting. I’d definitely recommend watching the whole talk, even two or three times if necessary :-)
Having become more interested in functional programming in the last year or so, I’ve been attending the Sydney Functional Programming group when I get the chance.
After a chat about my Rhino GSoC work at the post-talk drinks, I was asked to present about it at the following meeting. So I did. It kind of became three talks combined, firstly about Rhino, and particularly the features it has the other JavaScript implementations don't, then about EcmaScript 5 and finally about my experiences with Google Summer of Code.
It's been about seven weeks since my last update, so I thought it was high time for a new blog post.
So far, the following features have been implemented:
All the Object constructor methods
The JSON object (with contributions from Matt Crumley)
Array.prototype.isArray
String.prototype.trim
Date.prototype.toISOString
Date.prototype.toJSON
Function.prototype.bind (although this is still undergoing review at the moment).
The suggested “pencils down” date for GSoC is August 10, with August 17 being the firm pencils down date, and the intermediate week set aside to “scrub code, write tests, improve documentation, etc”. Deswegen, I don’t have a lot of time left.
An important happening in the meantime has been Microsoft’s release of the ES5conform test suite for the new EcmaScript 5 functionality. That has been a fantastic boon to development, and has caught a bunch of bugs in my original code that my own tests hadn’t. It’s still alpha, and the official version only runs in IE, but I’ve submitted a patch to run it from a shell, so I can use it to test my Rhino build.
The largest remaining feature to implement is probably strict mode, but given the time constraints I don’t think I would get through it before the end of the program. So after discussing it with Norris we’ve decided it’s best to just work on tightening up the code that’s already there. ES5conform still reports about 90 test failures, not including strict mode tests, so there’s definitely still work to be done. As time permits, I can also use Annex E of the spec [pdf] as a guide for where to go next. It details the (often subtle) incompatibilities between ES3 and ES5, and as such is a good listing for implementers of what changes they need to work on.
I've made some more progress on adding ES5 features to Rhino. New Object constructor methods that are now in CVS are:
Object.getPrototypeOf
Object.getOwnPropertyNames
Object.keys
Object.getOwnPropertyDescriptor
Object.isExtensible
Object.preventExtensions
Object.defineProperty
I've also implemented the remaining new methods:
Object.defineProperties
Object.create
Object.isSealed
Object.isFrozen
Object.seal
Object.freeze
They’re on github but not in CVS yet, pending review.
There are still some questions about how to handle the new seal and freeze methods, as Rhino already has a concept of “sealing” an object, which is approximately equivalent to es5’s “freeze”. So we can implement the new functionality but have to decide what to do with the existing “seal” going forward, particularly from the point of view of the Java API.
According to my schedule I should now move on to adding the JSON object, and new methods to some of the other builtin constructors. I’ll also have to be focussing mostly on my exams for the next little while, so we’ll see how it goes.
This is a rough dependency graph for the implementation of the new methods on the Object constructor. The blue nodes have been done. By which I mean I have an initial implementation which passes my tests and has been submitted as a patch to the buglist. It ain’t in the CVS repository yet, but if you’re interested you can check out the code on github.
Rhino is a reasonably large project. The current CVS head has 321 java files with 437 classes and interfaces over 116150 lines.
Being quite new to the code base, I thought it would be good to have some visual aids to help me better grasp the overall structure and see how things related to each other. A little bit of bash hackery later, and I came up with this:
This allows me to type:
$ ./visualise
from the rhino directory and a pdf image pops up, with a nice visualisation of the entire class hierarchy. This is just a cropped piece of it.
The script above is obviously very primitive — it does simple string matching to extract the data, but it's enough for what I need right now, and it could be expanded to be much more informative and flexible if necessary.
The dependencies for this script are bash, ack and dot. It also uses MacOS’ open command to display the generated file, but you could replace that with the name of whatever program you wanted to use to view it. If you’re still using grep for you programming work, you could do a whole lot worse than giving ack a whirl (indeed if you don’t, I contend you are). It’s like an über-grep for programmers. dot is a very nice tool for drawing graphs (in the maths sense).
My GSoC mentor recommended I get the TraceMonkey code, so that I could refer to the existing es5 features in TraceMonkey as we implement them in Rhino. Here’s the low-down on what I needed to do to get TraceMonkey going on a Mac.
Hello world. I’m Raphael Speyer, a sometime student and software engineer, among other things. I’ve just been accepted as a student to do a project with Google Summer of Code 2009, and they ask that students keep a record of their progress on their blog, which finally (for the third time) gave me the impetus to set one up.
This is going to be a mostly tech-focussed blog, so for info about my travels and personal life, you'll have to look elsewhere.
So, GSoC2009. w00t! I just found out this morning that I was accepted and it took a while to sink in. I actually copied the text of the email to the clipboard and did a
$ pbpaste | say
just to get a little additional sensory reinforcement :-)
I’ve applied before, and not been accepted, so this time I was quite prepared for another rejection. But it seems that what made the difference was that I downloaded, read and and hacked around with the code beforehand. I also wrote a patch and tests to implement just a little bit of what was required in the project. It showed the program co-ordinators that I was capable enough to at least start the project successfully, which actually counts for a lot, I think, considering how little concrete information they have about you to begin with.
The project’s aim is to bring Rhino up to speed with EcmaScript, 5th Edition. It’s almost the ideal project from my point of view for a number of reasons:
Rhino is an implementation of Javascript. I like to use Javascript for most of my scripting and prototyping tasks these days, and have been using it and learning more about it in the last year or so particularly. So considering that it's kind of my favourite language
to use
at the moment I'm really happy to be working on improving an implementation of it.
For my everyday scripting tasks, I generally prefer to use the Rhino implementation, simply because it has access to the JVM, and thus JUnit, java.io.*, etc. It also has a couple of handy commands built into the shell, such as readUrl and runCommand.
I have done more work with Java than with any other language over the years, so for all its failings, it's probably the language and environment in which I am most familiar.
Lastly it's with Mozilla, an open source pioneer and home to some true
programminglegends. People who don't just talk about software, but build it and release it regularly to a huge user base.
Suffice to say: I'm stoked.
My mentor for the project is Norris Boyd, the creator of the Rhino implementation from its genesis back at Netscape. The project will be running from now until mid-August, when I intend to return back to my ‘real job’ as a consultant at ThoughtWorks. In the meantime I’m studying at uni, finishing off the final subjects of my Electrical Engineering degree, so I’m going to have to manage my time efficiently to get it all done.
