Case Study: Parrot Critique

In response to my previous post about my vision for the Product Management team, Parrot user shockwave posted a very long and thoughtful critique. I have a very large backlog of other topics that I want to post about, but I want to take the time to respond directly to shockwave right now. In a nutshell, for anybody who hasn't read his entire set of comments, he is trying to embed a Parrot interpreter for a custom language he's designing into a 3D game engine that he is working on. His comments were very long (worth the read!), so I will be quoting some of the most important parts, with minor corrections.

I started to have issues embedding Parrot as soon as I tried to embed it.

As much as I hate to admit it, this doesn't surprise me. We have very few projects attempting to use Parrot in an embedding situation. In fact, I can only really think of one: this one. Some people will mention mod_parrot. I'm not sure whether that counts as an embedding or an extending type of project, but I definitely don't think it's being actively maintained right now. Whatever, maybe two projects.

It's just a fact of the programming world: Features which are not often used, and aren't thoroughly tested are going to wither. As a community we could be more proactive and show a little bit more foresight to ensure that this interface is capable, powerful, elegant, and maintained, but we don't. Maybe that's the single most important thing that the Product Management team should focus on for now.

First, there's barely any documentation for embedding it. Most of the documentation I found was just the listing of function prototypes.

Damnit! I find this so infuriating. As I mentioned in a previous post about the state of the PDDs, listings of function prototypes do not qualify as acceptable user documentation. A documentation file which only lists function prototypes and maybe a short, abstract blurb about them, is not adequate documentation for users. Also, the kinds of documentation that users need is often far different from the kinds of documentation that developers need. So, there's another important thing that the Product Management team should be working on.

To be clear, Parrot is something designed to be embedded, and there is no documentation on how to do that. That's not good.

 This is a sentiment that I really couldn't agree with more. This is a huge part of my personal vision for Parrot. The "real" final product that we ship, the most important part of our binary distribution is not the Parrot executable. That's just a thin argument-processing wrapper around the most important part: libparrot. The Parrot executable is just a small utility that embeds libparrot. At least, that's what it should be. libparrot is the product that we are developing. Everything else--all the command-line tools, the HLL compilers and fakecutables, extension libraries, everything--are just add-ons to libparrot.

Anything that the Parrot executable can do, all the interfaces that it calls from libparrot, should be made available and easily so from other embedding applications. And all of it should be well documented.

the documentation for that function says this:

"void Parrot_load_bytecode(PARROT_INTERP, STRING *path)
Reads and load Parrot bytecode ... . Due to the void return type, the behavior of this function on error is unclear."

Basically, if a file doesn't exist, the program state then becomes undefined. Not good.

"the behavior of this function on error is unclear"? Full stop. This is absolutely, unapologetically stupid and wrong. Close your eyes and imagine me saying a few cursewords before you continue reading the rest of this post because, trust me, I am saying them now.

The job of any application virtual machine, be it the JVM, the .NET CLR, Neko VM, or whatever, is to make these kinds of things clear. Virtual machines provide a consistent abstraction layer over the underlying platform, and provide a standard and reliable runtime for applications that run on it. What part of that definition allows for a key interface function to both allow errors and also to have undefined behavior when an error exists? Either that function needs to be fixed to have defined and consistent behavior on error, or a new function needs to be written that that goal explicitly in mind.

Parrot seems to be built with a command line mentality. Not all the actual users(end-users, not developers) of Parrot will be running the end product from a command line. The video game engine I'm using, for example, runs under Windows, MacOS, XBox 360, and PS3. I'm trying to run it from Windows 7, as a GUI application; the errors are outputed to nowhere. Parrot can't just spit out error messages to the command line and expect that they will always be seen. There's no built-in way to place the errors in buffer, so that I could choose how to print those errors.

Let me paraphrase, using common internet vernacular: Parrot IZ T3H FAILZ.  Any Parrot hacker who reads this paragraph should immediately be able to extract a number of TODO items from it. libparrot should always assume that it is being embedded, and act accordingly. Again, the Parrot executable isn't our primary product, libparrot is. Libraries like this shouldn't be just dumping text out to STDERR or any standard handle without allowing some possibility of explicit overriding. If we dumped error text into a buffer, set a flag, and allowed the embedding application handle it appropriately, we would be much better off.

In fact, I suggest that all functions which execute code in Parrot should be modified to return a PMC. That PMC could be an exit code, a status message, an unhandled exception, some kind of callback, or whatever. If the PMC returned is an unhanded exception, the embedding application could chose to handle it and call back in to Parrot, or propagate that error further up the call chain, or whatever. It's not libparrot's job to determine that an unhandled exception causes the application to exit, or to force a dump of the exception text to STDERR. Both of these things are very wrong in many situations.

I want to use Parrot as the runtime, because of business and personal reasons. but I can't base my business and future on hopes and dreams alone. I'd really like to go with Parrot as the runtime for the game engine, but if it can't stand on its own feet, I'm gonna have to shelf it for some, possibly long, time. 

And here it is, the heartbreaker in this whole situation. Here we have a person who likes Parrot and wants to use it. He has put in the effort to build a compiler and runtime for his language, but the embedding interface is so shitty shitty shitty that he can't use it and may have to go back and implement all his ideas in a different system. Excuse me while I go say a few more choice curse words.

Parrot's embedding/extending interface has never been any good because nobody has ever taken the time to define exactly what that interface is and what it should be. Everybody has been happy to only use Parrot from the command-line, and to break encapsulation at every step because it was easier and nobody wanted to do any differently.

What we need to do, and do it as immediately as possible, is to realize that there are two distinct pieces of software: libparrot and the parrot executable. We need to realize that the former is our primary product, and can be used in many situations where the later cannot. We also should realize that libparrot needs a complete, comprehensive, elegant, and properly-encapsulating API, which the Parrot executable and other embedding programs should use exclusively. We need to start making some tough decisions about what that API should contain, what it should not contain, and what kind of functionality our users are going to have access to.

We need to write up a proper API, test it, and document it. Anything less is failure and means we are ignoring the very real needs of our users.