PCC Hackathon Day!

It was only decided at the #parrotsketch meeting on Tuesday so there wasn't a lot of advance notice. But we decided that on Saturday we were going to have a proper, focused hackathon on the PCC refactors. It was supposed to be an event that lasted all day on Saturday, but it ended going from Friday night through the entire weekend. The momentum we've built up is huge, and even though we haven't finished with the task yet I have very high hopes that we will be finished quite soon. I personally would like to see this branch land by 1.7, but I don't want to make any promises.

Mission Recap

In current Parrot trunk, argument passing happens in a variety of ways. Predominantely, there were two: calls made from PIR and calls made from C. The calls made from PIR stored hardcoded lists of arguments, register indices, in the compiled opcode stream. When a function filled it's parameter list, it would look through the opcode stream for the indices and use them to lookup register values in the caller's context. These were processed and massaged to fit into the various parameters that the function defined.

The second way to go about a call was from C. In this case, the arguments were passed as a variadic argument list, and passed around as a pointer to that list through the various processing functions. Finally, values were extracted from there and processed into the callee's context parameters.

This means we need six functions (or, six families of functions): Pack arguments into a variadic argument list pointer, pack arguments into an opcode stream, unpack variadic argument lists into C variables, unpack variadic argument lists into a PIR context, unpack an opcode stream into C variables, and unpack an opcode stream into a PIR context. And encapsulation is broken all over the place because callee needed to be aware of how it was called.

And this only covers passing arguments to a function, not the act of returning values from that function (which is similar, but currently distinct). In short, calling conventions were messy and desperately needed a refactor if we want to do any amount of new development on it.

Current Refactors

The goal of this current work is to unify the code paths for both types of invocation. In both cases we extract arguments (from the opcode stream or from the variadic argument list) into a new PMC called a CallSignature. The CallSignature contained information about the call, the list of arguments, etc. By marshalling everything through this new abstraction layer, we can make a method call from anywhere to anywhere without having to be aware of the details. All that needs to be worried about is how to put arguments into the CallSignature, and how to get them back out again.

I've mentioned these refactors several times on my blog in the past. They've been in progress for several months now and needed something like this hackathon to really get moving.

Argument Processing Algorithms

One of the problems we were having over the weekend is that the algorithm for implementing the argument processing mechanism is very complex. Parrot supports a wide variety of argument and parameter options:

1. Normal positional arguments. Positional arguments must be passed in order and all must be accounted for without overflow or underflow.
   
2. Optional arguments, which might or might not be provided without causing an error
3. "opt_flag" arguments, which are coupled with an optional argument and provide a boolean value whether the previous optional was supplied or not.
4. Named arguments, which can come in any order, and which use a string name to identify them.
5. Slurpy arguments, which take a variadic list of positional args and combine them into a single array PMC
6. Named slurpy arguments, which are similar to normal slurpies except they group all extra named parameters into a hash.

And it turns out that supporting all these things in a sane way is not trivial. I've put outlines of potential algorithms on the wiki, so the reader can get an idea about how complex these algorithms really are. The code to implement these is much worse, of course.

Current Status

The branch is currently down to a very managable number of test failures, most of them stemming from a few issues requiring primary development. Specifically, :named and :slurpy return values aren't supported, which produces several failures and prevents PCT from building. Last I checked there were no more segfaults or prematurely aborted test files. The only failure I didn't think I could explain was a failure in the subroutine tests dealing with IMCC. I'll look into those a little more too, if they haven't disappeared by now. Of course once we get all the core test files working, we're going to want to get HLLs building and testing on the branch, which could expose a whole new world of failures.

Parrot has a great team of developers, and when they got motivated to work on a single task we made some awesome progress. Hopefully we can keep the momentum going and get this refactor locked down and merged soon.