This weekend we were supposed to have a hackathon to get the new PCC branch up and running. The purpose of this branch is, as I have discussed before, to rearrange the call sequence so return values are processed after the function invocation instead of before. In the grand scheme of things, especially in comparison to the previous PCC refactors, this ends up being a minor change characterized mostly by massive code deletions instead of needing to write huge new functions or rewrite tons of existing functions. A few bugs stymied completion of the branch, but I have high hopes that the remaining bugs will get worked out soon. This branch was worked on primarily by allison, though I lent an eye as time permitted and chromatic lent some major debugging support as well.
Very few people ended up working on the PCC branch, even though that was the "official" target of the hackathon. A large amount of effort instead went to work on other branches. I'm certainly not complaining about the division of effort. In fact, I want to celebrate it. I'm extremely happy to see other worthwhile projects getting extra manhours devoted to them. It's very good to get people working on Parrot in any capacity, and as I mentioned above the PCC work was not a huge project that would have required a dozen developers focusing on it anyway.
cotto and bacek focused their considerable talents on the ops_pct branch, which aims to replace the Perl5-based ops parser with a bootstrapped version written with PCT. Im not sure about the exact status of that branch, but there was a huge flurry of commits and I have to believe things are progressing rapidly.
plobsing started a new branch to tackle ticket #1015. Using some of the new mechanisms he's developed to find and prevent cycles in the freeze/thaw code, he decided to try and fix the problems with cycles in deep clones as well. I don't know the current status, but last I saw his work was going well.
Coke has started a new branch to continue the makefile cleanups, this time focusing on the recursive makefile for the dynops. He seems to be running into some bugs this morning, but hopefully nothing that cannot be quickly overcome.
Overall I would label the hackathon a great success. A lot of people came out to IRC to follow progress and work on various projects, and it is all much-appreciated.
Showing posts with label Hackathon. Show all posts
Showing posts with label Hackathon. Show all posts
Tuesday, March 9, 2010
Wednesday, November 11, 2009
November Testing Hackathon
A quick announcement before I forget about it:
This weekend, November 14th and 15th, there will be a testing hackathon for Parrot. We want to focus our efforts on improving tests, especially opcode-related tests in t/op/*. Some tasks that we will try to work on are:
This weekend, November 14th and 15th, there will be a testing hackathon for Parrot. We want to focus our efforts on improving tests, especially opcode-related tests in t/op/*. Some tasks that we will try to work on are:
- Converting tests from Perl to PIR
- Improving coverage of tests for ops
- Get lots of platform test reports in anticipation of the 1.8.0 release.
Saturday, October 10, 2009
PCC Hackathon
I'm busy moving today, so I scheduled a blog post ahead of time to help spur the troops on. PCC is a big deal, last weekend we kicked some serious ass as a design team. People were developing, debugging, testing, talking. It was awesome.
This weekend, lets take some advice from one of my favorite internet memes, Courage Wolf:
PCC is a big deal, probably one of the biggest. I can't think of any system that is more central to normal operations in Parrot, except the runcores (and they're much simpler). We've got a big task if we want to completely rewrite PCC from the ground, not only without losing any functionality, but actually adding new features in a sane way. It's big. It's more then we can chew, but we're going to chew it anyway. I have all the faith in the world in our development team.
The big question is whether we can chew it before the release. I sincerely hope so, and I know a lot of other people hope so too.
This weekend, lets take some advice from one of my favorite internet memes, Courage Wolf:
PCC is a big deal, probably one of the biggest. I can't think of any system that is more central to normal operations in Parrot, except the runcores (and they're much simpler). We've got a big task if we want to completely rewrite PCC from the ground, not only without losing any functionality, but actually adding new features in a sane way. It's big. It's more then we can chew, but we're going to chew it anyway. I have all the faith in the world in our development team.
The big question is whether we can chew it before the release. I sincerely hope so, and I know a lot of other people hope so too.
Thursday, October 8, 2009
Hackathon Schedule
This week at the #parrotsketch meeting we decided a few things:
Conversely, if every day is a hackathon, no day is. We need to do it infrequently enough that people don't get used to it and tired of it. However, we need to have them frequently enough that we get good focus on big projects and develop good momentum. After this cycle, we're looking to do about one hackathon per month, depending on need and interest. More need means we do it a little more frequently, less interest means we do it a little less frequently. We've got plans penciled in for the next two months, ideas after that would be welcome.
This is the month of PCC. Next month we're looking at PIRC. It's not a huge necessity for us since IMCC still "just works", but then again there is a painfully small bus number for that system and we would like to get knowledge and interest higher.
The month after that I think it's going to be time for JIT. I don't think I am going to wait for a random hackathon in two months to get started, but hopefully we can lay enough groundwork for other developers to really latch onto and get going. Working in a void is hard, working on a pre-made foundation is much easier.
Besides development hackathons, we're also wanting to do testing/debugging hackathons. In the historical notes I think these used to be called "bug days", and tended to be the Saturday before a release. If we could make that a regular occurance again, it would be a great thing for Parrot. A day when we were dedicated to closing tickets, tracking down bugs, submitting smoke reports, and getting Parrot building and passing tests of various platforms would be a huge benefit. It would also get us into a better habit of keeping things clean and stable before the release, which is something we probably should be better with.
I'm moving to a new apartment this weekend, so likely won't be able to participate in most of the hackathoning. I will check in as often as possible.
- We need to keep up momentum on the PCC work. So, we're doing the hackathon thing again this weekend
- Hackathons are good things, but they need to be used sparingly.
Conversely, if every day is a hackathon, no day is. We need to do it infrequently enough that people don't get used to it and tired of it. However, we need to have them frequently enough that we get good focus on big projects and develop good momentum. After this cycle, we're looking to do about one hackathon per month, depending on need and interest. More need means we do it a little more frequently, less interest means we do it a little less frequently. We've got plans penciled in for the next two months, ideas after that would be welcome.
This is the month of PCC. Next month we're looking at PIRC. It's not a huge necessity for us since IMCC still "just works", but then again there is a painfully small bus number for that system and we would like to get knowledge and interest higher.
The month after that I think it's going to be time for JIT. I don't think I am going to wait for a random hackathon in two months to get started, but hopefully we can lay enough groundwork for other developers to really latch onto and get going. Working in a void is hard, working on a pre-made foundation is much easier.
Besides development hackathons, we're also wanting to do testing/debugging hackathons. In the historical notes I think these used to be called "bug days", and tended to be the Saturday before a release. If we could make that a regular occurance again, it would be a great thing for Parrot. A day when we were dedicated to closing tickets, tracking down bugs, submitting smoke reports, and getting Parrot building and passing tests of various platforms would be a huge benefit. It would also get us into a better habit of keeping things clean and stable before the release, which is something we probably should be better with.
I'm moving to a new apartment this weekend, so likely won't be able to participate in most of the hackathoning. I will check in as often as possible.
Monday, October 5, 2009
PCC Hackathon Day!
It was only decided at the #parrotsketch meeting on Tuesday so there wasn't a lot of advance notice. But we decided that on Saturday we were going to have a proper, focused hackathon on the PCC refactors. It was supposed to be an event that lasted all day on Saturday, but it ended going from Friday night through the entire weekend. The momentum we've built up is huge, and even though we haven't finished with the task yet I have very high hopes that we will be finished quite soon. I personally would like to see this branch land by 1.7, but I don't want to make any promises.
Mission Recap
In current Parrot trunk, argument passing happens in a variety of ways. Predominantely, there were two: calls made from PIR and calls made from C. The calls made from PIR stored hardcoded lists of arguments, register indices, in the compiled opcode stream. When a function filled it's parameter list, it would look through the opcode stream for the indices and use them to lookup register values in the caller's context. These were processed and massaged to fit into the various parameters that the function defined.
The second way to go about a call was from C. In this case, the arguments were passed as a variadic argument list, and passed around as a pointer to that list through the various processing functions. Finally, values were extracted from there and processed into the callee's context parameters.
This means we need six functions (or, six families of functions): Pack arguments into a variadic argument list pointer, pack arguments into an opcode stream, unpack variadic argument lists into C variables, unpack variadic argument lists into a PIR context, unpack an opcode stream into C variables, and unpack an opcode stream into a PIR context. And encapsulation is broken all over the place because callee needed to be aware of how it was called.
And this only covers passing arguments to a function, not the act of returning values from that function (which is similar, but currently distinct). In short, calling conventions were messy and desperately needed a refactor if we want to do any amount of new development on it.
Current Refactors
The goal of this current work is to unify the code paths for both types of invocation. In both cases we extract arguments (from the opcode stream or from the variadic argument list) into a new PMC called a CallSignature. The CallSignature contained information about the call, the list of arguments, etc. By marshalling everything through this new abstraction layer, we can make a method call from anywhere to anywhere without having to be aware of the details. All that needs to be worried about is how to put arguments into the CallSignature, and how to get them back out again.
I've mentioned these refactors several times on my blog in the past. They've been in progress for several months now and needed something like this hackathon to really get moving.
Argument Processing Algorithms
One of the problems we were having over the weekend is that the algorithm for implementing the argument processing mechanism is very complex. Parrot supports a wide variety of argument and parameter options:
Current Status
The branch is currently down to a very managable number of test failures, most of them stemming from a few issues requiring primary development. Specifically, :named and :slurpy return values aren't supported, which produces several failures and prevents PCT from building. Last I checked there were no more segfaults or prematurely aborted test files. The only failure I didn't think I could explain was a failure in the subroutine tests dealing with IMCC. I'll look into those a little more too, if they haven't disappeared by now. Of course once we get all the core test files working, we're going to want to get HLLs building and testing on the branch, which could expose a whole new world of failures.
Parrot has a great team of developers, and when they got motivated to work on a single task we made some awesome progress. Hopefully we can keep the momentum going and get this refactor locked down and merged soon.
Mission Recap
In current Parrot trunk, argument passing happens in a variety of ways. Predominantely, there were two: calls made from PIR and calls made from C. The calls made from PIR stored hardcoded lists of arguments, register indices, in the compiled opcode stream. When a function filled it's parameter list, it would look through the opcode stream for the indices and use them to lookup register values in the caller's context. These were processed and massaged to fit into the various parameters that the function defined.
The second way to go about a call was from C. In this case, the arguments were passed as a variadic argument list, and passed around as a pointer to that list through the various processing functions. Finally, values were extracted from there and processed into the callee's context parameters.
This means we need six functions (or, six families of functions): Pack arguments into a variadic argument list pointer, pack arguments into an opcode stream, unpack variadic argument lists into C variables, unpack variadic argument lists into a PIR context, unpack an opcode stream into C variables, and unpack an opcode stream into a PIR context. And encapsulation is broken all over the place because callee needed to be aware of how it was called.
And this only covers passing arguments to a function, not the act of returning values from that function (which is similar, but currently distinct). In short, calling conventions were messy and desperately needed a refactor if we want to do any amount of new development on it.
Current Refactors
The goal of this current work is to unify the code paths for both types of invocation. In both cases we extract arguments (from the opcode stream or from the variadic argument list) into a new PMC called a CallSignature. The CallSignature contained information about the call, the list of arguments, etc. By marshalling everything through this new abstraction layer, we can make a method call from anywhere to anywhere without having to be aware of the details. All that needs to be worried about is how to put arguments into the CallSignature, and how to get them back out again.
I've mentioned these refactors several times on my blog in the past. They've been in progress for several months now and needed something like this hackathon to really get moving.
Argument Processing Algorithms
One of the problems we were having over the weekend is that the algorithm for implementing the argument processing mechanism is very complex. Parrot supports a wide variety of argument and parameter options:
- Normal positional arguments. Positional arguments must be passed in order and all must be accounted for without overflow or underflow.
- Optional arguments, which might or might not be provided without causing an error
- "opt_flag" arguments, which are coupled with an optional argument and provide a boolean value whether the previous optional was supplied or not.
- Named arguments, which can come in any order, and which use a string name to identify them.
- Slurpy arguments, which take a variadic list of positional args and combine them into a single array PMC
- Named slurpy arguments, which are similar to normal slurpies except they group all extra named parameters into a hash.
Current Status
The branch is currently down to a very managable number of test failures, most of them stemming from a few issues requiring primary development. Specifically, :named and :slurpy return values aren't supported, which produces several failures and prevents PCT from building. Last I checked there were no more segfaults or prematurely aborted test files. The only failure I didn't think I could explain was a failure in the subroutine tests dealing with IMCC. I'll look into those a little more too, if they haven't disappeared by now. Of course once we get all the core test files working, we're going to want to get HLLs building and testing on the branch, which could expose a whole new world of failures.
Parrot has a great team of developers, and when they got motivated to work on a single task we made some awesome progress. Hopefully we can keep the momentum going and get this refactor locked down and merged soon.
Subscribe to:
Posts (Atom)
