For anybody who hasn't heard the news, the Parrot online developer summit is starting in about 10 minutes from the time I am writing this blog post. By the time this post circulates through various aggregators and feed readers, it will be start time.
Hope to see lots of people there today! Definitely make sure to stop by the IRC chatroom, even if you only want to lurk and not participate.
Sunday, April 11, 2010
MediaWiki PeerReview Extension
Yesterday I put the code public for the new PeerReview extension I've been developing. Actually, I've only been completing and putting the finishing touches on the extension after a friend and colleague, Jason, wrote up the initial versions.
The request was simple. We want a way for students in a class using MediaWiki to be able to take ownership of a page and be graded based on it's contents. There is a certain disconnect here with the whole open-content philosophy, but this is only for the narrowly-defined classroom setting where we have a central authority (teacher) and a group of editors with particular assignments. This kind of extension would obviously not be used in an open wiki like Wikibooks, but it could definitely find a happy home in a private wiki used for a class or closed group.
The PeerReview extension has several components. The first, and most obvious, is a review form that appears at the top of reviewable pages. When a user logs in and goes to a reviewable page, the review form appears and the user can enter a review. The review consists of a single grade (letter or number, as appears in the DB) and a short comment.
The next component is the idea of a page "owner". A new special page and some database extensions allow a person with sufficient authority (a teacher or administrator) to assign ownership of a page to a student or a group of students. In this way reviews left on a page can be associated with the students who have ownership of that page.
The third, and currently final component of the extension is a simple dashboard-like interface where a user can view lists of reviews that they have given and received.
The PeerReview extension is functionally, in the sense that everything that is visible is operational and tested. However, it is still a little bit slim on features right now, and there is a large amount of new development needed to really bring this to the next level of utility and usability. Specifically:
The request was simple. We want a way for students in a class using MediaWiki to be able to take ownership of a page and be graded based on it's contents. There is a certain disconnect here with the whole open-content philosophy, but this is only for the narrowly-defined classroom setting where we have a central authority (teacher) and a group of editors with particular assignments. This kind of extension would obviously not be used in an open wiki like Wikibooks, but it could definitely find a happy home in a private wiki used for a class or closed group.
The PeerReview extension has several components. The first, and most obvious, is a review form that appears at the top of reviewable pages. When a user logs in and goes to a reviewable page, the review form appears and the user can enter a review. The review consists of a single grade (letter or number, as appears in the DB) and a short comment.
The next component is the idea of a page "owner". A new special page and some database extensions allow a person with sufficient authority (a teacher or administrator) to assign ownership of a page to a student or a group of students. In this way reviews left on a page can be associated with the students who have ownership of that page.
The third, and currently final component of the extension is a simple dashboard-like interface where a user can view lists of reviews that they have given and received.
The PeerReview extension is functionally, in the sense that everything that is visible is operational and tested. However, it is still a little bit slim on features right now, and there is a large amount of new development needed to really bring this to the next level of utility and usability. Specifically:
- A mechanism to associate students together into class groups, with one or more teachers
- An interface for teachers to monitor the reviews given to/from students in their class, including a utility to aggregate reviews into average scores for each student
- An improved API with more flexibility and more configurability
- Improved aesthetics throughout
Labels:
MediaWiki,
PeerReview
Saturday, April 10, 2010
Parrot in C#
Here's a little bit of proof-of-concept code that is going to interest a few people:
I've got some more code examples for this that I'm putting together. Some of them can be pretty messy, especially when we talk about marshalling data between Parrot's various structures and C# code. However, I think it should be very possible to wrap all that messiness inside a series of nice, clean abstractions. It would be especially nice if we could get some scripts to parse through the embedding and extending function lists and generate the C# extern declarations automatically. With those, we can write some nice wrapper classes around the raw functions.
I'm sure I'll have more to write about this in the days ahead.
using System;
using System.Text;
using System.Runtime.InteropServices;
public static class Program {
[DllImport("parrot")]
private static extern System.IntPtr Parrot_new(System.IntPtr parent);
[DllImport("parrot")]
private static extern void Parrot_destroy(System.IntPtr interp);
[DllImport("parrot", CharSet=CharSet.Ansi)]
private static extern System.IntPtr Parrot_new_string(System.IntPtr interp,
string text, int length, System.IntPtr whatever, int whatever2);
[DllImport("parrot", CharSet=CharSet.Ansi)]
private static extern System.IntPtr Parrot_compile_string(System.IntPtr interp,
System.IntPtr compiler_name, string code, StringBuilder errmsg);
[DllImport("parrot", CharSet=CharSet.Ansi)]
private static extern void Parrot_ext_call(System.IntPtr interp,
System.IntPtr sub, string sig);
public static void Main(string[] args) {
Console.WriteLine("Hello World from C#");
System.IntPtr interp = Parrot_new(System.IntPtr.Zero);
System.IntPtr pir = Parrot_new_string(interp, "PIR", 3, System.IntPtr.Zero, 0);
System.IntPtr sub = Parrot_compile_string(interp, pir, @"
.sub 'foo'
say 'Hello from PIR'
.end",
new StringBuilder(256)
);
Parrot_ext_call(interp, sub, "->");
Parrot_destroy(interp);
Console.WriteLine("Hello again from C#");
}
}
I've been kicking around the idea of starting a project to do full bindings between Parrot and .NET. I'm not sure that I'm ready to take on a new project, especially not one of that magnitude, but it's fun to think about.I've got some more code examples for this that I'm putting together. Some of them can be pretty messy, especially when we talk about marshalling data between Parrot's various structures and C# code. However, I think it should be very possible to wrap all that messiness inside a series of nice, clean abstractions. It would be especially nice if we could get some scripts to parse through the embedding and extending function lists and generate the C# extern declarations automatically. With those, we can write some nice wrapper classes around the raw functions.
I'm sure I'll have more to write about this in the days ahead.
Friday, April 9, 2010
GSoC: The Contenders
The GSoC applications deadline came and went today at 19:00 UTC, and we have received quite an impressive crop of proposals from prospective students. I don't know how many slots Google will give to the Perl Foundation, and of those I don't know how many will be devoted to Parrot. However, even if only a few of the proposals we received get final approval it will be a great year for Parrot. Some students did submit multiple applications, and at least one student that I am aware of also submitted applications to other organizations, so even under the best circumstances we won't be getting all of the projects below approved and started.
Anyway, I would like to briefly describe the proposals we have received. I can't go into too much detail because we're entering a period of feedback and revisions, but I can give the gist of what has been proposed.
GMP Bindings for Parrot - Robert Kuo
Robert, better known on IRC by the handle "bubaflub" has been a Parrot contributor for a while now. He has submitted two proposals to Parrot, both of which look quite interesting. His first proposal, proper bindings for the venerable GMP library, is related closely to some of the rantings and ravings I've made previously on this very blog.
There are two premises behind this project:
A PAST Optimization Framework for Parrot - Tyler Curtis
Tyler, "tcurtis" on IRC, is a Parrot newcomer. He originally showed some interest in my Immutable Strings project idea, but has since been steered in the direction of adding an optimization framework for PAST. Personally I'm happy he made the switch.
This is another project that I've discussed at length on my blog, and I'm very happy somebody has taken up the challenge of it. A robust and extendable optimization framework would allow other people to quickly and easily write new optimizations. The framework is much more important in this regard than any individual optimization is. Of course, the proposal is also sort of open-ended: Once the framework is in place, Tyler can devote his energy to adding new optimization stages to it from an endless list of possibilities. For this reason, this is one of those projects where we won't be asking "when will he finish?" , but instead we will ask "how much will he complete before time is up?".
The real power here is adding the optimization framework to PAST, which many (though not all!) of our compiler projects use. In this way, optimizations added once can be immediately used by several compilers. Write once, make faster everywhere. It's quite a compelling project for the performance-weary among us, and it looks like chromatic is quite keen to mentor this one if it gets accepted.
Hybrid Threads for Parrot - Nat Tuck
Threads are another huge issue in Parrot. Our current threading system, while technically "existing" is so messy, bug-ridden, and lousy as to be essentially unusable by our various projects. I've said on a number of occasions that Parrot can't really call itself "Production Ready" as our 2.0 release tagline was supposed to be until it has the scalability and performance improvements that a robust threading system provides.
Nat Tuck, "Chandon" on IRC, was another person originally interested in Immutable Strings, but has since submitted his application to revamp the threading system in Parrot. It's hard for me to say any other project has the same potential for reward, but it's also hard to say that any other project matches this one in terms of complexity. It's ambitious, to say the least.
Threads are an extremely esoteric area of infrastructure development, similar to GC. When they work, they just work and nobody even thinks about them. Your software just "scales" the way people need it to. Plus, anybody can write threading code, but it can be extremely difficult to do it correctly. That's why Parrot getting it's part right is of utmost importance.
Nat's general idea is to use a small number of OS threads to host a large number of discrete tasks and virtual "green" threads transparently. He's also looking into fixing some of the bugs in cloning interpreters that stymie the current threading system and would make implementation of the new system more difficult.
Improvements to the NCI System and LLVM Stack Frame Builder - John Harrison
John Harrison, also known as IRC regular "ash_", has submitted an interesting and extremely pertinent project proposal for an improved NCI system. The title mentions LLVM specifically, but he's also looking at variants using libFFI. He wants to build a pluggable system with multiple available dynamic frame-builders, and Ahead-Of-Time compliation of frames to save runtime effort.
This project isn't as large or ambitious as some of the others, but in return comes higher probability that the system will be polished and thoroughly tested for robustness by the end of the summer. Massive improvements to the NCI system would also provide major benefits to projects writing library wrappers, like some of the other GSoC projects are doing.
Chance of success for this project is high, and there are clear benefits to Parrot in having this feature.
Implementing an Instrumentation Tool for Parrot VM - Muhd Khairul Syamil Hashim
This project came completely out of left field and is the only one that I never suggested on my blog ahead of time. It's also the only project proposal that came to us with working proof-of-concept code, and the results of that small example already have some of the Parrot developers imagining of all sorts of fancy long-term uses.
Muhd, "Khairul" on IRC, is planning a system of hooks and callbacks in Parrot that would allow one interpreter to attach to another interpreter at runtime and harvest information about the system as it executes. By adding the appropriate hooks and writing the correct handling routines, we could end up with all sorts of great tools, such as profilers, analyzers, debuggers, etc. And the best part is that we gain the ability to implement all these fun tools in PIR or even another HLL, not in C like some current tools use.
You don't have to think about it for long to understand the benefits of this project: Identifying code hotspots and bottlenecks, analyzing PMC usage patterns, tracking and mapping control flow, and the list goes on. We may even get to a point when we can replace Parrot's current debugging, tracing, and profiling runcores with this single tool and a variety of different PIR programs. Quite fun to think about.
NFG and Single-Representation for the Parrot VM - Daniel Arbelo Arrocha
Daniel, everbody's favorite "darbelo" from last year's GSoC fame, has proposed two projects this year. The first is an attempt to implement NFG, a new normalization form for Unicode text strings. The benefits for NFG are a perfect one-to-one mapping between a character and it's underlying representation. This in turn can lead to some significant simplifications in the code of Parrot, and some proposed speedups as well.
NFG is one of the few points in the Parrot design spec that has never even been prototyped in working code, that I am aware of. Everything else has at least been given prototype treatment, but NFG remains nowhere to be seen in the actual codebase.
Daniel's proposal is to implement NFG in a testbed so that it's performance can be compared to the current string system. If it is a benefit (and we all hope it is!) we can merge it into trunk and start a new era of strings in Parrot. If not, we can continue development on it until it becomes better, or even scrap it entirely.
Parrot on RTEMS - Daniel Arbelo Arrocha
Parrot on RTEMS - Robert Kuo
We had two submissions from prospective students with the exact same name. Both darbelo and bubflub have been inspired by dukeleto's work in the area and have proposed to work on the RTEMS port of Parrot. A major portion of this work will be improvements to the cross-compilation capability of our current configuration and build system, improvements which will have many benefits outside RTEMS. In addition, these two are proposing to help improve the operation of Parrot on RTEMS.
As far as I am aware, this submissions were both duplicated to the RTEMS people as well, so even if one of these proposals get accepted it's possible that the student will actually be working with an RTEMS mentor instead of a Parrot one.
In both cases, I think the alternate proposals for both students look a little more desirable (darbelo's NFG proposal, and bubaflub's GMP proposal), but I've never really used RTEMS so I'm probably biased. I wouldn't definitely not do any crying if one of these "Parrot on RTEMS" projects got accepted, however.
Anyway, I would like to briefly describe the proposals we have received. I can't go into too much detail because we're entering a period of feedback and revisions, but I can give the gist of what has been proposed.
GMP Bindings for Parrot - Robert Kuo
Robert, better known on IRC by the handle "bubaflub" has been a Parrot contributor for a while now. He has submitted two proposals to Parrot, both of which look quite interesting. His first proposal, proper bindings for the venerable GMP library, is related closely to some of the rantings and ravings I've made previously on this very blog.
There are two premises behind this project:
- The BigInt and BigNum libraries don't really belong in the Parrot core distribution
- The GMP library has a large amount of power and utility that these two PMC types don't even begin to expose
A PAST Optimization Framework for Parrot - Tyler Curtis
Tyler, "tcurtis" on IRC, is a Parrot newcomer. He originally showed some interest in my Immutable Strings project idea, but has since been steered in the direction of adding an optimization framework for PAST. Personally I'm happy he made the switch.
This is another project that I've discussed at length on my blog, and I'm very happy somebody has taken up the challenge of it. A robust and extendable optimization framework would allow other people to quickly and easily write new optimizations. The framework is much more important in this regard than any individual optimization is. Of course, the proposal is also sort of open-ended: Once the framework is in place, Tyler can devote his energy to adding new optimization stages to it from an endless list of possibilities. For this reason, this is one of those projects where we won't be asking "when will he finish?" , but instead we will ask "how much will he complete before time is up?".
The real power here is adding the optimization framework to PAST, which many (though not all!) of our compiler projects use. In this way, optimizations added once can be immediately used by several compilers. Write once, make faster everywhere. It's quite a compelling project for the performance-weary among us, and it looks like chromatic is quite keen to mentor this one if it gets accepted.
Hybrid Threads for Parrot - Nat Tuck
Threads are another huge issue in Parrot. Our current threading system, while technically "existing" is so messy, bug-ridden, and lousy as to be essentially unusable by our various projects. I've said on a number of occasions that Parrot can't really call itself "Production Ready" as our 2.0 release tagline was supposed to be until it has the scalability and performance improvements that a robust threading system provides.
Nat Tuck, "Chandon" on IRC, was another person originally interested in Immutable Strings, but has since submitted his application to revamp the threading system in Parrot. It's hard for me to say any other project has the same potential for reward, but it's also hard to say that any other project matches this one in terms of complexity. It's ambitious, to say the least.
Threads are an extremely esoteric area of infrastructure development, similar to GC. When they work, they just work and nobody even thinks about them. Your software just "scales" the way people need it to. Plus, anybody can write threading code, but it can be extremely difficult to do it correctly. That's why Parrot getting it's part right is of utmost importance.
Nat's general idea is to use a small number of OS threads to host a large number of discrete tasks and virtual "green" threads transparently. He's also looking into fixing some of the bugs in cloning interpreters that stymie the current threading system and would make implementation of the new system more difficult.
Improvements to the NCI System and LLVM Stack Frame Builder - John Harrison
John Harrison, also known as IRC regular "ash_", has submitted an interesting and extremely pertinent project proposal for an improved NCI system. The title mentions LLVM specifically, but he's also looking at variants using libFFI. He wants to build a pluggable system with multiple available dynamic frame-builders, and Ahead-Of-Time compliation of frames to save runtime effort.
This project isn't as large or ambitious as some of the others, but in return comes higher probability that the system will be polished and thoroughly tested for robustness by the end of the summer. Massive improvements to the NCI system would also provide major benefits to projects writing library wrappers, like some of the other GSoC projects are doing.
Chance of success for this project is high, and there are clear benefits to Parrot in having this feature.
Implementing an Instrumentation Tool for Parrot VM - Muhd Khairul Syamil Hashim
This project came completely out of left field and is the only one that I never suggested on my blog ahead of time. It's also the only project proposal that came to us with working proof-of-concept code, and the results of that small example already have some of the Parrot developers imagining of all sorts of fancy long-term uses.
Muhd, "Khairul" on IRC, is planning a system of hooks and callbacks in Parrot that would allow one interpreter to attach to another interpreter at runtime and harvest information about the system as it executes. By adding the appropriate hooks and writing the correct handling routines, we could end up with all sorts of great tools, such as profilers, analyzers, debuggers, etc. And the best part is that we gain the ability to implement all these fun tools in PIR or even another HLL, not in C like some current tools use.
You don't have to think about it for long to understand the benefits of this project: Identifying code hotspots and bottlenecks, analyzing PMC usage patterns, tracking and mapping control flow, and the list goes on. We may even get to a point when we can replace Parrot's current debugging, tracing, and profiling runcores with this single tool and a variety of different PIR programs. Quite fun to think about.
NFG and Single-Representation for the Parrot VM - Daniel Arbelo Arrocha
Daniel, everbody's favorite "darbelo" from last year's GSoC fame, has proposed two projects this year. The first is an attempt to implement NFG, a new normalization form for Unicode text strings. The benefits for NFG are a perfect one-to-one mapping between a character and it's underlying representation. This in turn can lead to some significant simplifications in the code of Parrot, and some proposed speedups as well.
NFG is one of the few points in the Parrot design spec that has never even been prototyped in working code, that I am aware of. Everything else has at least been given prototype treatment, but NFG remains nowhere to be seen in the actual codebase.
Daniel's proposal is to implement NFG in a testbed so that it's performance can be compared to the current string system. If it is a benefit (and we all hope it is!) we can merge it into trunk and start a new era of strings in Parrot. If not, we can continue development on it until it becomes better, or even scrap it entirely.
Parrot on RTEMS - Daniel Arbelo Arrocha
Parrot on RTEMS - Robert Kuo
We had two submissions from prospective students with the exact same name. Both darbelo and bubflub have been inspired by dukeleto's work in the area and have proposed to work on the RTEMS port of Parrot. A major portion of this work will be improvements to the cross-compilation capability of our current configuration and build system, improvements which will have many benefits outside RTEMS. In addition, these two are proposing to help improve the operation of Parrot on RTEMS.
As far as I am aware, this submissions were both duplicated to the RTEMS people as well, so even if one of these proposals get accepted it's possible that the student will actually be working with an RTEMS mentor instead of a Parrot one.
In both cases, I think the alternate proposals for both students look a little more desirable (darbelo's NFG proposal, and bubaflub's GMP proposal), but I've never really used RTEMS so I'm probably biased. I wouldn't definitely not do any crying if one of these "Parrot on RTEMS" projects got accepted, however.
Thursday, April 8, 2010
Writing Ops in PIR
I put together an interesting little code snippet a few days ago that, when finished, would allow writing ops in PIR. I am probably not going to complete work on this idea myself (at least, not any time soon), so I want to put the partially-written code up here for other people to look at and maybe run with.
This code, intended to become a dynop library one day, introduces a single new op: register_op_sub. register_op_sub takes three arguments: the string name of the new op, the string signature of the new op, and the Sub PMC for the new op. An entry is prepared for the interpreter's op_lib table (I haven't found a good way to insert that record yet), and the sub is stored in the "pir_op_lib" namespace. The body of the new op is the function Parrot_pir_op_lib_wrapper. This function body looks up the name of the current op in the op_lib table, then looks up the Sub PMC in the namespace, and will attempt to invoke that sub.
I haven't worked out all the mechanics of everything, obviously. But once a few more questions get answered and a modest amount of code gets written, this could turn into a very useful and very cool extension library for Parrot.
BEGIN_OPS_PREAMBLE
opcode_t *
Parrot_pir_op_lib_wrapper_func(opcode_t *cur_opcode, PARROT_INTERP)
{
// TODO: Set dest to the address of the next instruction, after all arguments.
// need to parse the number of args from the op signature
opcode_t * dest = cur_opcode + 3;
INTVAL opnum = (INTVAL)(*cur_opcode);
char * const opname = interp->op_info_table[opnum]->name;
STRING * name = string_make(interp, opname, 8, "ascii",
PObj_constant_FLAG|PObj_external_FLAG);
PMC * const root_ns = interp->root_namespace;
PMC * const ns = Parrot_get_namespace_keyed(interp, root_ns, CONST_STRING(interp, "pir_op_lib"));
PMC * sub = VTABLE_get_pmc_keyed_str(interp, ns, name);
// TODO: Need to convert the op signature into a PCC signature here
PMC * call_sig = Parrot_pcc_build_sig_object_from_op(interp, PMCNULL, signature,
cur_opcode);
Parrot_pcc_set_signature(interp, CURRENT_CONTEXT(interp), call_sig);
interp->current_cont = NEED_CONTINUATION;
dest = VTABLE_invoke(interp, sub, dest);
goto ADDRESS(dest);
}
END_OPS_PREAMBLE
op register_op_sub(IN STR, IN STR, IN PMC) {
PMC * const root_ns = interp->root_namespace;
PMC * const sub = $3;
PMC * const key = Parrot_pmc_new(interp, enum_class_ResizableStringArray);
STRING * const opname = $1;
STRING * const sig = $2;
PMC * const ns = Parrot_get_namespace_keyed(interp, root_ns, CONST_STRING(interp, "pir_op_lib"));
INTVAL argcount = 3; // TODO: Count number of args in signature
STRING * const separator = CONST_STRING(interp, "_");
STRING * longname = Parrot_str_append(interp, opname, separator);
longname = Parrot_str_append(interp, longname, sig);
VTABLE_set_string_keyed_int(interp, key, 0, );
VTABLE_set_pmc_keyed_str(interp, ns, $2, sub);
op_info_t * new_op = (op_info_t *)malloc(sizeof(op_info_t));
new_op->name = Parrot_str_to_cstring(interp, opname);
new_op->full_name = Parrot_str_to_cstring(interp, longname);
new_op->jump = 0;
new_op->opcount = argcount + 1;
// TODO: Initialize ->type based on what's in signature
new_op->type = (arg_type_t *)malloc(sizeof(arg_type_t) * PARROT_MAX_ARGS);
// TODO: Initialize based on the signature
new_op->dirs = (arg_dir_t *)malloc(sizeof(arg_dir_t) * PARROT_MAX_ARGS);
// TODO: what is this?
new_op->labels = (char *)malloc(sizeof(char) * PARROT_MAX_ARGS);
// TODO: Insert the new op into the optable.
}This code, intended to become a dynop library one day, introduces a single new op: register_op_sub. register_op_sub takes three arguments: the string name of the new op, the string signature of the new op, and the Sub PMC for the new op. An entry is prepared for the interpreter's op_lib table (I haven't found a good way to insert that record yet), and the sub is stored in the "pir_op_lib" namespace. The body of the new op is the function Parrot_pir_op_lib_wrapper. This function body looks up the name of the current op in the op_lib table, then looks up the Sub PMC in the namespace, and will attempt to invoke that sub.
I haven't worked out all the mechanics of everything, obviously. But once a few more questions get answered and a modest amount of code gets written, this could turn into a very useful and very cool extension library for Parrot.
Wednesday, April 7, 2010
Parrot Virtual Developer Meeting
Allison sent a message to the list today to schedule a virtual developer meeting for all Parrot developers. The meeting will be this Sunday, 11 April, at 7:30 UTC.
All Parrot devs should try to attend. All prospective Parrot devs and well-wishers (GSoC students?) should either try to watch or read the logs afterwards. It will definitely be extremely interesting stuff.
I hope to see lots of people there, and I hope we get some good planning work accomplished!
All Parrot devs should try to attend. All prospective Parrot devs and well-wishers (GSoC students?) should either try to watch or read the logs afterwards. It will definitely be extremely interesting stuff.
I hope to see lots of people there, and I hope we get some good planning work accomplished!
Labels:
Parrot
Subscribe to:
Posts (Atom)




