<?xml version='1.0' encoding='UTF-8'?><?xml-stylesheet href="http://www.blogger.com/styles/atom.css" type="text/css"?><feed xmlns='http://www.w3.org/2005/Atom' xmlns:openSearch='http://a9.com/-/spec/opensearchrss/1.0/' xmlns:georss='http://www.georss.org/georss' xmlns:gd='http://schemas.google.com/g/2005' xmlns:thr='http://purl.org/syndication/thread/1.0'><id>tag:blogger.com,1999:blog-4146794174400139442</id><updated>2011-11-27T19:08:51.337-05:00</updated><category term='Personal'/><category term='Python'/><category term='GSOCIdeas'/><category term='Microsoft'/><category term='Parrot2.0'/><category term='Contexts'/><category term='Performance'/><category term='Prism'/><category term='ParrotUsers'/><category term='Parrot3.0'/><category term='Release'/><category term='C'/><category term='VirtualBox'/><category term='Wikibooks'/><category term='ParrotProductManagement'/><category term='BadCode'/><category term='Plumage'/><category term='SecretProjects'/><category term='Threads'/><category term='Security'/><category term='Boehm'/><category term='Kakapo'/><category term='Strings'/><category term='libJIT'/><category term='Deprecation'/><category term='Politics'/><category term='Testing'/><category term='GSOC'/><category term='LLVM'/><category term='Wittie'/><category term='Community'/><category term='Trac'/><category term='JIT'/><category term='Git'/><category term='Roadmap'/><category term='AIO'/><category term='PCC'/><category term='Software'/><category term='Work'/><category term='Matrixy'/><category term='ParrotTeams'/><category term='ExecutionObject'/><category term='Policy'/><category term='CodeAnthem'/><category term='Macintosh'/><category term='LAPACK'/><category term='EmbedVideo'/><category term='lost'/><category term='Parrot-Linear-Algebra'/><category term='VisualStudio'/><category term='MediaWiki'/><category term='PMC'/><category term='IO'/><category term='PeerReview'/><category term='Parrot-Data-Structures'/><category term='PIR'/><category term='Engineering'/><category term='ParrotFoundation'/><category term='Neko'/><category term='Parrot4Newbies'/><category term='VisualBasic'/><category term='Objects'/><category term='Editor'/><category term='Wikimedia'/><category term='Microprocessors'/><category term='C#'/><category term='PHP'/><category term='Parrot'/><category term='Win64'/><category term='ParrotSketch'/><category term='ParrotTheory'/><category term='ParrotDesign'/><category term='MockObject'/><category term='Smolder'/><category term='Agile'/><category term='YAPC'/><category term='Linux'/><category term='Rakudo'/><category term='Lorito'/><category term='GC'/><category term='Benchmarks'/><category term='BookDesigner'/><category term='JavaScript'/><category term='Github'/><category term='Wiki'/><category term='ParrotProjects'/><category term='Exceptions'/><category term='Education'/><category term='Optimization'/><category term='Silverlight'/><category term='Hackathon'/><title type='text'>Whiteknight's World</title><subtitle type='html'>This is a test</subtitle><link rel='http://schemas.google.com/g/2005#feed' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/posts/default'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default?max-results=100'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/'/><link rel='hub' href='http://pubsubhubbub.appspot.com/'/><link rel='next' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default?start-index=101&amp;max-results=100'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><generator version='7.00' uri='http://www.blogger.com'>Blogger</generator><openSearch:totalResults>421</openSearch:totalResults><openSearch:startIndex>1</openSearch:startIndex><openSearch:itemsPerPage>100</openSearch:itemsPerPage><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-8898751680505883376</id><published>2010-11-04T09:40:00.000-04:00</published><updated>2010-11-04T09:40:49.646-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='Personal'/><title type='text'>Blogger Blog is Closed</title><content type='html'>This morning I finished with the final important bits of my blog migration.&lt;br /&gt;&lt;br /&gt;&lt;ul&gt;&lt;li&gt;&lt;a href="http://feeds.feedburner.com/afwknight"&gt;New feed for all posts&lt;/a&gt;&lt;/li&gt;&lt;li&gt;&lt;a href="http://feeds.feedburner.com/afwknight-parrot"&gt;New feed for Parrot posts&lt;/a&gt;&lt;/li&gt;&lt;li&gt;I now have comments powered by DISQUS&lt;/li&gt;&lt;/ul&gt;I'm locking this blog down so it will no longer receive any modifications from me, and won't be open to any new comments. I will keep this blog around as read-only for now.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-8898751680505883376?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/8898751680505883376/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/11/blogger-blog-is-closed.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/8898751680505883376'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/8898751680505883376'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/11/blogger-blog-is-closed.html' title='Blogger Blog is Closed'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-8697786307592767542</id><published>2010-11-02T21:37:00.000-04:00</published><updated>2010-11-02T21:37:07.548-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='Personal'/><title type='text'>Blog Migration</title><content type='html'>I am migrating my blog off of Blogger.com. My first post is up on &lt;a href="http://whiteknight.github.com/"&gt;my new blog&lt;/a&gt; at Github:&lt;br /&gt;&lt;br /&gt;&lt;a href="http://whiteknight.github.com/2010/11/02/blog_migration.html"&gt;http://whiteknight.github.com/2010/11/02/blog_migration.html&lt;/a&gt; &lt;br /&gt;&lt;br /&gt;I am only midway through the migration, but there ain't no stopping this train we're on. I will no longer be publishing complete posts here at Blogger, only links to posts on the new site until I get the last few details in place. At that point, I will lock this blog, and it will no longer be used at all.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-8697786307592767542?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/8697786307592767542/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/11/blog-migration.html#comment-form' title='2 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/8697786307592767542'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/8697786307592767542'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/11/blog-migration.html' title='Blog Migration'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>2</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-3615692424491768668</id><published>2010-10-24T09:10:00.001-04:00</published><updated>2010-10-24T16:06:09.087-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='Parrot'/><category scheme='http://www.blogger.com/atom/ns#' term='ParrotProductManagement'/><title type='text'>Case Study: Parrot Critique</title><content type='html'>In response to my previous post about &lt;a href="http://wknight8111.blogspot.com/2010/10/product-management-team.html"&gt;my vision for the Product Management team&lt;/a&gt;, Parrot user shockwave posted a very long and thoughtful critique. I have a very large backlog of other topics that I want to post about, but I want to take the time to respond directly to shockwave right now. In a nutshell, for anybody who hasn't read his entire set of comments, he is trying to embed a Parrot interpreter for a custom language he's designing into a 3D game engine that he is working on. His comments were very long (worth the read!), so I will be quoting some of the most important parts, with minor corrections.&lt;br /&gt;&lt;br /&gt;&lt;blockquote&gt;I started to have issues embedding Parrot as soon as I tried to embed it. &lt;/blockquote&gt;As much as I hate to admit it, this doesn't surprise me. We have very few projects attempting to use Parrot in an embedding situation. In fact, I can only really think of one: this one. Some people will mention mod_parrot. I'm not sure whether that counts as an embedding or an extending type of project, but I definitely don't think it's being actively maintained right now. Whatever, maybe two projects.&lt;br /&gt;&lt;br /&gt;It's just a fact of the programming world: Features which are not often used, and aren't thoroughly tested are going to wither. As a community we could be more proactive and show a little bit more foresight to ensure that this interface is capable, powerful, elegant, and maintained, but we don't. Maybe that's the single most important thing that the Product Management team should focus on for now.&lt;br /&gt;&lt;br /&gt;&lt;blockquote&gt;First, there's barely any documentation for embedding it. Most of the  documentation I found was just the listing of function prototypes.&lt;/blockquote&gt;&lt;br /&gt;Damnit! I find this so infuriating. As I mentioned in a previous post about the state of the PDDs, listings of function prototypes &lt;b&gt;do not&lt;/b&gt; qualify as acceptable user documentation. A documentation file which only lists function prototypes and maybe a short, abstract blurb about them, is not adequate documentation for users. Also, the kinds of documentation that users need is often far different from the kinds of documentation that developers need. So, there's another important thing that the Product Management team should be working on.&lt;br /&gt;&lt;br /&gt;&lt;blockquote&gt;To be clear, Parrot is something designed to be embedded, and there is no documentation on how to do that. That's not good. &lt;/blockquote&gt;&amp;nbsp;This is a sentiment that I really couldn't agree with more. This is a huge part of my personal vision for Parrot. The "real" final product that we ship, the most important part of our binary distribution is not the Parrot executable. That's just a thin argument-processing wrapper around the most important part: libparrot. The Parrot executable is just a small utility that &lt;i&gt;embeds&lt;/i&gt; libparrot. At least, that's what it should be. libparrot is the product that we are developing. Everything else--all the command-line tools, the HLL compilers and fakecutables, extension libraries, everything--are just add-ons to libparrot.&lt;br /&gt;&lt;br /&gt;Anything that the Parrot executable can do, all the interfaces that it calls from libparrot, should be made available and easily so from other embedding applications. And all of it should be well documented.&lt;br /&gt;&lt;blockquote&gt;&lt;br /&gt;the documentation for that function says this:&lt;br /&gt;&lt;br /&gt;"void Parrot_load_bytecode(PARROT_INTERP, STRING *path)&lt;br /&gt;Reads and load Parrot bytecode ... . Due to the void return type, the behavior of this function on error is unclear."&lt;br /&gt;&lt;br /&gt;Basically, if a file doesn't exist, the program state then becomes undefined. Not good.&lt;/blockquote&gt;&lt;br /&gt;&lt;br /&gt;"the behavior of this function on error is unclear"? Full stop. This is absolutely, unapologetically stupid and &lt;i&gt;wrong&lt;/i&gt;. Close your eyes and imagine me saying a few cursewords before you continue reading the rest of this post because, trust me, I &lt;i&gt;am&lt;/i&gt; saying them now.&lt;br /&gt;&lt;br /&gt;The job of any application virtual machine, be it the JVM, the .NET CLR, Neko VM, or whatever, is to make these kinds of things clear. Virtual machines provide a consistent abstraction layer over the underlying platform, and provide a standard and reliable runtime for applications that run on it. What part of that definition allows for a key interface function to both allow errors and also to have undefined behavior when an error exists? Either that function needs to be fixed to have defined and consistent behavior on error, or a new function needs to be written that that goal explicitly in mind.&lt;br /&gt;&lt;br /&gt;&lt;blockquote&gt;Parrot seems to be built with a command line mentality. Not all the  actual users(end-users, not developers) of Parrot will be running the  end product from a command line. The video game engine I'm using, for  example, runs under Windows, MacOS, XBox 360, and PS3. I'm trying to run  it from Windows 7, as a GUI application; the errors are outputed to  nowhere. Parrot can't just spit out error messages to the command line  and expect that they will always be seen. There's no built-in way to  place the errors in buffer, so that I could choose how to print those  errors. &lt;/blockquote&gt;&lt;br /&gt;Let me paraphrase, using common internet vernacular: Parrot IZ T3H FAILZ.&amp;nbsp; Any Parrot hacker who reads this paragraph should immediately be able to extract a number of TODO items from it. libparrot should always assume that it is being embedded, and act accordingly. Again, the Parrot executable isn't our primary product, libparrot is. Libraries like this shouldn't be just dumping text out to STDERR or any standard handle without allowing some possibility of explicit overriding. If we dumped error text into a buffer, set a flag, and allowed the embedding application handle it appropriately, we would be much better off.&lt;br /&gt;&lt;br /&gt;In fact, I suggest that all functions which execute code in Parrot should be modified to return a PMC. That PMC could be an exit code, a status message, an unhandled exception, some kind of callback, or whatever. If the PMC returned is an unhanded exception, the embedding application could chose to handle it and call back in to Parrot, or propagate that error further up the call chain, or whatever. It's not libparrot's job to determine that an unhandled exception causes the application to exit, or to force a dump of the exception text to STDERR. Both of these things are very wrong in many situations.&lt;br /&gt;&lt;br /&gt;&lt;blockquote&gt;I want to use Parrot as the runtime, because of business and personal  reasons. but I can't base my business and future on hopes and dreams  alone. I'd really like to go with Parrot as the runtime for the game  engine, but if it can't stand on its own feet, I'm gonna have to shelf  it for some, possibly long, time.&amp;nbsp; &lt;/blockquote&gt;&lt;br /&gt;And here it is, the heartbreaker in this whole situation. Here we have a person who likes Parrot and wants to use it. He has put in the effort to build a compiler and runtime for his language, but the embedding interface is so shitty shitty shitty that he can't use it and may have to go back and implement all his ideas in a different system. Excuse me while I go say a few more choice curse words.&lt;br /&gt;&lt;br /&gt;Parrot's embedding/extending interface has never been any good because nobody has ever taken the time to define exactly what that interface is and what it should be. Everybody has been happy to only use Parrot from the command-line, and to break encapsulation at every step because it was easier and nobody wanted to do any differently.&lt;br /&gt;&lt;br /&gt;What we need to do, and do it as immediately as possible, is to realize that there are two distinct pieces of software: libparrot and the parrot executable. We need to realize that the former is our primary product, and can be used in many situations where the later cannot. We also should realize that libparrot needs a complete, comprehensive, elegant, and properly-encapsulating API, which the Parrot executable and other embedding programs should use exclusively. We need to start making some tough decisions about what that API should contain, what it should not contain, and what kind of functionality our users are going to have access to.&lt;br /&gt;&lt;br /&gt;We need to write up a proper API, test it, and document it. Anything less is failure and means we are ignoring the very real needs of our users.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-3615692424491768668?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/3615692424491768668/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/10/case-study-parrot-critique.html#comment-form' title='1 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/3615692424491768668'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/3615692424491768668'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/10/case-study-parrot-critique.html' title='Case Study: Parrot Critique'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>1</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-7486934225093550457</id><published>2010-10-23T20:00:00.001-04:00</published><updated>2010-10-23T20:00:01.610-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='Work'/><category scheme='http://www.blogger.com/atom/ns#' term='Personal'/><title type='text'>Taking a new Job</title><content type='html'>Friday was my last day at &lt;a href="http://www.ionxlive.com/"&gt;IONX&lt;/a&gt;, the first job I had out of school. IONX was a pretty cool place to work, not the ordinary kind of first job that some people get doing the "grunt" work that more senior programmers didn't want for themselves. It was a relatively small place, and though it's been growing quickly, I was still able to carve out a place for myself as a productive, influential, and respected programmer. &lt;br /&gt;&lt;br /&gt;The time came eventually to start looking for something else. I started putting out resumes in a pretty non-committal way. Maybe one or two a month if I found really compelling job postings. For several months I hardly heard back from anybody. But this month, the rain began to pour. Ultimately I decided to take a position with &lt;a href="http://weblinc.com/"&gt;Weblinc&lt;/a&gt;, an e-commerce company based in Philadelphia.&lt;br /&gt;&lt;br /&gt;Starting on Monday I'll be taking the train to center-city Philadelphia every day, again. It's going to be a fun new adventure and I'm looking forward to getting started. I'll post more information about what exactly my new job entails when I learn a little bit more about it myself.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-7486934225093550457?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/7486934225093550457/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/10/taking-new-job.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/7486934225093550457'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/7486934225093550457'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/10/taking-new-job.html' title='Taking a new Job'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-3928066459157510486</id><published>2010-10-21T17:00:00.000-04:00</published><updated>2010-10-21T17:00:02.578-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='Parrot'/><category scheme='http://www.blogger.com/atom/ns#' term='ParrotProductManagement'/><title type='text'>Product Management Team</title><content type='html'>Yesterday I talked about the new Teams concept for the Parrot community. The goal of which is to take some important jobs and assign them to small groups instead of individuals. For a team like the architecture team, this task was previously assigned to only one individual. For other teams like Project Management, Community, Product Management, and QA, these are pretty new and didn't previously belong to any one dedicated individual.&lt;br /&gt;&lt;br /&gt;With that in mind, I need to start putting together a comprehensive vision for what the Product Management Team will be, since we haven't ever had a "Project Manager" before to show me the way forward. The description in the teams proposal goes as follows:&lt;br /&gt;&lt;br /&gt;&lt;blockquote&gt;This team is responsible for the vision of Parrot as a user-facing  product.  They also act as an advocate for the needs of Parrot's users  (e.g. HLLs and libraries such as Rakudo and Kakapo) and as an  intermediary between Parrot and its users.&lt;/blockquote&gt;&lt;br /&gt;There are a few different tasks here (and I think a few more are implied), but&amp;nbsp; it all really boils down to a single core concept: Making Parrot "work" for current and prospective users.&lt;br /&gt;&lt;br /&gt;Who are Parrot's users? That's really a good first question to ask. John Q Bag-of-donuts, running an instance of the November Wiki at home, isn't really a Parrot user. Sure, his software is running on top of Parrot, but the reality is that he is using November and Rakudo, not Parrot directly. I think that relatively few people would be (or should be) using Parrot directly. As far as most people should ever know or care, Parrot is just another library dependency for some higher-level software that &lt;i&gt;just works. &lt;/i&gt;&lt;br /&gt;&lt;br /&gt;Parrot users are the developers of HLL compilers, PIR libraries, PMC extensions, and embedding hosts. In many cases, especially historically, the group of Parrot users overlapped pretty heavily with the group of Parrot's developers, so the two groups would often get conflated. We can't do that anymore.&lt;br /&gt;&lt;br /&gt;Users need Parrot to &lt;i&gt;do&lt;/i&gt;. Users need Parrot to &lt;i&gt;be&lt;/i&gt;. Parrot's capabilities, it's external interfaces, it's execution speed and it's memory usage; our users need all these things and need them to be done correctly. Parrot, in turn, needs users. The relationship is cyclic: A better Parrot attracts and retains more satisfied users. Satisfied users, in turn, provide feedback to the development team and make Parrot appear to be a more popular and compelling platform. There's word-of-mouth advertising too that we want to bank on. Of course, all the advertising and good will in the world don't replace rich feature sets and improved performance.&lt;br /&gt;&lt;br /&gt;In a nutshell, these are the tasks that I think the Product Management team needs to be focused on:&lt;br /&gt;&lt;br /&gt;&lt;ol&gt;&lt;li&gt;&lt;b&gt;Communication&lt;/b&gt;: We need to stay in touch with the users, bring their concerns to the rest of the Parrot development team, and share their successes as well&lt;/li&gt;&lt;li&gt;&lt;b&gt;Marketing&lt;/b&gt;: We need to advertise Parrot. We need to attract new users, and help encourage people to build awesome things on top of our platform.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Interfaces&lt;/b&gt;: We need to work to make and keep Parrot's external interfaces sane. Our command-line interface, extending interface, embedding interface, and PIR/PASM/PBC. These things all need to be sane and either stable or in the midst of dramatic improvements. We will elicit feedback from users to find out what features people use and what features people don't use. This will help facilitate feature deprecations and experimental feature promotions in the future. As everybody can imagine, I would love to get my hands around the throat of the deprecation policy and start squeezing.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Advocacy&lt;/b&gt;: We need to find out exactly what the users want and advocate for that to the rest of the developer team. Parrot isn't a product developed in a vacuum, we have real users with real needs, and we need to make sure that we meet those needs. If we don't meet the needs of our users, Parrot is nothing for nobody. &lt;/li&gt;&lt;/ol&gt;&amp;nbsp;I'm looking for interested and dedicated members to join the Product Management team for Parrot. People who are interested in doing the kinds of things I talk about above, or people who have their own ideas for what this team should be responsible for, would make perfect additions. Ideally, I would like to have at least 3-4 people on the team besides myself, and more would be welcomed too.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-3928066459157510486?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/3928066459157510486/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/10/product-management-team.html#comment-form' title='10 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/3928066459157510486'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/3928066459157510486'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/10/product-management-team.html' title='Product Management Team'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>10</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-290044024492308168</id><published>2010-10-20T15:45:00.000-04:00</published><updated>2010-10-20T15:45:52.071-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='Parrot'/><category scheme='http://www.blogger.com/atom/ns#' term='ParrotTeams'/><title type='text'>Parrot Teams</title><content type='html'>Parrot contributor Christoph Otto (cotto) posted &lt;a href="http://lists.parrot.org/pipermail/parrot-dev/2010-October/004916.html"&gt;an email to the list a few days ago&lt;/a&gt; about a new &lt;a href="http://trac.parrot.org/parrot/wiki/ParrotTeams"&gt;Teams concept&lt;/a&gt; for the Parrot community. This idea is essentially an extension of an idea I had been kicking around in my own head, and I'm very enthusiastic about it.&lt;br /&gt;&lt;br /&gt;The idea is to organize Parrot developers into particular teams, and give each team a particular area of focus and "authority" to get things done. We obviously don't want to get too overzealous and focus more on rigid organization and structures, but we do want to make sure that important jobs are being performed and some people get the chance to take personal responsibility for ensuring that things in the project get done.&lt;br /&gt;&lt;br /&gt;There are five teams slated so far, though I'm sure the number will change as we see how teams work and we divide up tasks that need to get done. &lt;br /&gt;&lt;br /&gt;One important team, for example, is the architectural team, which will be taking over for the position of architect. The idea is that we get more people involved in that role, increase our bus number, and become less reliant on a single person for such an important task. Let's all commend Allison for the great job she has done in this role for the past few years. However, let's also remember that it was a pretty significant drain on her time, and it's a hard job to do for just one person. It looks like Cotto will be the team leader of the architectural team, though I'm already a member of it as well.&lt;br /&gt;&lt;br /&gt;I'm going to be the team leader for the new "Product Management" team. I'm still coming to grips myself with what this new team is supposed to be doing, because the proposal is still young and there are many roles and things that need defining. At the basic level, the Product Management team is supposed to interact with users, and form a comprehensive vision of Parrot as a user-facing product. This is very likely going to have implications in several areas of Parrot, including the embedding/extension interface, compilers for PIR/Lorito, tools, libraries, etc. My goal, at least initially, is to make sure that Parrot is serving the needs of our users, and that Parrot becomes a compelling platform for people to work on. You can damn sure expect more blog posts about these things!&lt;br /&gt;&lt;br /&gt;I am going to try to solicit new membership to this team as best as I can. It's going to be important that we have some enthusiastic contributors here, especially people who have experience developing compilers, extensions, and other related parrot-dependent projects. We're also going to want to establish and maintain close contact with other developers of these kinds of projects, to ensure that we are meeting their needs and ensure that we are pushing to meet and exceed those needs in future releases.&lt;br /&gt;&lt;br /&gt;Other teams include the Project Management team currently headed by Jim Keenan (kid51), the Community team headed by Jonathan Leto (dukeleto), and a Quality Assurance team, which currently has no team lead. All of these teams are going to have to take some time to ensure that they properly define their own roles, and find the best ways to perform them going forward. We also need to attract several developers to join all of these teams, to make sure they all have enough "staff" to get the necessary jobs done.&lt;br /&gt;&lt;br /&gt;Signing up for a team is easy. Talk to the team lead and ask about how to get involved. You don't need to be a member of a team in order to be a committer for Parrot. I'm looking for people to join the Product Management Team, and I know other teams are going to be looking for new members as well, so definitely get involved if you are interested in something we are doing.&lt;br /&gt;&lt;br /&gt;I'll try to put together another post soon with a more comprehensive vision for what the Product Management team will be and what it will do.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-290044024492308168?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/290044024492308168/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/10/parrot-teams.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/290044024492308168'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/290044024492308168'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/10/parrot-teams.html' title='Parrot Teams'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-4138879656075980376</id><published>2010-10-14T18:47:00.001-04:00</published><updated>2010-10-14T18:48:12.916-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='Personal'/><title type='text'>Know Your Fundamentals</title><content type='html'>I'm still not able to make any real posts or other contributions to the outside world at large because of &lt;a href="http://wknight8111.blogspot.com/2010/10/offline.html"&gt;ongoing family medical issues&lt;/a&gt;. However, today I did see a &lt;a href="http://www.wired.com/thisdayintech/2010/10/1014cplusplus-released/all/1"&gt;great quote from everybody's favorite Bjarne Stroustrup&lt;/a&gt; that I think is extremely accurate:&lt;br /&gt;&lt;br /&gt;&lt;blockquote&gt;Know  your fundamentals (algorithms, data structures, machine architecture,  systems) and know several programming languages to the point where you  can use them idiomatically.&lt;br /&gt;&lt;br /&gt;Know some non-computer field of study well — math, biology, history,  optics, whatever. Learn to communicate effectively in speech and in  writing. Spend an unreasonable amount of time on some difficult topic to  really master it. Try to do something that might make a difference in  the world.&lt;/blockquote&gt;&lt;br /&gt;How many people do I know that don't know their fundamentals, who barely know one programming language and certainly don't know it idiomatically? The answer, unfortunately, is many.&lt;br /&gt;&lt;br /&gt;Here's one more gem from the same article:&lt;br /&gt;&lt;blockquote&gt;I’ll  just note that I consider the idea of one language, one programming  tool, as the one and only best tool for everyone and for every problem  infantile. If someone claims to have the perfect language he is either a  fool or a salesman or both.&lt;/blockquote&gt;And on that note, I'm crawling back into my hole for a few days.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-4138879656075980376?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/4138879656075980376/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/10/know-your-fundamentals.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/4138879656075980376'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/4138879656075980376'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/10/know-your-fundamentals.html' title='Know Your Fundamentals'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-7651479994578802542</id><published>2010-10-11T19:07:00.000-04:00</published><updated>2010-10-11T19:07:26.893-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='Personal'/><title type='text'>Offline</title><content type='html'>I've been mostly offline for the past few days. I've been pretty busy and don't have the time to keep up with things the way I would like. There's been a bit of a nexus of things happening all at once:&lt;br /&gt;&lt;br /&gt;&lt;ol&gt;&lt;li&gt;My father has been in the hospital with a pretty serious illness. This has been eating up most of my time and my concentration&lt;/li&gt;&lt;li&gt;I'm moving to a new job. I'll post more details later&lt;/li&gt;&lt;li&gt;My kid is teething and has a bit of a stomach bug. He's just a little bundle of joy.&lt;/li&gt;&lt;li&gt;My dev laptop is broken and disassembled. I'm waiting for new unbroken hinges to be shipped in from Hong Kong. Apparently they are shipping it by camel.&lt;/li&gt;&lt;/ol&gt;I'll have more blog posts to write and more code to commit when things calm down a little bit.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-7651479994578802542?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/7651479994578802542/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/10/offline.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/7651479994578802542'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/7651479994578802542'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/10/offline.html' title='Offline'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-1124918997839719120</id><published>2010-09-27T17:00:00.000-04:00</published><updated>2010-09-27T17:00:01.463-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='PCC'/><category scheme='http://www.blogger.com/atom/ns#' term='ParrotDesign'/><category scheme='http://www.blogger.com/atom/ns#' term='Parrot'/><title type='text'>PDD03 Calling Conventions Critique</title><content type='html'>I've wanted to get back into this habit for a while, and today is the day to do it. Following a short conversation with Parrot hacker plobsing yesterday, I've decided to tackle PDD03, the "Calling Conventions" design document first. In the coming days I would also like to take a close look at some other PDDs for systems which are receiving current developer attention. Quotes are all from the &lt;a href="http://trac.parrot.org/parrot/browser/trunk/docs/pdds/pdd03_calling_conventions.pod"&gt;text of PDD03&lt;/a&gt;.&lt;br /&gt;&lt;br /&gt;&lt;blockquote&gt;FAQ: Given Parrot's internal use of continuation-passing style ["CPS"], it&lt;br /&gt;would be possible to use one pair of opcodes for both call and return, since&lt;br /&gt;under CPS returns are calls.&amp;nbsp; And perhaps someday we will have only two&lt;br /&gt;opcodes. But for now, certain efficiency hacks are easier with four opcodes.&lt;/blockquote&gt;&lt;br /&gt;"Perhaps someday"? Is this an authoritative design document or a dumping ground for assorted wishful thinking? This document should say exactly what Parrot wants in the long run. Do we want two opcodes for simplicity, or do we want four opcodes because of optimization potential? I would suggest that we have two opcodes only, and we can implement optimizing behaviors by having multiple types of Continuation object, with at least one type specifically optimized for simple subroutine returns. We used to have the RetContinuation PMC type, and while it wasn't the right solution for the problem it did have a glimmer of a good idea buried in it.&lt;br /&gt;&lt;blockquote&gt;set_opcode "flags0, flags1, ..., flagsN", VAL0, VAL1, ... VALN&lt;br /&gt;get_opcode "flags0, flags1, ..., flagsN", REG0, REG1, ... REGN&lt;br /&gt;get_opcode "..., 0x200, flags0, ...", ..., "name", REG0, ...&lt;/blockquote&gt;It's hard to talk about needing to add extra opcodes to facilitate "efficiency hacks", and then saying that for every single parameter pass and retrieval that we need to parse integer values from a constant string. Mercifully, this isn't what we do anymore. In all these cases the first argument is a FixedIntegerArray of flags, which is computed by the assembler and serialized as a constant into the bytecode. At the very least, the design document should be updated to reflect that bit of sanity.&lt;br /&gt;&lt;br /&gt;What's interesting here is the fact that these opcodes are &lt;i&gt;variadic&lt;/i&gt;. That is, these opcodes (which, I believe, are unique among all 1200+ opcodes in Parrot) take an open-ended list of arguments. This makes traversing bytecode extremely difficult, which in turn makes tools for generating, reading, and analyzing bytecode extremely difficult, and needlessly so. Far superior to this would be to serialize information about the register indices into the same PMC that contains the flags for those parameters.&lt;br /&gt;&lt;br /&gt;Right now, we use a special PMC called a CallContext to help facilitate subroutine calls. The CallContext is used to pack up all the arguments to the subroutine, and then it also serves as the dynamic context object for the subroutine. It contains the actual storage for the registers used, and handles unpacking arguments into those registers. It also manages some other things for context, like lexical associations between subs and other details.&lt;br /&gt;&lt;br /&gt;In short, the CallContext stores some static data that we usually know about at compile time: argument lists and argument signatures, parameter signatures, etc. It also knows where arguments are coming from, whether they are coming from constants or registers, and which registers exactly are being used. All this information is needlessly calculated at runtime when it could easily be computed at compile time and stored in the constants table of&amp;nbsp; the PBC. If we split CallContext up into a dynamic part (CallContext) and a static part (CallArguments and maybe CallParameters), we could serialize the static part at compile time and avoid a lot of runtime work.&lt;br /&gt;&lt;br /&gt;Here's an example call:&lt;br /&gt;&lt;br /&gt;&lt;pre&gt;set_args callargs # A CallArguments PMC, which can be loaded from PBC as a constant&lt;br /&gt;invokecc obj, meth&lt;br /&gt;&lt;/pre&gt;&lt;br /&gt;Or, we could cut out the separate opcode entirely:&lt;br /&gt;&lt;br /&gt;&lt;pre&gt;invokecc obj, meth, callargs&lt;br /&gt;&lt;/pre&gt;&lt;br /&gt;Here's an example subroutine using a new mechanism:&lt;br /&gt;&lt;br /&gt;&lt;pre&gt;.pir sub foo&lt;br /&gt;callcontext = get_params foo&lt;br /&gt;callparams = get_param_defs foo&lt;br /&gt;unpack_params callcontext, callparams&lt;br /&gt;&lt;/pre&gt;&lt;br /&gt;Here, again, "callparams" is a PMC containing static information about the parameter signature of the subroutine, and can easily be serialized to bytecode to avoid runtime calculations.&lt;br /&gt;&lt;br /&gt;With this kind of a system we have three PMCs which can work together in tandem to implement a calling sequence, and can be easily overridden by HLLs to implement all sorts of custom behaviors. Plus, we get good separation of concerns, which is a typical component of good design. We have CallContext, which serves as the runtime context of the subroutine and acts as storage for registers and lexical associations. CallArguments performs a mapping of callee registers into a call object. Then we have a CallParameters which performs the inverse mapping of call arguments into parameter registers. With these three things anybody could write their own calling conventions and have them be seamlessly integrated into Parrot without too much trouble. At any time you can pack arguments into a CallContext using a CallArguments PMC (which you can created at runtime or serialize to PBC), and then unpack them again using a CallParameters PMC (which can also be created at runtime or compile time). Tailcall optimizations, loop unwinding, recursion avoidance, and all sorts of other optimizations become trivially possible to implement at any level.&lt;br /&gt;&lt;blockquote&gt;For documentation purposes we'll number the bits 0 (low) through 30 (high).&lt;br /&gt;Bit 31 (and higher, where available) will not be used.&lt;/blockquote&gt;Is there a particular reason why bit 31 is off limits? Is this only because INTVAL is a signed quantity, and we don't want to be monkeying with the sign bit (because an optimizing compiler may monkey with it right back)? That would make sense, but is absolutely unexplained here.&lt;br /&gt;&lt;blockquote&gt;The value is a literal constant, not a register.&amp;nbsp; (Don't set this bit&lt;br /&gt;yourself; the assembler will do it.)&lt;/blockquote&gt;This is something that upsets me to no end, and I would be extremely happy to change this particular behavior. Here's the current behavior, in a nutshell: Every single parameter has a flag that determines whether the parameter comes from a register or from the constants table in the bytecode. That means for every single parameter, on every single function call, we need to do a test for constantness and branch to appropriate behavior. &lt;i&gt;For every single parameter, for every single function call&lt;/i&gt;. Let that sink in for a minute.&lt;br /&gt;&lt;br /&gt;Consider an alternative. We have a new opcode like this:&lt;br /&gt;&lt;br /&gt;&lt;pre&gt;register = get_constant_p_i index&lt;br /&gt;&lt;/pre&gt;&lt;br /&gt;The "_p_i" suffix indicates that the opcode returns a PMC and takes an integer argument. We could have variants for "_i_i", "_n_i" and "_s_i" too.&lt;br /&gt;&lt;br /&gt;Let's compare two code snippets. First, the current system:&lt;br /&gt;&lt;br /&gt;&lt;pre&gt;$I0 = 0&lt;br /&gt;loop_top:&lt;br /&gt;foo($I0, "bar")&lt;br /&gt;inc $I0 &lt;br /&gt;if $I0 &amp;lt; 100 goto loop_top&lt;br /&gt;&lt;/pre&gt;&lt;br /&gt;And now, in a system with a get_constant opcode:&lt;br /&gt;&lt;br /&gt;&lt;pre&gt;$I0 = get_constant 0&lt;br /&gt;$S1 = get_constant 1&lt;br /&gt;$I2 = get_constant 2&lt;br /&gt;loop_top:&lt;br /&gt;foo($I0, $S1)&lt;br /&gt;inc $I0&lt;br /&gt;if $I0 &amp;lt; $I2 goto loop_top&lt;br /&gt;&lt;/pre&gt;&lt;br /&gt;It looks like more opcodes total, but this second example would probably execute &lt;i&gt;faster&lt;/i&gt;. Why?&lt;br /&gt;&lt;br /&gt;The foo() function is called 100 times. In the current system, every single time the function is called, &lt;i&gt;every single time&lt;/i&gt;, the PCC system must ask whether the first argument is a constant (it never is) and whether the second argument is a register (it never is). Then, &lt;i&gt;every single time&lt;/i&gt;, it needs to load the value of the second argument from the constant table. This is also not to mention the fact that the "if" opcode at the bottom could be loading the value of it's second argument from the constants table if that argument was a string or a PMC.&amp;nbsp; INTVAL arguments are stored directly in the PBC stream, so they don't need to be loaded from the constants table. Luckily, I think serialized PMCs are thawed once when the PBC is loaded and don't need to be re-thawed from the PBC each time that they are loaded.Of course, I could nit-pick and suggest we should only thaw PMCs lazily on-demand and cache them, but that's a detail that probably doesn't matter a huge amount in the long run (unless we start saving a hell of a lot more PMCs to bytecode). &lt;br /&gt;&lt;br /&gt;Of course, for code generators we're going to need a way to get the unique integer index for each stored constant, which likely means we're going to need an improved assembler, but it is doable.&lt;br /&gt;&lt;br /&gt;&lt;blockquote&gt;If this bit [:flat] is set on a PMC value, then the PMC must be an aggregate.&amp;nbsp; The&lt;br /&gt;contents of the aggregate, rather than the aggregate itself, will be passed.&lt;br /&gt;If the C&lt;named&gt; bit is also set, the aggregate will be used as a hash; its&lt;br /&gt;contents, as key/value pairs, will be passed as named arguments.&amp;nbsp; The PMC&lt;br /&gt;must implement the full hash interface.&amp;nbsp; {{ &lt;a href="http://trac.parrot.org/parrot/ticket/1288"&gt;TT #1288: Limit the required interface.&lt;/a&gt; }}&lt;br /&gt;&lt;br /&gt;If the C&lt;named&gt; bit is not set, the aggregate will be used as an array; its&lt;br /&gt;contents will be passed as positional arguments.&lt;/named&gt;&lt;/named&gt;&lt;br /&gt;&lt;br /&gt;The meaning of this bit is undefined when applied to integer, number, and&lt;br /&gt;string values.&lt;/blockquote&gt;&lt;br /&gt;I understand what this passage is trying to say, but it's still pretty confusing. Plus, I always find it funny when our design documents contain references to tickets (and in this case, a ticket that hasn't drawn a single comment in 10 months from anybody who could make a reasonable decision on the issue). There's probably a bigger discussion to be had about what it means when a PMC declares that it "provides hash". Parrot does have some support for roles, but that support is thin and mostly untested. Plus, nowhere do we define what any of our built-in roles like "hash" and "array" actually mean. That's a different problem for a different blog post, but it is worth mentioning here.&lt;br /&gt;&lt;br /&gt;Here's a slightly better description: If you use the ":flat" flag, Parrot is going to create an iterator for that PMC and iterate over all elements in the aggregate, passing each to the subroutine. If the ":named" flag is used, that iteration will be done like hash iteration (values returned from the iterator are keys). Otherwise, the iteration will be normal array-like iteration. If the given PMC does not provide a get_iter VTABLE, an exception will be thrown. There's no sense talking about how the PMC must satisfy any kind of interface, since the only thing we require is that the aggregate is iterable.&lt;br /&gt;&lt;br /&gt;It's worth noting that after looking at the code I don't think Parrot follows this design. I don't think the presence of the :named flag with the :flat flag changes the behavior at all. It appears from reading the code that hashes or hash-like PMCs are always iterated as hashes and the contents will always be passed as name/value pairs. Arrays and array-like PMCs are always iterated as arrays and their contents are always passed individually. Where a PMC is both array-like and hash-like at the same time, it's contents are iterated as an array and passed individually. I do not know whether this behavior is acceptable (and the design should be updated) or whether the implementation is lacking and the design is to be followed. I may try to put together some tests for this behavior later to illustrate.&lt;br /&gt;&lt;br /&gt;If you're C-savvy, take a look at the "&lt;span class="nf"&gt;dissect_aggregate_arg" function in &lt;a href="http://trac.parrot.org/parrot/browser/trunk/src/call/args.c"&gt;src/call/args.c&lt;/a&gt; file for the actual implementation.&lt;/span&gt;&lt;br /&gt;&lt;br /&gt;&lt;blockquote&gt;As the first opcode in a subroutine that will be called with&lt;br /&gt;invokecc or a method that will be called with call_methodcc, use&lt;br /&gt;the get_params opcode to tell Parrot where the subroutine's or&lt;br /&gt;method's arguments should be stored and how they should be expanded.&lt;/blockquote&gt;&lt;br /&gt;It's interesting to me that there would be any requirement on this being the first opcode. It seems to me that we should be able to unpack the call object in any place, at any time. That's a small nit, things work reasonably well with this weird restriction in place. I think we can support a wider range of behaviors, though I won't say anything about the cost/benefit ratio of the effort needed to do it.&lt;br /&gt;&lt;br /&gt;&lt;blockquote&gt;Similarly, just before (yes, before) calling such a subroutine or&lt;br /&gt;method, use the get_results opcode to tell Parrot where the return&lt;br /&gt;values should be stored and how to expand them for your use.&lt;/blockquote&gt;&lt;br /&gt;I don't think this is the case anymore, but I do need to double-check the code that IMCC is currently generating. Obviously in a pure-CPS system we don't want to be going through this nonsense. Either Parrot does the sane thing and we need to update the docs, or Parrot doesn't do the sane thing and we need to update the design. Either way, kill this passage.&lt;br /&gt;&lt;br /&gt;&lt;blockquote&gt;If this bit [:slurpy] is set on a P register, then it will be populated with an&lt;br /&gt;aggregate that will contain all of the remaining values that have not already&lt;br /&gt;been stored in other registers.&lt;br /&gt;&lt;br /&gt;...&lt;br /&gt;&lt;br /&gt;If the named bit is not set, the aggregate will be the HLL-specific array&lt;br /&gt;type and the contents will be all unassigned positional arguments.&lt;/blockquote&gt;&lt;br /&gt;Which array type? We have several of them. If we mean ResizablePMCArray, we should say "ResizablePMCArray" so people know how to do the overriding.&lt;br /&gt;&lt;br /&gt;&lt;blockquote&gt;An I register with this bit set is set to one if the immediately preceding&lt;br /&gt;optional register received a value; otherwise, it is set to zero.&amp;nbsp; If the&lt;br /&gt;preceding register was not marked optional, the behavior is undefined; but&lt;br /&gt;we promise you won't like it.&lt;/blockquote&gt;&lt;br /&gt;Undefined behavior? In &lt;i&gt;my&lt;/i&gt; Parrot? Why can't we define what the behavior is? We can promise that the behavior will be bad, but we can't even hint about what that behavior must be? That's pretty generous of us!&lt;br /&gt;&lt;br /&gt;We could trivially identify these kinds of issues at PIR compile time, or we could catch these situations at runtime and throw an exception. Undefined behavior is precisely the kind of thing that design documents should be looking to clear up, not institutionalize. I am interested to know whether we have any tests for this, or if we could start writing some.&lt;br /&gt;&lt;br /&gt;&lt;blockquote&gt;If this bit is set on a P register that receives a value, Parrot will ensure&lt;br /&gt;that the final value in the P register is read-only (i.e. will not permit&lt;br /&gt;modification).&amp;nbsp; If the received value was a mutable PMC, then Parrot will&lt;br /&gt;create and set the register to a {not yet invented} read-only PMC wrapper&lt;br /&gt;around the original PMC.&lt;br /&gt;&lt;br /&gt;Future Notes: Parrot's algorithm for deciding what is writable may be&lt;br /&gt;simplistic.&amp;nbsp; In initial implementations, it may assume that any PMC not of a&lt;br /&gt;known read-only-wrapper type is mutable.&amp;nbsp; Later it may allow the HLL to&lt;br /&gt;provide the test.&amp;nbsp; But we must beware overdesigning this; any HLL with a truly&lt;br /&gt;complex notion of read-only probably needs to do this kind of wrapping itself.&lt;/blockquote&gt;&lt;br /&gt;Ah, something that looks pretty smart, though it's clearly listed in the PDD as "XXX - PROPOSED ONLY - XXX", which is not really a good sign. I've never been too happy with Parrot's current mechanism for marking PMCs as read-only anyway. This is a pretty interesting feature, though in current Parrot I'm not sure it could be implemented to any great effect. I may also like to see something like a ":clone" flag that forces a copy to be passed instead of a reference, or a ":cow" flag which produces a copy-on-write reference. Either way, we would probably like some kind of mechanism to specify that a caller will not be playing with data referenced by a passed PMC. This is especially true when you start to consider alternate object metamodels, or complex HLL type mappings: we don't want libraries modifying objects that the don't understand, and creating results that are going to destabilize the HLL. Having a guarantee that mistakes in the callee can't be propagated back through references to the caller would be a nice feature to have. Eventually.&lt;br /&gt;&lt;br /&gt;&lt;blockquote&gt;Named values (arguments, or values to return) must be listed textually after&lt;br /&gt;all the positional values.&amp;nbsp; fla and non-flat values may be mixed in any&lt;br /&gt;order.&lt;/blockquote&gt;&lt;br /&gt;Is this true? I see no reason in the code why named &lt;i&gt;arguments&lt;/i&gt; must be passed after positional arguments. I do see a reason why named &lt;i&gt;parameters&lt;/i&gt; must be specified after positional parameters, however. Consistency is good, but I tend to prefer that Parrot not implement unnecessary restrictions. Plus, it should be very possible for an HLL to override the default CallContext and other related PMC types and implement their own behaviors for things like ordering, overflow, and underflow.&lt;br /&gt;&lt;br /&gt;That brings me to a point of particular unhappiness with this PDD: It is extremely focused on the behavior of the combination of IMCC, PIR, and built-in data types. It's not hard, with all the Lorito talk flying around, to imagine that in a relatively short time Parrot could be PIR free: System languages like NQP and Winxed could compile directly down to Lorito bytecode, and not involve PIR at all. We could be using HLL mapped types for CallContext and other things to completely change all this behavior. Explaining what the defaults are is certainly important, and suitable for in-depth documentation. Explaining how the defaults work and interoperate with HLL overriding types, and how the system should be extremely dynamic and pluggable is absolutely missing from PDD 03.&lt;br /&gt;&lt;br /&gt;&lt;blockquote&gt;Named targets can be filled with either positional or named values.&lt;br /&gt;However, if a named target was already filled by a positional value, and&lt;br /&gt;then a named value is also given, this is an overflow error.&lt;/blockquote&gt;&lt;br /&gt;I find this a little bit confusing. Why can a named parameter be filled with a positional argument, but not the other way around? I suggest that a named parameter only takes a named argument, and a positional parameter only takes a positional argument. We should either provide other types (like :lookahead) if we want other behaviors, or we should allow the user to subclass the PMCs that implement this behavior and allow them to put in all the crazy, complicated rules that they want. Parrot defaults should be sane and general. Everything else should be subclassable or overridable.&lt;br /&gt;&lt;br /&gt;The details included in this PDD are almost as troubling as some of the details omitted. Information about signature strings, which are used everywhere and are the only way to call a method from an extension or embedding program, are completely omitted. Being able to specify a signature as a string is a central part of the current PCC implementation, so that makes no sense to me. Information about central PMC types, like CallContext is nowhere to be found either, much less information about how to override these types, and the interfaces that they are expected to implement. Making calls from extension or embedding programs using variadic argument lists is completely missing. The differences between method and subroutine invocations, the difference between invoke and invokecc opcodes (And the implications of CPS in general) is missing. MMD is never mentioned. Tailcalls and optimizations related to them is missing. Details about passing arguments and extracting parameters from continuations and exceptions is missing. New and proposed flags like :call_sig and :invocant are not mentioned. &lt;br /&gt;&lt;br /&gt;In my previous blog post, I mentioned four common problems that our PDDs suffered from. This document suffers from several. First, it's more descriptive than prescriptive, doing it's best to document what the defaults in Parrot &lt;i&gt;were&lt;/i&gt; in 2008. Second, this document is rapidly losing touch with reality as changes the the PCC system are pushing the capabilities of Parrot beyond what the document accounts for. Third, it has an extremely narrow focus on IMCC/PIR, and is vague or is completely silent about any other possibilities, especially those (such as Lorito) that may play a dramatic role in the implementation of this system in the future.&lt;br /&gt;&lt;br /&gt;PDD 03 doesn't tell our current users how to effectively use the calling conventions system of Parrot, and does nothing to direct our developers on how to improve it going forward. It really needs to be completely deleted and rewritten from the ground up. When it is, I think we will find some gold, both in terms of exposing virtues of the current implementation and describing plenty of opportunities for drastic improvement.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-1124918997839719120?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/1124918997839719120/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/09/pdd03-calling-conventions-critique.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/1124918997839719120'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/1124918997839719120'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/09/pdd03-calling-conventions-critique.html' title='PDD03 Calling Conventions Critique'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-6086245969540733758</id><published>2010-09-23T08:00:00.001-04:00</published><updated>2010-09-23T08:23:32.778-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='Parrot'/><category scheme='http://www.blogger.com/atom/ns#' term='Parrot-Linear-Algebra'/><category scheme='http://www.blogger.com/atom/ns#' term='Release'/><title type='text'>PLA: Release 1</title><content type='html'>I'm happy to &lt;i&gt;finally&lt;/i&gt; announce the &lt;a href="http://github.com/Whiteknight/parrot-linear-algebra/tree/Release_1"&gt;first official release&lt;/a&gt; of &lt;a href="http://whiteknight.github.com/parrot-linear-algebra/"&gt;Parrot-Linear-Algebra&lt;/a&gt; (PLA). PLA is an extension project for the &lt;a href="http://parrot.org/"&gt;Parrot Virtual Machine&lt;/a&gt; which brings linear algebra support and bindings to the BLAS library.&lt;br /&gt;&lt;br /&gt;PLA has several dependencies (see below). Once you have those installed properly, you can obtain and install PLA using &lt;a href="http://gitorious.com/parrot-plumage/parrot-plumage"&gt;Plumage&lt;/a&gt;, the Parrot package manager:&lt;br /&gt;&lt;br /&gt;&lt;pre&gt;plumage install parrot-linear-algebra&lt;br /&gt;&lt;/pre&gt;&lt;br /&gt;Or, if you prefer to do things the hard way, you can use this series of commands to obtain and build PLA like this:&lt;br /&gt;&lt;br /&gt;&lt;pre&gt;git clone git://github.com/Whiteknight/parrot-linear-algebra.git pla&lt;br /&gt;cd pla&lt;br /&gt;parrot-nqp setup.nqp build&lt;br /&gt;parrot-nqp setup.nqp test&lt;br /&gt;parrot-nqp setup.nqp install&amp;nbsp;&lt;/pre&gt;&lt;pre&gt;&amp;nbsp;&lt;/pre&gt;&lt;span style="font-size: x-large;"&gt;Dependencies&lt;/span&gt;&lt;br /&gt;&lt;br /&gt;Before you get started with PLA, you must install some dependencies.&lt;br /&gt;&lt;br /&gt;&lt;span style="font-size: large;"&gt;Parrot&lt;/span&gt;&lt;br /&gt;&lt;br /&gt;You must have an installed copy of the Parrot Virtual Machine, &lt;a href="http://parrot.org/news/2010/Parrot-2.8.0"&gt;version 2.8.0&lt;/a&gt; or later.&lt;br /&gt;&lt;br /&gt;&lt;span style="font-size: large;"&gt;BLAS&lt;/span&gt;&lt;br /&gt;&lt;br /&gt;PLA links to the BLAS library. You can use either the &lt;a href="http://netlib.org/blas/"&gt;standard reference BLAS library from netlib.org&lt;/a&gt;, or you can use one of the &lt;a href="http://math-atlas.sourceforge.net/"&gt;ATLAS&lt;/a&gt; or CBLAS variants. There are &lt;a href="http://en.wikipedia.org/wiki/Basic_Linear_Algebra_Subprograms#Implementations"&gt;other implementations of BLAS&lt;/a&gt;, though PLA may not currently be compatible with all of them (patches and error reports welcomed!)&lt;br /&gt;&lt;br /&gt;&lt;span style="font-size: large;"&gt;Kakapo&lt;/span&gt;&lt;br /&gt;&lt;br /&gt;&lt;a href="http://www.gitorious.com/kakapo/kakapo"&gt;Kakapo&lt;/a&gt; is a &lt;a href="http://code.google.com/p/kakapo-parrot/"&gt;development framework for the NQP language&lt;/a&gt;. It provides a unit testing library which PLA uses to implement its test suite. A special version of Kakapo is required for PLA release 1. You can use this sequence of commands to get and install that version:&lt;br /&gt;&lt;br /&gt;&lt;pre&gt;git clone git://github.com/Whiteknight/kakapo.git kakapo&lt;br /&gt;cd kakapo&lt;br /&gt;checkout tag PLA-version-1&lt;br /&gt;parrot-nqp setup.nqp build&lt;br /&gt;parrot-nqp setup.nqp install&lt;br /&gt;&lt;/pre&gt;&lt;br /&gt;&lt;span style="font-size: large;"&gt;Linux&lt;/span&gt;&lt;br /&gt;&lt;br /&gt;Sorry Windows guys! At the moment PLA only runs on Linux and other Unixy systems. Windows support is planned for future releases, but didn't make it into this one.&lt;br /&gt;&lt;br /&gt;&lt;span style="font-size: x-large;"&gt;Next Release&lt;/span&gt;&lt;br /&gt;&lt;br /&gt;PLA releases are not on a regular schedule. A new release might not come out again until new features are added or until something in the toolchain becomes incompatible.&lt;br /&gt;&lt;br /&gt;Here are a list of TO-DO features that will eventually make it into future releases:&lt;br /&gt;&lt;ol&gt;&lt;li&gt;Bindings to the LAPACK library, to add more linear algebra utilities&lt;/li&gt;&lt;li&gt;Windows support, including a Windows installer&lt;/li&gt;&lt;li&gt;Writing bindings for other programming languages which run on Parrot, including &lt;a href="http://rakudo.org/"&gt;Rakudo Perl 6&lt;/a&gt;.&lt;/li&gt;&lt;li&gt;Adding additional types, including special vector types and multi-dimensional tensor types.&lt;/li&gt;&lt;/ol&gt;If any of these things interest you, or if you have other cool ideas, please feel free to let me know and get involved!&lt;br /&gt;&lt;br /&gt;&lt;span style="font-size: x-large;"&gt;New Participants Welcome!&lt;/span&gt;&lt;br /&gt;&lt;br /&gt;Interested in PLA or linear algebra in general? You can fork a copy of PLA and start contributing &lt;i&gt;right now&lt;/i&gt;! I'm happy to talk to interested contributors, and to add patches for bugfixes and new features into the core repository. Please let me know if you're interested in helping out.&lt;br /&gt;&lt;br /&gt;If you have specific feature requests that you would like to see added to PLA, but aren't able to implement them yourself, please let me know. &lt;br /&gt;&lt;br /&gt;&lt;span style="font-size: x-large;"&gt;Thanks&lt;/span&gt;&lt;br /&gt;&lt;br /&gt;I want to offer a big thanks to the greater Parrot community for helping in various ways. PLA relies on several tools and libraries, the result of many man-hours of work by Parrot community contributors.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-6086245969540733758?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/6086245969540733758/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/09/pla-release-1.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/6086245969540733758'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/6086245969540733758'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/09/pla-release-1.html' title='PLA: Release 1'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-6535795586282962231</id><published>2010-09-21T17:00:00.114-04:00</published><updated>2010-09-26T15:32:38.538-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='Parrot'/><title type='text'>Parrot Fundamentals: Design</title><content type='html'>Several days ago &lt;a href="http://wknight8111.blogspot.com/2010/09/parrot-as-mature-platform.html"&gt;I wrote a sizable blog entry about the Parrot project&lt;/a&gt;, many of the ideas therein were discussed at length with fellow Parrot Foundation board member Jim Keenan when he came down to my area for a face-to-face meeting. Today I'm going to expand on some of those ideas, especially with respect to Parrot's design and roadmap.&lt;br /&gt;&lt;br /&gt;Earlier this year&amp;nbsp; I wrote a &lt;a href="http://wknight8111.blogspot.com/2010/02/pdd23-exceptions-critique.html"&gt;pretty negative review of PDD23&lt;/a&gt;, the exceptions system design document. Actually, that's something I really should repeat with some of the other design documents as well. There are certainly a few that deserve the treatment. Maybe I'll try to do that again soon.&lt;br /&gt;&lt;br /&gt;Anyway, there is nothing special about PDD23; it isn't an especially bad example. That's telling, actually. Looking through that directory, I find that the documents generally suffer from four common problems:&lt;br /&gt;&lt;ol&gt;&lt;li&gt;&lt;b&gt;Vague or Incomplete&lt;/b&gt;. Some PDDs are so incomplete, vague, or filled with holes that they are absolutely unusable for forming the decisions of our developers. PDD25 (concurrency) immediately comes to mind as one that contains very little actual design content. The majority of text in PDD25 consists of descriptions of technology, and reads more like an excerpt for "Threading for Dummies" than a real design document. PDD05, PDD06, PDD10, PDD14, PDD16, and maybe even PDD30 would also get put into this category. Notice that some of these PDDs are labeled as "draft" (and have been for years). I'm not really sure what a "draft" designation means, or how we go about getting them out of draft. I'll expand on that later.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Not Forward-Thinking&lt;/b&gt;. There are many cases of PDDs which act as little more than copies of function-level documentation. PDD09 is one that I am eminently familiar with which is in this category, and unfortunately &lt;i&gt;I've written much of the text in that document myself&lt;/i&gt;. At the time I rewrote it, I really didn't have much of a concept for what a design document should be, and what it could be. PDD09 describes what the GC &lt;i&gt;was&lt;/i&gt; about a year ago, and things have changed significantly since then (and are changing rapidly now). PDDs which are not forward thinking provide absolutely no direction to developers doing new work. PDD08, PDD11, PDD20, PDD24 and PDD27 are good examples of this.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Not In Sync with Reality&lt;/b&gt;. Some PDDs do not match the current implementation of things, and never really have (unlike the case above, where the implementation once matched the design, but has surpassed it). We have to ask in these situations why the design does not match the implementation: Is the design a lofty goal which the implementation is approaching by increments? Is the design unsuitable, and practical concerns have dictated a different approach? Did the implementation pre-date the design, and no attempt has ever been made to change it? PDD17 and PDD23 have some of this. PDD18 is also, by virtue of having never really been implemented. &lt;/li&gt;&lt;li&gt;&lt;b&gt;Not Good Design&lt;/b&gt;. Some PDDs really just don't represent the kind of good design that Parrot needs in the long term. Think about how often and how loudly people complain about certain topics: The object metamodel (PDD15). The lexicals implementation (PDD20). Parrot's bytecode format (PDD13). Other PDDs may fall into this category too, as the implementation approaches the design and we start to find the flaws.&lt;/li&gt;&lt;/ol&gt;The first question I suppose we need to ask ourselves as a community is this: &lt;i&gt;What is the purpose of these design documents?&lt;/i&gt; The second question we might want to ask is: &lt;i&gt;Do we want to keep these documents at all&lt;/i&gt;? A good third question is, depending on the answer to the previous question:&amp;nbsp; &lt;i&gt;How do we want to go about improving the design documents to be what we want and need them to be?&lt;/i&gt;&lt;br /&gt;&lt;br /&gt;The Parrot design documents can really be one of two things. The first is a form of summary documentation. Basically, the design documents would be a set of documents that distills what Parrot is currently capable of, and how it can be used. In other words, "interface documentation", or "man pages". A variant of this is to use the design docs as an &lt;i&gt;a posteriori&lt;/i&gt; way to justify decisions that are made off-the-cuff by developers after they've already made their commits.&amp;nbsp; A second possibility is that the design documents should be the forward-thinking technical goals of the project, the lofty goals that every commit and every release strives to reach, even if we never quite attain it perfectly. I think there is far more value in the later option, and I'll explain why. &lt;br /&gt;&lt;br /&gt;First off, we have lots of function-level documentation. We have automated tests which read our source code files and verify that we have documented (at least in a minimal way) every single function. We also have lots of tests, although admittedly tests make for pretty lousy documentation in a general sense. They can be used as a kind of reference, of course, to see how something works, but you often need to know what you're looking for before you go out to find it. We have tons of code examples too. We also have an &lt;a href="http://www.amazon.com/Parrot-Developers-Guide-Allison-Randal/dp/0977920127/ref=sr_1_1?s=gateway&amp;amp;ie=UTF8&amp;amp;qid=1284925046&amp;amp;sr=8-1"&gt;entire book&lt;/a&gt; to teach some things, though it could use some work in it's own right. Our documentation for how to use Parrot is not always great, but we do have it in plenty of other places that we don't need to use the PDDs for that purpose. If documentation is lacking, we should improve the documentation, not subvert the design documents to serve as another layer of it.&lt;br /&gt;&lt;br /&gt;Forward-thinking, lofty designs are extremely valuable. Consider the example of a coder who finds a missing feature in Parrot and no design for such a feature available. So she takes it upon herself to come up with a design and implement it. Weeks later she emerges and shows the fruits of her labors to the community with a request to merge her work into trunk. We as a community look it over, find a few problems and, with good cause, we reject it as bad design. Not just something that we can tweak to become suitable, but something that is fundamentally wrong for us. &lt;i&gt;Thanks but no thanks&lt;/i&gt;. &lt;i&gt;Smell you later, Alligator.&lt;/i&gt; That contributor will probably storm off and will never be heard from again. Also, with good cause.&lt;br /&gt;&lt;br /&gt;Now let's look at a similar example, except we as a community have done the work ahead of time of writing out our intended designs for this fancy new feature. We describe exactly what we want, and what any implementation is going to need. Our same intrepid developer follows this design, and when she emerges with her labored fruit, it is much more acceptable. With some feedback and small tweaks, it is approved and merged to trunk. &lt;br /&gt;&lt;br /&gt;I don't want to say this is super common, but it isn't unheard of either for people to show up to an open source project, unannounced, with a gigantic patch for a new feature. It also isn't completely unheard of for those gigantic patches to be summarily rejected. The more common case is where we have interested and energetic developers showing up to the Parrot project looking for problems to tackle. Saying "We don't have JIT, we could really use one" is far more daunting than "we don't have a JIT, but we have a design, and a list of prior art that we would like to model on." Following a map to your destination is much easier than having to design your own map first and then try to follow it.&lt;br /&gt;&lt;br /&gt;There's an issue of motivation too. A person is much more willing to start working on a project when they have certainty that they are doing the correct thing, and that the software they produce will be usable and desirable. It's much easier to follow a path, even a very bare one, than it is to cut your own. Not to mention that the destination is much more certain.&lt;br /&gt;&lt;br /&gt;This year, Parrot had 5 GSoC students assigned to it. Of those five, four of them contacted me personally about specific projects&lt;a href="http://wknight8111.blogspot.com/search/label/GSOCIdeas"&gt; I discussed here on this very blog&lt;/a&gt; &lt;i&gt;before submitting applications&lt;/i&gt;. I don't take any credit for anything, I'm certain Parrot would have had several high-quality applications and projects without me. But I do know that people are more quickly and easily able to latch onto fully-formed ideas more than they can attach to nebulous and vague ideas. Also, people may not even be aware that their interests and skills align with things that Parrot needs until they know exactly what Parrot needs.&lt;br /&gt;&lt;br /&gt;If we--as a community-driven open source project--want to increase the size of our developer pool (and I suggest that we should &lt;i&gt;always&lt;/i&gt; want that), we need to communicate what we need and help prospective developers align themselves with those needs.&lt;br /&gt;&lt;br /&gt;When a new person comes to the Parrot chatroom, or the Parrot mailing list, and says "I would like to get involved in a Parrot development project, what can I do?", we can say something stupid like "Look around and try to determine for yourself what needs to be done", or "We need everything". That's not helpful and not encouraging, even if it's the truth! Instead, we can say "Look, we have a &lt;a href="http://trac.parrot.org/parrot/wiki/BigProjectIdeas"&gt;list of projects&lt;/a&gt; that we've designed and prepared for, but we haven't been able to implement yet. Want to take a stab at it?" The former usually leads to a confused developer who never comes back. The later can lead to a new active, empowered, permanent member of our development team.&lt;br /&gt;&lt;br /&gt;For the sake of this discussion, we'll accept the axiom that &lt;b&gt;more active developers in the project is generally a good thing, and losing existing developers, or raising the barrier to entry so high that new developers do not join is a bad thing&lt;/b&gt;. I'll argue the point till I'm blue in the face, if anybody wants to take me up on it.&lt;br /&gt;&lt;br /&gt;In this sense, having better designs and plans means a lower barrier to entry for new users since it's easier for them to find a project to work on and they can begin work with more surety that what they are doing will ultimately be desirable and acceptable to the community. It's also a good motivator for existing community members. When I finish a project and have some (rare) spare time on my hands, it's better for me to be able to go right down to the next item on a checklist instead of having to look around blindly trying to find something that needs to be done. Sure, there are ways I could focus my search, but I still have to hope that something obvious appears in my focused search that I can work on.&lt;br /&gt;&lt;br /&gt;All that said, I think I can answer the next few questions pretty quickly:&lt;i&gt;&lt;br /&gt;&amp;nbsp;&lt;/i&gt;&lt;br /&gt;&lt;i&gt;Do we need these documents at all?&lt;/i&gt; Yes, I think we do. They can serve as an important tool to guide new and old developers alike. They can help inform and populate specific tasklists on the wiki and elsewhere, and serve as an organizational focus for teams of developers looking to improve specific areas of Parrot. Good design documents can also be used as a tool to initiate a bidirectional communications with our consumers: projects that use Parrot as an integral part of their tool chains. I'll expand on this issue in particular in a later blog post.&lt;br /&gt;&lt;br /&gt;&lt;i&gt;How do we want to improve these documents? &lt;/i&gt;The time when we can all sit back and wait for a design to magically appear from nowhere is over. Good riddance. We have plenty of people in our project who are not only capable developers, but actually pretty great software engineers and software designers. Beyond that, we have lots of people, both in our core project and throughout the ecosystem, who know all the kinds of things that are going to be necessary for projects built on Parrot to be successful. Getting enough smart minds together to tackle a design challenge should be trivial.&lt;br /&gt;&lt;br /&gt;My suggestion is this, though it's only one possible suggestion and I am not going to argue at all about details. For each design, we form a team of dedicated developers who could be considered experts on the particular topic. It would be trivial, for almost any design document we have, to put together a team of 3-4 people with some expertise in that subject. With a team, we could go through a regular checklist to produce a decent design document:&lt;br /&gt;&lt;br /&gt;&lt;ol&gt;&lt;li&gt;Survey existing research, and find papers and prior art that we like.We could do this as a community before even creating the design team.&lt;/li&gt;&lt;li&gt;Get input from the ecosystem, and maybe a special advisory panel, to get an idea for what kinds of features are required, which features would be nice, and what kinds of things to avoid.&lt;/li&gt;&lt;li&gt;Give the design team some time to prepare and present a draft to the community&lt;/li&gt;&lt;li&gt;We all paint the bikeshed for a few days&lt;/li&gt;&lt;li&gt;We accept the design (assuming we actually accept it), and start developing towards it&lt;/li&gt;&lt;/ol&gt;If we have Parrot developer design meetings (PDS) every 6 months, that would seem like a great demarcation for this kind of process. At one&amp;nbsp; PDS we identify an area that needs design help and assemble an initial team to pursue it. At the next PDS we check out the findings, maybe approve them, and start the concrete development work. If we need to look things over and push approval off, we can do it at a subsequent weekly #parrotsketch meeting.&lt;br /&gt;&lt;br /&gt;In any 6-month stretch, we are working on a set of development priorities which have already been properly designed, and we are preparing designs to work on for the next six month stretch. Our last PDS meeting was &lt;a href="http://trac.parrot.org/parrot/wiki/pds-2010-04"&gt;April 2010&lt;/a&gt;. That means we're due for one coming up in October or November if we want to stick to a 6 month schedule. I definitely think we should try to have a meeting thereabouts so we can re-focus on our current development and current &lt;i&gt;design&lt;/i&gt; priorities.&lt;br /&gt;&lt;br /&gt;TL;DR: Parrot's PDDs are in a bad state, but they really do serve an important purpose and we need to make sure they get updated. PDDs should not be short summaries of other existing documentation. Instead, they should be forward-looking documents that describe what goals we are trying to reach. We need to get input from developers and also Parrot's consumers and end-users in shaping those PDDs. I'll post more about this later.&lt;br /&gt;&lt;ul&gt;&lt;/ul&gt;&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-6535795586282962231?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/6535795586282962231/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/09/parrot-fundamentals-design.html#comment-form' title='2 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/6535795586282962231'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/6535795586282962231'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/09/parrot-fundamentals-design.html' title='Parrot Fundamentals: Design'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>2</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-8036643520709276960</id><published>2010-09-20T17:00:00.282-04:00</published><updated>2010-09-20T17:55:18.078-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='Parrot'/><title type='text'>Woe Is Parrot!</title><content type='html'>I received two quick comments to &lt;a href="http://wknight8111.blogspot.com/2010/09/parrot-as-mature-platform.htm"&gt;my blog post from yesterday&lt;/a&gt;. One of which was from Parrot developer &lt;a href="http://modernperlbooks.com/mt/index.html"&gt;chromatic&lt;/a&gt;, who always makes good points and always deserves a thoughtful reply. We did have a quick back-and-forth in the comments of that post, and he &lt;a href="http://modernperlbooks.com/mt/index.html"&gt;wrote a blog post of his own on the topic&lt;/a&gt;. Given all that, I wanted to write a follow up and try to be a little bit more clear about things. First, here's a part of my original post that drew his ire in particular:&lt;br /&gt;&lt;blockquote&gt;Name me one other mature, production-quality application VM  or language interpreter which does not support threading. This isn't an  empty set, but it's not a particularly large list either. Name me one  other application VM that does not currently sport an aggressive, modern  JIT. &lt;/blockquote&gt;His first comment reply, in full, is:&lt;br /&gt;&lt;blockquote&gt;I can name lots of VMs which fit your "Woe is Parrot!" criteria (Python,  Perl 5, Ruby, PHP).  Consider also the first few years of the JVM. &lt;/blockquote&gt;These are great examples and are extremely true, in no small part because I was not nearly specific enough in the point I was trying to make. &lt;i&gt;Mea Culpa&lt;/i&gt;.&lt;br /&gt;&lt;br /&gt;Before going any further, I want to clearly state my thesis with this series of posts and comments. It consists of three parts:&lt;br /&gt;&lt;blockquote&gt;One, Parrot isn't nearly as competitive now in  the dynamic language runtime realm as it could (and maybe should) be.  Two, we can increase our rate of improvement with focused objectives and  more developer motivation. Three, if we do that I project Parrot to be  among the top-tier of dynamic language runtime environments within the  next 1-5 years.&lt;/blockquote&gt;If you don't agree with that, stop reading. Everything that I say after this point is in direct support of that statement. Where it might appear that something I say contradicts that, I've probably mistyped something.&lt;br /&gt;&lt;br /&gt;If we look at the case of multithreading, three of the examples that chromatic listed above do have some support. &lt;a href="http://docs.python.org/library/threading.html"&gt;Python&lt;/a&gt;, &lt;a href="http://perldoc.perl.org/threads.html"&gt;Perl5&lt;/a&gt;, and &lt;a href="http://ruby-doc.org/core/classes/Thread.html"&gt;Ruby&lt;/a&gt; all do support some variety of threading, of some usability level. PHP is the odd man out in this regard, though the argument could easily be made that the webserver niche where PHP primarily lives really doesn't need and doesn't want threading. I won't expand on that topic here, but I will say this: PHP does not support threading, and is definitely a production-quality language interpreter in use by many companies. Point made.&lt;br /&gt;&lt;br /&gt;In terms of JIT, I was drawing an unnecessary and mostly fictitious line between programs which have far more similarities than differences, and I did little to clarify what I meant. So, I'll throw away that entire question.&lt;br /&gt;&lt;br /&gt;The second part of chromatic's statement is a little bit easier to respond to. The first Java Virtual Machine was released in 1995. That's 15 years ago, and even though that initial release did not stand the test of time it was temporarily considered to be the state of the art. No, Java 1.0 did not support threading as we would expect it now. But then again, in 1995 it would have been much harder to find a multicore processor capable of exploiting the scalability of threaded application. In a single-processor environment, cheap green threads were definitely a competitive and acceptable alternative to true native threading. 15 years later, multicore processors are the norm, and a threading strategy based completely on green threads is hardly acceptable by itself. This is all not to mention a dozen other subsystems in the original JVM that were immature then and would be absolutely laughable now. &lt;br /&gt;&lt;br /&gt;Trying to compare Parrot in 2010 to Java in 1995 is both telling and depressing. Sure, it's a victory for Parrot, but not one that we should ever mention again. I'm bigger and stronger than my 9 month old son. That doesn't mean I want to get into a fist fight with him (even though I would totally win).&lt;br /&gt;&lt;br /&gt;Let me pose a question though that should provoke a little bit of thought (and, I'm sure, more anger): Consider a world that had no Perl language. Larry Wall got busy and worked on other projects for 25 years, and never released Perl to the world. Then, in a flash of light and unicorns in 2010 he releases, from nowhere, a fully-formed Perl 5.12 interpreter as we know it today. Like Athena springing fully-formed from the head of Zeus. Would you use that Perl today?&lt;br /&gt;&lt;br /&gt;Would you use a language like Perl 5.12 if you hadn't already been using it, if your job wasn't using because it had been using it for years, if neither you nor anybody in your company had prior expertise in it and were not demonstrably able to work miracles with it? Would you use Perl 5.12 today if there wasn't a huge preexisting community and if there wasn't the miracle of &lt;a href="http://search.cpan.org/"&gt;CPAN&lt;/a&gt;? Would you use Perl 5.12 knowing about some of it's idiosyncrasies, the weaknesses of it's default exceptions system, the uninspiring nothingness of it's default object metamodel, or the sheer fact that in 2010 you still can't define function parameters with a meaningful function signature? Is that an investment of time, energy, and maybe money that you would make considering some of the other options available &lt;i&gt;today?&lt;/i&gt;&lt;br /&gt;&lt;br /&gt;Now, &lt;i&gt;I'm not ragging on Perl&lt;/i&gt;. I want to state that very clearly before chromatic buys a plane ticket, travels to Pennsylvania, and punches me in the back of the head. Perl 5 is a fine language and obviously doesn't exist in the kind of vacuum that I contemplate above. There is a &lt;i&gt;huge&lt;/i&gt; community of Perl programmers. There are vast amounts of institutional knowledge. There is the entirety of the CPAN. There are modules like &lt;a href="http://search.cpan.org/%7Enuffin/Try-Tiny-0.06/lib/Try/Tiny.pm"&gt;Try::Tiny&lt;/a&gt;, and &lt;a href="http://search.cpan.org/%7Edrolsky/Moose-1.13/lib/Moose.pm"&gt;Moose&lt;/a&gt;, and &lt;a href="http://search.cpan.org/%7Emschwern/Method-Signatures-20100730/lib/Method/Signatures.pm"&gt;Method::Signatures&lt;/a&gt; which can be used to build some pretty impressive modern (and even "post-modern") things on top of Perl 5's default &lt;i&gt;tabula rasa&lt;/i&gt;. On top of all that, Perl is demonstrably stable. Robust. Flexible. Usable. Coders in other languages invent terms like "Rapid Application Development" and "Rapid Prototyping" to describe what the Perl people call "a slow day at the office". People everywhere may debate the aesthetics of sigils and the multitude of operators, but nobody questions the fitness of Perl for use as a programming tool. It's competency and utility are unassailable. &lt;br /&gt;&lt;br /&gt;Here's my point: Take a look at the Perl 5 base technology. Take a serious, hard look at it. At the very most, the stand-alone Perl 5 interpreter is flexible, but &lt;i&gt;technologically unimpressive&lt;/i&gt;. Nothing that the base Perl interpreter provides is the jaw-dropping, nerd-orgasming state-of-the-art. I could point to a dozen performance benchmarks that pit modern Perl 5 against modern Python, Ruby, Java, and whatever else, and Perl 5 almost always comes in dead last (notice that we're not benchmarking the time it takes to write the code). That's fine. It is in the context of history, community and ecosystem that Perl 5 becomes a strong competitive force in the world of computer programming. We know that a great Perl coder can write more functionality in one line of apparent gibberish than a Java coder can write in a whole page of code. We know that the same great Perl coder can write his solution &lt;i&gt;faster&lt;/i&gt;. It is because people have written the tools and modules, that people have identified best practices, that people can do so much in so little time, and because people have taken the time to distill down to the elegant essence of Modern Perl, that we love and use Perl. &lt;br /&gt;&lt;br /&gt;The problem I identify in Parrot is a bootstrapping problem. Perl 5 has plenty of reasons to use it besides just performance and a technical feature set. Parrot does not. Parrot needs to provide a massive, overwhelming technological impetus for new developers to use it and to get involved with. Attracting those new developers further accelerates the pace of development, both of the core VM and the ecosystem. All these improved components, in turn, attract more people. It's a bootstrapping problem, and Parrot needs something compelling to get the cycle started.&lt;br /&gt;&lt;br /&gt;Make a great product. Attract more minds. Develop a bigger ecosystem. Build a better product. Repeat.&lt;br /&gt;&lt;br /&gt;In his blog post, chromatic takes exception to the word "mature" I used in the previous post. I won't use that word any more. In his comments, he also expressed a dislike of the word "enterprise". I won't use that word either. They were probably bad choices.&lt;br /&gt;&lt;br /&gt;In his blog post, chromatic says :&lt;br /&gt;&lt;blockquote&gt;His argument is that a focus on threading and a focus on JIT is  necessary for enterprises or language communities to consider Parrot a  useful platform.  &lt;br /&gt;I can see his point, and yet (as usual) I challenge the terms of the debate itself.&lt;/blockquote&gt;Do challenge it. That's not really what I said. I mentioned the two particular cases of threading and JIT as things that I think Parrot is going to need to be competitive in the world of 2010. Perl 5, and Python, and Ruby, and all sorts of other things that chromatic mentions don't have both these things, so the counter-argument appears that none of these are suitable, competitive platforms either. That's not what I said either. Keeping up with the comparison I've been trying to make between Perl 5 and Parrot, here is a summary view of what both bring to the table:&lt;br /&gt;&lt;ul&gt;&lt;li&gt;&lt;b&gt;Perl 5&lt;/b&gt;: Reasonable, but not blockbuster performance. &lt;i&gt;Huge&lt;/i&gt; ecosystem of modules, add-ins, tools, and applications written in Perl. &lt;i&gt;Huge &lt;/i&gt;preexisting developer base with large amounts of institutional knowledge. Long and storied history of robustness, stability and fitness. Institutional inertia (people using Perl next year, at least partially because they have been using it this year).&lt;/li&gt;&lt;li&gt;&lt;b&gt;Parrot&lt;/b&gt;: Reasonable, but not blockbuster performance. &lt;i&gt;Extremely small&lt;/i&gt; (but growing) ecosystem of dependent projects. &lt;i&gt;Extremely small&lt;/i&gt; (but growing) preexisting developer base. &lt;i&gt;Very little &lt;/i&gt;history of Parrot being stable and robust, considering the huge changes that the project has to make on a regular basis to improve itself.&lt;/li&gt;&lt;/ul&gt;So if you're a developer, or a manager, a graduate student, or a hobbyist, or anybody else who has a great idea and is looking for a platform on which to implement it, which of these two would you choose? I'll give you a hint: If you reach into your pocket for a coin to flip, or reach to the shelf for a magic 8-ball to help answer the question, you probably need to re-read the choices more carefully.&lt;br /&gt;&lt;br /&gt;JIT is a feature that Parrot can use to set itself apart from the pack, not something that's a necessary requirement to join in. JIT is a leg up that Parrot can use to gain some traction against another runtime environmnt like Perl 5, Python 3, Ruby, or PHP 5 which have so many compelling stand-apart features of their own. Stable and scalable threading is another one. And Parrot needs to be &lt;i&gt;fast&lt;/i&gt;. When groups like Facebook are talking about compiling PHP code down to C, you know that performance is an issue in the world of dynamic languages. It is foolhardy to think Parrot can succeed (for any definition thereof) without dramatically improved performance over what it offers now.&lt;br /&gt;&lt;br /&gt;In the end, this is really a fallacious argument anyway. I'm sure chromatic has pointed that out by now. Parrot isn't a language like Perl 5 is a language, so the two aren't really comparable in a direct way. Parrot doesn't target the same kind of audience that Perl 5 targets. Parrot targets people like the ones who make Perl 5.&lt;br /&gt;&lt;br /&gt;The idea of porting Perl 5 to run on top of Parrot was once kicked around in a semi-serious kind of way. I don't remember what number it was exactly, I think it was something like the 5.14 release was supposed to be running happily on top of Parrot. Let's revive that discussion a little bit. What kind of feature set would Parrot need to have to make a compelling argument for the Perl 5 development team to focus their energy whole-hog on porting to Parrot instead of improving Perl 5 in place? What kinds of tools would Parrot need to provide to smooth the way? If I want to see Perl 5.98 be released on Parrot, what do we need to do to make that happen? In answering this, I'm more interested in hearing about the shortcomings of Parrot (which I can work to fix) than the shortcomings of Perl (which I will not).&lt;br /&gt;&lt;br /&gt;Rumors have been floating around for over a year now about a complete rewrite of PHP called PHP 6. They want unicode support built-in. We have unicode support built-in. What do we need to do to make a compelling argument in favor of building a new PHP 6 language on top of Parrot? What do the PHP designers and developers need to see to be convinced that Parrot is the way to go, instead of pulling out the old C compiler and starting from scratch?&lt;br /&gt;&lt;br /&gt;In 5 years when maybe Python and Ruby people are looking to rewrite their languages, what do we need to have on the table to convince them to use Parrot as a starting point.&lt;br /&gt;&lt;br /&gt;These are the important questions. Nothing else really matters. If language designers and compiler developers don't use Parrot and &lt;i&gt;don't want to use Parrot, &lt;/i&gt;we've lost.&lt;br /&gt;&lt;br /&gt;Parrot needs honest, constructive criticism. It is neither offensive nor overly aggressive to provide it. We need to set aggressive, but realistic goals as a team. There are several planned parts of Parrot that need to be implemented, and several existing parts that need to be re-implemented. Good goals will help to inform those designs and tune those implementations. Eventually, our wildest dreams can become reality.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-8036643520709276960?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/8036643520709276960/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/09/woe-is-parrot.html#comment-form' title='7 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/8036643520709276960'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/8036643520709276960'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/09/woe-is-parrot.html' title='Woe Is Parrot!'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>7</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-4773764426012613762</id><published>2010-09-19T08:42:00.001-04:00</published><updated>2010-09-19T08:43:01.072-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='Parrot'/><title type='text'>Parrot as a Mature Platform</title><content type='html'>Yesterday I met with Jim Keenan, fellow neophyte on the Parrot Foundation board. We got together for an informal get-together yesterday at a Barnes and Nobel, and had a highly productive little chat about Parrot: the foundation, the culture, the community, and the software. Obviously no "official" business happened at this little meetup, but we did get to know each other better and discussed a number of things. In this post, and others in the coming days, I'm going to talk about some of the points that came up in this meeting.&lt;br /&gt;&lt;br /&gt;I've said many times, on this blog and elsewhere, that I don't think  Parrot is currently a mature platform. It is certainly not suitable for  use in a professional, production environment. I mentioned this sentiment to Jim, and he asked for some clarification. What did I mean by that?&lt;br /&gt;&lt;br /&gt;Let me illustrate with a few examples.&lt;br /&gt;&lt;br /&gt;You're employed as a system designer, and are preparing sketches for your companies next-generation software product. The success or failure of your design will have extreme effects on your current position, and maybe even on your career in the long term. In short, the design needs to be solid, flexible, expandible, robust, and all sorts of other good things. So here's the question: Do you choose Parrot as the basis for your new system, right now? If not, why?&lt;br /&gt;&lt;br /&gt;Thought about it? Let me share a few answers with you, in the form of more questions. Name me one other mature, production-quality &lt;a href="http://en.wikipedia.org/wiki/Virtual_Machine#Process_virtual_machines"&gt;application VM&lt;/a&gt; or language interpreter which does not support threading. This isn't an empty set, but it's not a particularly large list either. Name me one other application VM that does not currently sport an aggressive, modern JIT.&lt;br /&gt;&lt;br /&gt;Here's another example. Go to Google Scholar and search for papers and patents involving virtual machines. What percentage of the resulting papers use the &lt;a href="http://en.wikipedia.org/wiki/HotSpot"&gt;Java Hotspot VM&lt;/a&gt; as the basis for their research? The .NET CLI? Smalltalk? Now take a closer look and count up how many results are based on Parrot? How many papers even mention Parrot in the footnotes?&lt;br /&gt;&lt;br /&gt;You're a graduate student pursuing a PhD in CompSci. You want to spend the next three years of your life researching some new feature on virtual machines, the results of which will have major effects on your career and maybe will even influence whether you graduate or not. Do you choose to implement and study your fancy new feature on Java Hotspot, or on Parrot? Why?&lt;br /&gt;&lt;br /&gt;Here's another example from a different direction. You work as the president of a large company which is reasonably sympathetic to the goals of the Parrot project. A Parrot contributor comes forward with a grant proposal to implement an exciting new feature. Do you cut a grant a sizable grant to support that developer through the production of that feature on Parrot, or do you spend your money elsewhere? In other words, do you expect to see a return on your investment, and do you expect that money spent on Parrot &lt;i&gt;as it currently is&lt;/i&gt; will be well and efficiently spent? If you were doing your research and read over the long-term Parrot design, and if you were looking at the current state of the Parrot community and the community leadership, would you feel comfortable and confident to invest in it? Why or why not?&lt;br /&gt;&lt;br /&gt;Turning that same example around a little bit, pretend that you're &lt;i&gt;me.&lt;b&gt; &lt;/b&gt;&lt;/i&gt;As a foundation board member, I'm going to take that grant request from the developer, put together all the necessary paperwork and approach a philanthropist with it. What do you think are the odds that I would get laughed out of the room and told never to come back?&lt;br /&gt;&lt;br /&gt;Even though it's been 10 years since the start of it, Parrot really is a young project. Our long-term designs and goals are lacking. We have some extremely talented, enthusiastic, and energetic contributors, but we don't always do a great job of organizing and motivating them. There are plenty of areas where we do pretty well, but I can't think of a single aspect of the project or the community that we couldn't tune and improve. How much better do you want Parrot to be?&lt;br /&gt;&lt;br /&gt;I want to be very clear about one thing here: I am &lt;i&gt;not&lt;/i&gt; being insulting or disparaging about Parrot. It is not an insult to say that Parrot is not ready for enterprise-level production deployment. It is not disparaging to say that Parrot isn't a sure bet to make when careers and livelihoods are on the line. What we do need is honest self-assessment, and to use that as a basis for making long term plans and goals. &lt;br /&gt;&lt;br /&gt;Starting from that honest self-assessment, we can start asking the important questions. Where do we go from here?&lt;br /&gt;&lt;br /&gt;Hypothetically, we approach the Python Foundation and say "in 5 years, we want Parrot to be at a level where your premier, standard Python interpreter implementation could be implemented on top of it. What would you need to see in order to comfortably make that kind of decision in favor of Parrot?" Ask the same question of the Ruby, PHP, Perl 6, and Perl 5 interpreters. What does Parrot need to do in order to convince the leaders, architects, and developers of these projects that Parrot is a modern, competitive and even a desirable platform on which to build the next generation of their software? How long do people reasonably think it would take for Parrot to get into that condition? 1 year? 3 years? 5 years? There are no wrong answers, but the more honest we are with ourselves, the more certain we can be laying out a comprehensive roadmap to get us there.&lt;br /&gt;&lt;br /&gt;In the coming days I'll address some of these issues. The point of this post was to get people thinking, dreaming, and doing both &lt;i&gt;big&lt;/i&gt;.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-4773764426012613762?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/4773764426012613762/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/09/parrot-as-mature-platform.html#comment-form' title='8 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/4773764426012613762'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/4773764426012613762'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/09/parrot-as-mature-platform.html' title='Parrot as a Mature Platform'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>8</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-8922452152436940084</id><published>2010-09-15T20:47:00.000-04:00</published><updated>2010-09-15T20:47:04.074-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='Smolder'/><category scheme='http://www.blogger.com/atom/ns#' term='Testing'/><category scheme='http://www.blogger.com/atom/ns#' term='Parrot'/><category scheme='http://www.blogger.com/atom/ns#' term='Parrot-Linear-Algebra'/><title type='text'>Parrot has Smolder Again!</title><content type='html'>Parrot's smolder server, previously hosted by plusthree.com, has been down for some days. Today, due to some concerted effort by particle, dukeleto, and the friendly folks at OSUOSL got a new instance of smolder set up for use by Parrot.&lt;br /&gt;&lt;br /&gt;I introduce you to &lt;a href="http://smolder.parrot.org/"&gt;http://smolder.parrot.org&lt;/a&gt;&lt;br /&gt;&lt;br /&gt;Reports for Parrot proper are already flying in, but what makes this smolder server so special is that we can add support for other projects as well. Half a dozen other projects are also able to tracked on Smolder: &lt;a href="http://smolder.parrot.org/app/projects/smoke_reports/2"&gt;PLA&lt;/a&gt;, Lua, PLParrot, Partcl, Rakudo, and Plumage. Not all of these projects have the necessary infrastructure to perform the actual uploads yet, but within a few days I'm sure they all will.&lt;br /&gt;&lt;br /&gt;Tonight I updated the PLA setup program and test harness to support uploads to smolder, and a few minutes ago I posted the first automated report. We posted a few test reports manually before that too. Everything is looking good so far, though I do have a few tweaks to make. Specifically, I want to include more information about the version of BLAS and LAPACK that are used in the report, which should be easy enough.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-8922452152436940084?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/8922452152436940084/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/09/parrot-has-smolder-again.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/8922452152436940084'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/8922452152436940084'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/09/parrot-has-smolder-again.html' title='Parrot has Smolder Again!'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-5666160939004086975</id><published>2010-09-10T22:15:00.001-04:00</published><updated>2010-09-11T20:45:29.631-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='Parrot'/><category scheme='http://www.blogger.com/atom/ns#' term='Parrot-Linear-Algebra'/><title type='text'>PLA Documentation Is Up</title><content type='html'>I've finally gotten some &lt;a href="http://whiteknight.github.com/parrot-linear-algebra/"&gt;online documentation for PLA up at Github Pages&lt;/a&gt;. The colors are the same standard dark scheme that I use for everything.&amp;nbsp; There are some things in this world that I consider myself very good at, but graphic design and effective use of colors are not in that list. If somebody with a better grasp of the color wheel would like to take a stab at a redesign, I would be very happy to accept patches.&lt;br /&gt;&lt;br /&gt;Documentation for PLA had been written in &lt;a href="http://en.wikipedia.org/wiki/Plain_Old_Documentation"&gt;POD&lt;/a&gt;, and embedded directly in the PMC source files. This is a decent system, and is what Parrot uses, but I was becoming unhappy with it. The problem I was having was that even though converting POD to HTML is supposed to be a well-understood and oft-solved problem, I couldn't find a converter I liked that produced good-looking output. Also, I was having a hell of a time finding a tool that would give me any kind of flexibility with how the output HTML was generated, formatted, or organized. The output of the standard pod2html is horrendous, and I've found it to be very difficult to style without modifying the generated HTML by hand.&lt;br /&gt;&lt;br /&gt;I also am not really too happy with the way POD embeds in source code. It's too abusive of vertical space, and causes files to become completely bloated. Not to mention the fact that I don't think it's really the right solution to the problem. I could have looked into something like &lt;a href="http://en.wikipedia.org/wiki/Doxygen"&gt;Doxygen&lt;/a&gt;, but that's only marginally better. Sure, Doxygen uses less vertical space in a file, but it's still embedded documentation that attempts to do more than it should. Documentation for prospective users should be different than documentation for prospective hackers. If it's not different, I can guarantee that one of the two groups is getting the short end of the stick. If you have documentation for hackers (as you would expect to find embedded in the C source code file), the users are going to be stuck sifting through page after page of internals minutia. What the users really want to see is information about the interface and how to use the tools, not details about every function in the C code file.&lt;br /&gt;&lt;br /&gt;I thought about writing a POD parser in NQP that would do what I wanted it to do, but by the time I realized how much effort that would take me, I had basically decided against using POD entirely. I do think that a standalone POD parser written in a Parrot-targetting language (NQP comes to mind immediately) would be a good thing, but I am not inclined to make it myself just yet.&lt;br /&gt;&lt;br /&gt;I thought about adding separate POD files to form user documentation, separate from the hacker documentation embedded in the source code. However, this didn't sit well with me either. First, as I mentioned above, the generated HTML I could get always looked terrible (and I couldn't find any compelling alternate tool which might generate better-looking pages) and it didn't really give me the flexibility that I wanted to have. It got to the point that it would have taken me more effort to write the tool I needed to get the resulting output that I wanted than it would have taken me just to rewrite the documentation using a different markup language.&lt;br /&gt;&lt;br /&gt;Finally I decided to embrace Github Pages wholesale. Github Pages uses the &lt;a href="http://github.com/mojombo/jekyll"&gt;Jekyll preprocessor&lt;/a&gt;, which takes input in &lt;a href="http://en.wikipedia.org/wiki/Textile_%28markup_language%29"&gt;Textile&lt;/a&gt;, &lt;a href="http://en.wikipedia.org/wiki/Markdown"&gt;Markdown&lt;/a&gt;, or HTML. It gives me a lot more flexibility to break documentation up into arbitrary little chunks and keeps pages themed in a consistent way. I decided on Textile, which in my mind is easier to read and write than I ever found POD to be. So I rewrote most of the documentation in Textile with some Jekyll processor magic thrown in, and I'm pretty happy with the result.&lt;br /&gt;&lt;br /&gt;So happy in fact that I've considered maybe changing my entire blog format over to using Jekyll and Textile. I'm not ready for that kind of changeover just yet, but it is something I probably want to look into eventually. &lt;br /&gt;&lt;br /&gt;Along with some of the fixes I've made to the PLA test harness and test coverage in the last few days, this is basically one of &lt;a href="http://wknight8111.blogspot.com/2010/09/pla-release-is-near.html"&gt;the last requirements I had&lt;/a&gt; for putting out a new release of PLA. I now feel like I'm prepared to cut the release shortly after Parrot 2.8.0 comes out.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-5666160939004086975?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/5666160939004086975/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/09/pla-documentation-is-up.html#comment-form' title='4 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/5666160939004086975'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/5666160939004086975'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/09/pla-documentation-is-up.html' title='PLA Documentation Is Up'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>4</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-90988010013449375</id><published>2010-09-07T17:00:00.000-04:00</published><updated>2010-09-07T17:00:02.726-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='Testing'/><category scheme='http://www.blogger.com/atom/ns#' term='Parrot'/><category scheme='http://www.blogger.com/atom/ns#' term='Parrot-Linear-Algebra'/><category scheme='http://www.blogger.com/atom/ns#' term='Release'/><title type='text'>PLA Release Is Near</title><content type='html'>I added in support for HLL mapping of autboxed types in &lt;a href="http://github.com/Whiteknight/parrot-linear-algebra"&gt;Parrot-Linear-Algebra&lt;/a&gt;, and with that I feel like I'm getting pretty close to a good point for cutting a release. I don't yet have any tests for the autoboxing behavior, so I do need to write those first. Shortly thereafter I think I can get to work on the release.&lt;br /&gt;&lt;br /&gt;HLL mapping, for readers who may not be familiar with the term, is a very cool feature in Parrot. It allows the user to manually re-map basic types to user-defined types instead of the built-in varieties. When the VM would normally create an Integer PMC, for instance, it would instead create a custom "MyInteger" type (or whatever you called it). You can use the HLL mapping functionality to override many built-in types in many operations. What's super-cool about HLL mapping is that the mapping is defined in a particular HLL namespace, so programs with modules written in different languages could allow each module to define it's own type mappings that do not conflict with each other.&lt;br /&gt;&lt;br /&gt;In Parrot, an "HLL namespace" is a special type of top-level namespace object which allows the use of type mappings, among other things. In PIR code, defining an HLL is simple with the .HLL directive:&lt;br /&gt;&lt;br /&gt;&lt;pre&gt;.HLL "MyLanguage"&lt;br /&gt;&lt;/pre&gt;&lt;br /&gt;All regular namespaces defined below that directive will be inside the HLL namespace. This means that so long as we make proper use of .HLL directives, we can maintain almost perfect encapsulation between modules written in different languages, which can be an extremely valuable thing for proper interoperation.&lt;br /&gt;&lt;br /&gt;The default HLL is called "parrot". If you don't specify an HLL directive, you will automatically be inside the "parrot" HLL namespace. To get back to there, you can type (in PIR):&lt;br /&gt;&lt;br /&gt;&lt;pre&gt;.HLL "parrot"&lt;br /&gt;&lt;/pre&gt;&lt;br /&gt;I started writing some tests on Saturday, and discovered two problems that brought me to a halt: First, NQP doesn't have any built-in way to specify an HLL namespace. I also wasn't able to find any crafty, sneaky way to inject one either. Second, HLL type mapping doesn't work in the parrot namespace.&lt;br /&gt;&lt;br /&gt;The second problem turned out to be the most frustrating because the test programs I was writing were silently failing for no visible reason. The mapping method appeared to execute and return correctly, but none of my types were being mapped. Dutifully, &lt;a href="http://trac.parrot.org/parrot/ticket/1771"&gt;I filed a bug about it&lt;/a&gt;. It turns out that this is by design, not accident, although I wish that it had been a little bit better documented. HLL mapping lookup operations can be a little bit expensive, so we don't want to be doing that all the time. Also, the parrot HLL is supposed to be a default, neutral space and one module shouldn't be able to break encapsulation and modify that HLL namespace in a way that would adversely affect other modules written in other languages. NotFound put together a commit that throws an exception when a type mapping is attempted in this HLL, so that allays my concerns that not only was it failing, but it was failing silently.&lt;br /&gt;&lt;br /&gt;I also started &lt;a href="http://github.com/Whiteknight/nqp-rx/commit/0156e5d76bafea8d87d0db8b8790ab15311f93b5"&gt;putting together a patch in my fork&lt;/a&gt; of NQP-RX that adds HLL support, but the patch isn't very mature yet. If I can get this patch ready and merged into NQP-RX master in time for the 2.8.0 release, I will write up the last remaining PLA tests for this feature in NQP. Otherwise, I will write them in PIR.&lt;br /&gt;&lt;br /&gt;The PLA release is going to target Parrot 2.8.0. There are a few things that I want to do first, before the release is out the door:&lt;br /&gt;&lt;ol&gt;&lt;li&gt;Finish writing the tests for the HLL mapping behavior, which might involve finishing up that patch to NQP&lt;/li&gt;&lt;li&gt;Write up some decent public documentation. The default output of pod2html is pretty ugly looking, so I may end up writing a custom converter. I've started experimenting with Github Pages, though my experiments so far in using the pod2html output there have not been too attractive. I may go through and reorganize all the POD documentation source anyway. I definitely want to expand documentation and examples of certain features.&lt;/li&gt;&lt;li&gt;Get a release, or pseudo-release of &lt;a href="http://gitorious.org/kakapo/kakapo"&gt;Kakapo&lt;/a&gt; that targets Parrot 2.8.0. If Austin isn't able to get a working release of that software that's up to his standards in the next two weeks, I may pick a revision that works well for my purposes and tag it on &lt;a href="http://github.com/Whiteknight/kakapo"&gt;my Github fork&lt;/a&gt;. It won't be the same as a real release of Kakapo, but at least it won't be a stumbling block for me.&lt;/li&gt;&lt;li&gt;I need to check and double-check that the setup script for PLA is doing all the correct things with respect to releasing. I need to check that the generated &lt;a href="http://gitorious.org/parrot-plumage/parrot-plumage"&gt;Plumage&lt;/a&gt; metadata is correct and allows complete and functional installations using Plumage. I also need to check that I can generate correct .deb and .rpm packages for those systems.&lt;/li&gt;&lt;li&gt;I want to look into creating a windows installer, but I make absolutely no promises about that. I certainly haven't done any testing whatsoever on Windows so far, and I do not have high hopes that it will work at all there. This may be a task for the next release, or later.&lt;/li&gt;&lt;/ol&gt;In the span of about two weeks, we could have a release of a high-performance linear algebra toolkit for Parrot. It obviously doesn't have a huge amount of functionality yet, but it is a good start and provides a solid base of some important standard operations. I've got a lot of plans for the future of this little project, but we're at a good point right now and I think it will make for a very nice release.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-90988010013449375?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/90988010013449375/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/09/pla-release-is-near.html#comment-form' title='1 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/90988010013449375'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/90988010013449375'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/09/pla-release-is-near.html' title='PLA Release Is Near'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>1</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-3335636734079779669</id><published>2010-09-06T21:45:00.000-04:00</published><updated>2010-09-06T21:45:41.298-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='Parrot'/><category scheme='http://www.blogger.com/atom/ns#' term='ParrotFoundation'/><title type='text'>Parrot Foundation: Current Work</title><content type='html'>I've only been a member of the Parrot Foundation board for a few days now, and I already feel exhausted by it. I've read through dozens of pages in IRS &lt;a href="http://www.irs.gov/pub/irs-pdf/p557.pdf"&gt;Publication 557&lt;/a&gt;, I've been pouring over several incomplete drafts of &lt;a href="http://www.irs.gov/pub/irs-pdf/f1023.pdf"&gt;Form 1023&lt;/a&gt;. I'm not certain that either of these two documents are written completely in English.&lt;br /&gt;&lt;br /&gt;Luckily for me, because of the time I spent in the Wikimedia Chapters Committee, and because of some of the work I did with &lt;a href="http://meta.wikimedia.org/wiki/Wikimedia_New_York_City"&gt;Wikimedia NYC&lt;/a&gt; and the now-defunct &lt;a href="http://wikinortheast.blogspot.com/2007/06/new-name-wmf-pennsylvania.html"&gt;Wikimedia Pennsylvania&lt;/a&gt;, I'm not unfamiliar with this documentation or the process of gaining tax exempt status for a US non-profit corporation. It has been at least a year or two since I've really looked at this kind of paperwork, however, so I'm taking the crash-course to get back up to speed with it all. Some of it does seem more daunting than I remember! &lt;br /&gt;&lt;br /&gt;I've also been digging through invoices and financial records, and lots of other paperwork as well. Some of the information and documentation that I need is readily available. Some of it might not be (or, I may be looking in all the wrong places). Either way, I'm setting a break-neck pace trying to get through it all and form a comprehensive view of the current state of the Parrot Foundation.&lt;br /&gt;&lt;br /&gt;My goal is to put together a report about the current legal and financial state of the Foundation, and present that report to the other directors at the end of the month so we can start coming up with a plan of action for the coming year. Shortly thereafter I would love to send a status update to the general foundation membership, so we can make sure everybody is well informed about the state of the foundation and also so we can solicit some input about the future direction of it.&lt;br /&gt;&lt;br /&gt;I will definitely post updates, either here or to the members mailing list as I get more information. However, don't expect too much until I am ready with my full report, probably around the end of September. As a member of the board I have a lot of plans for the year, but I don't want to start anything until I make sure the foundation is on a solid footing with all the right documents properly filled out and filed with all the right organizations. It might be a big job, but it's one that I hope to complete by the end of 2010.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-3335636734079779669?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/3335636734079779669/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/09/parrot-foundation-current-work.html#comment-form' title='1 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/3335636734079779669'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/3335636734079779669'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/09/parrot-foundation-current-work.html' title='Parrot Foundation: Current Work'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>1</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-3327671209738057979</id><published>2010-08-31T20:56:00.000-04:00</published><updated>2010-08-31T20:56:41.701-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='Parrot'/><category scheme='http://www.blogger.com/atom/ns#' term='ParrotFoundation'/><title type='text'>Parrot Foundation Board</title><content type='html'>Today at the weekly Parrotsketch meeting there was also a meeting of members of the Parrot Foundation, with the aim of electing board members for the coming year. All but one of the current board members chose not to run for re-election, so it was a pretty open field.&lt;br /&gt;&lt;br /&gt;Four candidates were nominated, and today all of them were elected by simple majority: Jerry Gay (particle), Jim Keenan (kid51), Jonathan Leto (dukeleto), and myself.&lt;br /&gt;&lt;br /&gt;&lt;br /&gt;Here's a brief discussion of what I want to do this year as a Parrot Foundation board member:&lt;br /&gt;&lt;br /&gt;&lt;ol&gt;&lt;li&gt;Re-read and re-reread the foundation bylaws. I've read through them a while ago back when the foundation was first forming, but don't think I've looked at them since. Plus, I've read through so many sets of bylaws for so many young organizations in my time, the details all run together in my mind.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Money&lt;/b&gt;. The foundation doesn't have a whole ton of money, of course I can't think of too many non-profits of this size which do. There are lots of things that can be done with money: grants and bounties being two that immediately come to mind, but there are plenty of other things. Finding ways to raise money and increase the PaFo coffers should probably be a pretty big priority for us.&lt;/li&gt;&lt;li&gt;Create a proper membership committee. Parroteer "smash" has been running the elections and doing a fantastic job, but some problems have been exposed in the membership process. Many people do not know whether they are members or not, and many people are confused by how a person becomes a member. Clearing up confusion in this area will help everybody.&lt;/li&gt;&lt;li&gt;Recruit new people. People are the fuel that runs an open-source project, and you can never have too many people. Parrot is by no means a small open source project, but it is far from being a big one. More developers create more/better software, which attracts more end users, which increases the prestige of the project, which attracts more developers, which.... It's a feedback loop that we should be trying much harder to feed.&lt;/li&gt;&lt;/ol&gt;These are just four things that I would like to focus on this term, however this is not a definitive list. I am hoping to get, within the next few days, some information from the current board members about what kinds of tasks they have left unfinished, or what kinds of things they would try to accomplish if given another term.&amp;nbsp; Since there is so much new blood on the committee, we need to make sure to thoroughly pick the brains of the current board so that we can get this coming year off to a running start.&amp;nbsp;&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-3327671209738057979?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/3327671209738057979/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/08/parrot-foundation-board.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/3327671209738057979'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/3327671209738057979'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/08/parrot-foundation-board.html' title='Parrot Foundation Board'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-7276084271113741673</id><published>2010-08-29T08:00:00.000-04:00</published><updated>2010-08-29T08:00:00.245-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='CodeAnthem'/><title type='text'>Code Anthem: What Can Change</title><content type='html'>Friday night &lt;a href="http://wknight8111.blogspot.com/2010/08/code-anthem.html"&gt;I wrote an introduction&lt;/a&gt; to a site called &lt;a href="http://www.codeanthem.com/"&gt;Code Anthem&lt;/a&gt;, which tests the skills of a programmer and uses those results to help employers find talented individuals. As I said then, I think the idea is a pretty awesome one, and I think it has a lot of potential to help employers with the tricky technical screens that so many companies get wrong.&lt;br /&gt;&lt;br /&gt;Today I'm going to talk a little bit about what I think Code Anthem can do differently, and how they can grow the service into something that would have real, industry-wide value.&lt;br /&gt;&lt;br /&gt;The tests at Code Anthem tend to focus pretty narrowly on algorithms: Determine whether one array is a subset of another array, Determine if a string is a palindrome, Calculate the perimeter of a polygon, Validate a password. These are all relatively simple things and once you know the necessary algorithm it tends to be a simple matter of translating it into the target language. With a little bit of a brush up for me to re-familiarize myself with the core classes and methods, I could probably do as well in the Java test as I did in the C# test, and I absolutely would not consider my mad Java &lt;i&gt;skillz&lt;/i&gt; to be at the same level that my C# abilities are. I was decent with the language in college when I was using it, but skills do fade if you don't use them and I have certainly not been using Java.&lt;br /&gt;&lt;br /&gt;A test for problem-solving and basic language syntax tests exactly that: problem solving and basic language syntax.&amp;nbsp; One question involved calculating the Fibonacci sequence, and while my solution was pretty efficient, I could easily have used the &lt;a href="http://blogs.msdn.com/b/ericlippert/archive/2004/05/19/135392.aspx"&gt;classic, naive recursive implementation&lt;/a&gt; and thrown performance considerations to the wind. Would this provide me the correct answers to test input? yes. Would this be an acceptable answer? Absolutely not. So, testing that I can solve a problem which has several known solution algorithms doesn't necessarily mean that I will pick a good one, and doesn't necessarily mean that I've solved the problem &lt;i&gt;well&lt;/i&gt;.&lt;br /&gt;&lt;br /&gt;Looking at this another way, I know that the recursive solution (which is perfectly acceptable input) starts to go hell-crazy above a certain input level. Assuming that Code Anthem isn't evaluating solutions on a supercomputer at the NSA, it's pretty reasonable to assume that they aren't going to be testing the solution with inputs above 30ish, and absolutely no inputs above 45. With this devious knowledge in mind, I don't need an algorithmic solution at all, all I need to do is provide an answer that must work for the set of inputs that they can reasonably use to test:&lt;br /&gt;&lt;br /&gt;int fib(int idx) {&lt;br /&gt;return (new int[] {1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, ...})[idx - 1];&lt;br /&gt;}&lt;br /&gt;&lt;br /&gt;Again, not an acceptable solution to the problem under any circumstances, but it does work and it does require a little bit more insight than the recursive solution does.&lt;br /&gt;&lt;br /&gt;Any programmer who really thinks that the naive recursive solution to the Fibonacci problem is an acceptable one really should fail the test no matter how accurate the results are. It may be technically correct, but it is an example of extremely poor problem solving, lazy coding, and a complete disregard for real-world performance issues. It would be relatively trivial to run tests in a sandbox thread with a timed kill-switch: If the timer goes off before a solution is found, the complexity must have been too high and the result is failure. &lt;br /&gt;&lt;br /&gt;I've digressed a little bit: If you're only testing problem-solving and an ability to dump a solution from your brain into the syntax of the target language, you're not testing much. I've known a lot of really lousy programmers who would be able to solve many of these problems, even if the quality of the final solution was severely lacking.&lt;br /&gt;&lt;br /&gt;When I taught Embedded C coding in college, I used to hear from students all the time that their grades should have been higher because "it compiles", and somehow that always seems good enough. The reality is that compilation is the milestone that separates a file of source code from a file of unintelligible gibberish. Just compiling means that what you wrote is software, but not that it's correct, robust, maintainable, or even acceptable. To earn these other distinctions, more work above simply getting the syntax right is required. In the Code Anthem test, they're testing a slightly higher level, but not by much. Yes they can show that the code compiles and that it correctly produces the correct results, but they cannot show that the code is "good" nor that the programmer is "competent", or "capable" or "worth hiring".&lt;br /&gt;&lt;br /&gt;C# is a pretty rich language, even if you stick with the popular 3.5 version and not the new 4.0. Of course, 4.0 doesn't add a whole hell of a lot to the core language, so we can really ignore it for now. Features like contravariance of generic types are certainly nice, but the new version doesn't have huge market penetration yet so we can't expect most people to know about it. Testing for a basic ability to write functions and loops, and a basic understanding of core types really misses out on many of the important features of C#, and really does nothing to separate the coders who don't know the language well from those that do.&lt;br /&gt;&lt;br /&gt;A C# or Java test which does nothing to test knowledge of object-oriented concepts, such as classes and inheritance, really isn't a comprehensive test of those languages. If you can't use inheritance, don't understand the difference between is-a and has-a relationships, and don't know what keywords like "interface", "abstract" and "virtual" do, you really don't know C#.&lt;br /&gt;&lt;br /&gt;In C#, if you don't understand delegates, you could get in trouble pretty quickly. Understanding the difference between a method group identifier and a delegate can be a simple and confusing mistake for inexperienced programmers. Not understanding the difference between normal and multicast delegates can lead to some very weird runtime effects. Not understanding anonymous delegates, closures, and dynamic invocation really can really separate out the entry-level C# coders from from the advanced programmers and gurus.&lt;br /&gt;&lt;br /&gt;Beginner C# coders will know the basics about syntax: classes, functions, loops, variables, operators. Median C# coders will be using interfaces, inheritance, delegates, exceptions, and built-in generics. Good C# coders will&amp;nbsp; be doing all the previous things correctly and at a higher level, plus mixing and matching all these tools and techniques to solve complex problems.&lt;br /&gt;&lt;br /&gt;All of the tests in Code Anthem (at least those that I saw) gave the test-taker a function prototype and some instructions, and asked for the body of the function to be filled in. A different, more comprehensive type of problem would give an interface definition and ask the user to implement an entire class. An extremely comprehensive test would present many types of questions:&lt;br /&gt;&lt;br /&gt;&lt;ol&gt;&lt;li&gt;Implement a function to have a particular behavior, given a signature and a description.&lt;/li&gt;&lt;li&gt;Implement a class to provide a specific interface, given the interface and a description of behavior.&lt;/li&gt;&lt;li&gt;Given a piece of pre-written code with bugs, identify and fix the bugs.&lt;/li&gt;&lt;li&gt;Multiple choice questions about language features which would be hard to test practically.&lt;/li&gt;&lt;/ol&gt;You'd probably want to put some kind of time limit on those multiple choice questions, but there are some things that you're not going to really be able to test without them.&lt;br /&gt;&lt;br /&gt;There are plenty of ways to run this kind of a test. You could do it like a game show, such as "Who wants to be a Millionaire": People start with a score of zero, and gradually move up the scale as they answer ever harder and harder questions. The really good ones might have to spend some time on the test, so it would be good to break it up into stages that can be tackled at different times.&lt;br /&gt;&lt;br /&gt;Another option would be to break up the test into several smaller tests focusing on different issues. Some tests could even be language-agnostic, such as figuring out how to solve particular problems where language-specific syntax or semantics doesn't play much of a role in the solution.&lt;br /&gt;&lt;br /&gt;What Code Anthem does now is a basic test to weed the programming bozos from non-bozos. This is certainly a good first step, but really doesn't do anything to differentiate between the coders who are good at what they do from the true &lt;i&gt;creme de la creme&lt;/i&gt; of potential applicants (and everybody in between). Weeding out the obvious bozos is still a good service to render, but it really falls short of a potential that a system like this has.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-7276084271113741673?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/7276084271113741673/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/08/code-anthem-what-can-change.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/7276084271113741673'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/7276084271113741673'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/08/code-anthem-what-can-change.html' title='Code Anthem: What Can Change'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-5118048841249917101</id><published>2010-08-27T23:02:00.000-04:00</published><updated>2010-08-27T23:02:37.283-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='CodeAnthem'/><title type='text'>Code Anthem</title><content type='html'>I'm always &lt;a href="http://en.wikipedia.org/wiki/Trolling_%28fishing%29"&gt;trolling&lt;/a&gt; around the internet for cool new ideas and blogs about programming. A while back I found the &lt;a href="http://www.codeanthem.com/blog/"&gt;blog&lt;/a&gt; for a new site called &lt;a href="http://www.codeanthem.com/"&gt;Code Anthem&lt;/a&gt; and started following it. The Code Anthem blog gives an often-refreshing look at programming in the workplace, with specific focus on hiring and getting hired.&lt;br /&gt;&lt;br /&gt;Code Anthem isn't just a blog though. Recently they've started beta testing on their pay service, a coding test system that's designed to work something like an online portfolio. &lt;a href="http://www.codeanthem.com/blog/2010/07/how-code-anthem-works/"&gt;It works like this&lt;/a&gt;: Coders take standard tests in one of a variety of programming languages to gauge skill levels. Employers come to Code Anthem and ask "I want a programmer with X skill level in Y programming language". Code Anthem then matches up coders with that skill set in that geographic area with the prospective employer. Everybody wins. Well, not the really bad programmers, but I suspect that they never win.&lt;br /&gt;&lt;br /&gt;Yesterday, as part of the private beta, I took the test in C#. They're only offering Java and C# for the beta test so far, and of the two I am much better with the later. I went in with pretty high hopes, since I write a lot of C# code at work and consider myself to be pretty damn good at it. I logged in, answered 5 questions, and scored a solid 3/10.&lt;br /&gt;&lt;br /&gt;Well, &lt;i&gt;that sucks&lt;/i&gt; in a major way. I mean, I don't think I'm the best C# coder that ever graced the language, and I maybe rushed through some of the questions faster than I should have, but I have to believe I am above the 30% mark. My pride a little wounded and having a few spare moments open up, I decided to re-take the test. Through the next 5 questions I was much more careful, took my time writing the solutions, and was sure to perform testing on each solution to verify the results. At the end I looked over everything one last time, clicked submit, and ... 3/10.&lt;br /&gt;&lt;br /&gt;&lt;b&gt;Damnit&lt;/b&gt;. Maybe I suck &lt;i&gt;much worse&lt;/i&gt; at this whole programming gig than I thought I did. Maybe I'm really bad at it. Like, "&lt;i&gt;time to change careers&lt;/i&gt;" bad, Or maybe "&lt;i&gt;time to join the army&lt;/i&gt;" bad. It was really quite a shock, and for a few moments I got kind of upset about it. I quickly got over it, and did what I think any good programmer should do in the face of failure: &lt;b&gt;start debugging&lt;/b&gt;.&lt;br /&gt;&lt;br /&gt;I didn't save my answers to all my questions, but did have a few and I started rigorous testing. I also had taken some notes about the various questions: some were worded in particularly difficult or vague ways, and I had made note of those things as I went. I put together my findings into a nice email and sent it off to the powers that be. After all, the point of a beta test is to find bugs and elicit feedback.&lt;br /&gt;&lt;br /&gt;Some of the answers I had originally gotten wrong exposed bugs in the test and the results were flipped in my favor. My score didn't change too much though because the test is adaptive: as you get questions wrong it gives you easier questions later. So even though I was getting some results changed, they were the low-tier easy questions and didn't improve my scores much. What would have been needed at that point would be for me to go back and retake the test from scratch, getting positive results at the beginning which in turn would lead to harder but more valuable questions as the test progressed, improving my score substantially.&lt;br /&gt;&lt;br /&gt;What happened instead was that Code Anthem realized that mean scores were far too low, and &lt;a href="http://www.codeanthem.com/blog/2010/08/lessons-learned-from-private-beta/"&gt;they fixed a few things&lt;/a&gt;. First, they fixed some bugs in some tests, then re-evaluated the relative difficulty (and therefore value) of tests based on how often they were legitimately answered correctly, then they recalibrated results to move the mean score up from about 30% to 50%. The end result is that my paltry 3.0 turned into &lt;a href="http://www.codeanthem.com/member/Whiteknight"&gt;a hefty 9.0&lt;/a&gt;, much more in line with the result I was expecting to get from the beginning. Of course, this score will probably be wiped when the beta ends and the real test gets put in place, but it's nice to see such a high number now anyway.&lt;br /&gt;&lt;br /&gt;In the end, what do I think about Code Anthem? For starters, I think it's a really &lt;b&gt;awesome idea&lt;/b&gt;. I've been on a number of job interviews, mostly two years ago when I graduated and was looking for my first real job. What I remember most about them is that they were almost laughably non-technical.&lt;br /&gt;&lt;br /&gt;One interview I had was a phone interview, and I distinctly remember because of the awkwardness of trying to "write" code to an interviewer over the phone (Actually, the thing I remember most about this particular interview was a terrible blunder I made in describing the purpose of the POSIX "fork" routine, a mistake that I might not ever forgive myself for).&lt;br /&gt;&lt;br /&gt;A second interview I went on had a written programming test, that I remember being a very vaguely-worded "real world" question, a problem that the engineer administering the test had run into not long before the interview began and hadn't taken the time to really formulate it into a test-worthy question. I spent a little bit too much time writing out a stack class that I would have needed to solve the actual problem (the test was in embedded C, so I didn't have a fancy "stack" type to play with) and when time got short I ended up just verbally describing my solution to the guy.&lt;br /&gt;&lt;br /&gt;Another interview I went on featured a short technical phone screen. When I got to the in-person interview, I was given a test mostly involving mathematics with very little programming. It was a company specializing in numerical computing, so it's not strange that they focused so much on my math skills.&lt;br /&gt;&lt;br /&gt;In terms of technical interviews, these three examples are really the most rigorous demonstrations of my ability that any company saw before making a hiring decision. The only other information they would have had would be my transcripts, my resume, and the short descriptions of some past projects I gave verbally to interviewers. There would be absolutely no way for them to know, given such little reliable information, what I or any application would be really capable of. Saying that I worked on such-and-such a project when I was in school, or that I was "proficient" in languages X, Y, and Z are really uninformative and subjective personal assessments, and hardly a reliable measure of my value to a company.&lt;br /&gt;&lt;br /&gt;So this is where Code Anthem shines. They provide answers to a very fundamental question: &lt;i&gt;Can this person take a set of specifications and provide a sample of code that works&lt;/i&gt;? I think this is a very cool idea, and I would be very happy if the concept gained traction in the industry. It certainly is a huge improvement over the current system where professionally-crafted resumes, awkward phone screens, and off-the-cuff written tests are used to make hiring decisions which could potentially have huge effects on the company.&lt;br /&gt;&lt;br /&gt;Of course there are some things that I think the site could improve. I'll write some critiques in another post.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-5118048841249917101?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/5118048841249917101/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/08/code-anthem.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/5118048841249917101'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/5118048841249917101'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/08/code-anthem.html' title='Code Anthem'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-1210372532633804853</id><published>2010-08-23T17:00:00.000-04:00</published><updated>2010-08-23T17:00:00.134-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='Parrot'/><category scheme='http://www.blogger.com/atom/ns#' term='LAPACK'/><category scheme='http://www.blogger.com/atom/ns#' term='Parrot-Linear-Algebra'/><title type='text'>PLA Planning: Next Steps</title><content type='html'>I did a little bit more infrastructure work on &lt;a href="http://github.com/Whiteknight/parrot-linear-algebra"&gt;Parrot-Linear-Algebra&lt;/a&gt; this weekend. I wasn't able to get a lot done because we spent Saturday with family, but I got some nice groundwork laid for important future developments.&lt;br /&gt;&lt;br /&gt;I started thinking about bindings for the &lt;a href="http://netlib.org/lapack/"&gt;LAPACK library&lt;/a&gt;, and much of the work I did this weekend was in preparation for that work. LAPACK is particularly tricky because it's written in &lt;a href="http://adaptiveoptics.blogspot.com/2007/05/fortran-sucks-balls.html"&gt;FORTRAN&lt;/a&gt; and unlike the &lt;a href="http://math-atlas.sourceforge.net/"&gt;ATLAS library&lt;/a&gt; I use for my &lt;a href="http://www.netlib.org/blas/"&gt;BLAS&lt;/a&gt; &lt;a href="http://en.wikipedia.org/wiki/Basic_Linear_Algebra_Subprograms#Implementations"&gt;implementation&lt;/a&gt;, there aren't any good C bindings to LAPACK around that I can find. Yes there is the &lt;a href="http://www.netlib.org/clapack/"&gt;CLAPACK library&lt;/a&gt;, but I am not a particular fan of the way that library is generated and there &lt;a href="http://packages.ubuntu.com/search?keywords=clapack&amp;amp;searchon=names&amp;amp;suite=lucid&amp;amp;section=all"&gt;aren't any premade packages for CLAPACK on Ubuntu&lt;/a&gt;. In installing PLA on Linux, I would like users to be able to get all prerequisites from their distribution's package manager and not have to compile these other prerequisites manually. This is a good thing since some of these can be extremely tricky to compile.&lt;br /&gt;&lt;br /&gt;I've decided that, in the foreseeable future, we will be linking to the LAPACK library directly. This does mean that we are going to need to write our own bindings and wrappers for the functions we want to call, but that's not a huge issue. Plus, I can start moving forward on something I've wanted to have for a while now: more flexible and configurable interfaces to BLAS and LAPACK. There are several implementations of each of these libraries (BLAS especially), and I want PLA to support as many of them as possible.&lt;br /&gt;&lt;br /&gt;To get these kinds of flexible interfaces, we're going to need to support large swaths of conditionally-compiled code. This will get extremely unweildy if we try to keep all the logic in the few monolithic PMC files I've been writing, so I decided that we're going to need to upgrade our build machinery to support linking in additional C library files into the dynpmc build. This weekend &lt;a href="http://github.com/Whiteknight/parrot-linear-algebra/commit/4e23a2f7854e32641b6aba606976c01220eddc0b"&gt;I added that machinery&lt;/a&gt;, and started the process of factoring out some of the supporting routines from the various PMC types into those files. With common functions defined in one place, I can go through and make sure all my types are making consistent use of them. Plus, I can start writing up complicated library bindings without worrying about making my large PMC files any larger.&lt;br /&gt;&lt;br /&gt;BLAS doesn't actually supply too many routines that the PLA matrix types need. The most important, and only one I call so far, is the &lt;a href="http://en.wikipedia.org/wiki/GEMM"&gt;GEMM routine&lt;/a&gt;, which performs the generalized matrix multiply/addition operation:&lt;br /&gt;&lt;br /&gt;Y = αAB + βC&lt;br /&gt;&lt;br /&gt;Where α and β are scalar values (FLOATVAL in NumMatrix2D and Complex in ComplexMatrix2D), and A, B, and C are matrices of compatible dimension. This is a very powerful operation, and really the only one from BLAS that I need. Other routines from BLAS are more highly optimized versions of this same routine for specific types of matrices (diagonal, triangular, etc), and operations between matrices and vectors which I really don't support yet. Because there is really only this one routine, and because it's so powerful and common, I've included it as a method directly in the PMC types themselves. I may rethink that decision later, but for now it seems like a decent fit where it is.&lt;br /&gt;&lt;br /&gt;LAPACK is a little bit different, however. LAPACK contains hundreds of routines, all of which are named in an idiosyncratic and difficult-to-understand way. These routines are broken up into &lt;b&gt;drivers&lt;/b&gt;, &lt;b&gt;computational routines&lt;/b&gt;, and &lt;b&gt;auxiliary routines&lt;/b&gt;. The driver routines perform the most important operations and typically call sequences of the lower-level computational and auxiliary routines to perform their work. Driver routines include calculating linear least squares, finding eigenvalues and eigenvectors, and singular value decomposition.These are the ones that PLA is going to target first, though access to other, lower-level routines might be nice to have as well, later down the road.&lt;br /&gt;&lt;br /&gt;I am torn as to how I want to provide LAPACK bindings in Parrot. The obvious thing I could do is write them all up as methods on the PLA matrix types. This is a system that provides the easiest access, but also starts to get bloated and unweildy very quickly. Plus, if you don't need LAPACK routines for your program, I would be forcing you to load the LAPACK library and create Sub PMCs for all those methods. That can get to be a pretty large cost to pay. I really don't like this idea because it adds huge bloat to my PMC files and adds extra runtime overhead of adding in all sorts of extra method PMCs when we load PLA. Plus, it doesn't scale well, any time we want to add a new LAPACK-based routine we need to add a new method to several PMC types present and future.&lt;br /&gt;&lt;br /&gt;One alternative is to provide a separate library of functions and dynamically load it using Parrot's NCI tools. I don't know what the current status of the NCI system is, but without significant improvement there, attempting to load a complex library like LAPACK, even if I provided some simplified Parrot-specific wrappers, would be a nightmare. This option does provide a little bit more flexibility because we can load in the methods that we want without having to deal with ones that we don't. Also, I don't think the distutils library that PLA is using supports creating standalone shared libraries that aren't dynpmc or dynops libraries, so I would have to write all that machinery myself. I'm not sure I really like this option either, for all these reasons.&lt;br /&gt;&lt;br /&gt;I'm also torn about whether I want to make some prerequisites for PLA  optional. &lt;a href="http://gitorious.com/kakapo/kakapo"&gt;Kakapo&lt;/a&gt;, for instance, is only used for the test suite. If you  don't have Kakapo installed you can still build and use PLA, you just  can't run the test suite. BLAS is currently required because it's used  to implement the GEMM method but also the various multiply vtables. I  suppose I could make it optional, but I think it would be a little  sneaky if different builds of PLA supported a different list of VTABLEs  and METHODs for core types, including for something as fundamental as  multiplication. LAPACK is a little bit different, because even without  it we do have a pretty solid amount of core functionality. I am strongly  looking into an ability to make LAPACK an optional prerequisite, and  only include bindings for it if we have the library installed. If I wrote a dynamic library wrapper for LAPACK and had the user include that file manually to get those routines, you could still use the core library.&lt;br /&gt;&lt;br /&gt;I had also considered the idea (though I've been fighting it) of building some kind of PLALibrary singleton PMC type which would control this kind of stuff. In addition to providing information about library version and configuration options, a PLALibrary PMC could allow dynamic loading of methods, and could forcibly inject those methods into the necessary PMC types if requested. This would allow me to build everything together into a single dynamic library without forcing me to define all the methods upfront which I don't need. I may like this idea the best, though I've been resisting creating a library singleton PMC until now.&lt;br /&gt;&lt;br /&gt;There's a lot to consider, and I probably won't make any hard decisions about future plans for the next few days. In the interim I do want to put together some proof-of-concept wrappers for some common LAPACK routines, and start playing with those. If I could get routines to calculate Eigenvalues, Eigenvectors, Inverses, and Singular Value Decompositions, I think that would be a pretty great start.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-1210372532633804853?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/1210372532633804853/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/08/pla-planning-next-steps.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/1210372532633804853'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/1210372532633804853'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/08/pla-planning-next-steps.html' title='PLA Planning: Next Steps'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-6242513867170954402</id><published>2010-08-21T08:00:00.000-04:00</published><updated>2010-08-21T08:00:06.103-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='Parrot'/><category scheme='http://www.blogger.com/atom/ns#' term='GSOC'/><category scheme='http://www.blogger.com/atom/ns#' term='Threads'/><title type='text'>GSoC Threads: Chandon's Results</title><content type='html'>The pencil's down date for the 2010 GSoC program has come and gone, and now we're starting the process of sorting through the rubble and seeing what all got accomplished this summer. I have been purposefully trying not to provide too many of my own status updates for Chandon's project this summer, instead wanting him to post &lt;a href="http://parrot.org/blog/836"&gt;his own updates&lt;/a&gt; as he went without me getting in the way. It's his project after all, and I wanted him to get all the credit for all the good work he was doing.&lt;br /&gt;&lt;br /&gt;The project is now over, and he's &lt;a href="http://parrot.org/content/hybrid-threads-gsoc-project-results"&gt;posted a nice final review&lt;/a&gt; of what he managed to accomplish this summer. While it isn't everything that he mentioned in his proposal, it still is quite a nice step in the right direction and a lot of the necessary framework is now available to bring threading to Parrot in a real and usable way.&lt;br /&gt;&lt;br /&gt;So what did he manage to do? In his branch there is now a more-or-less working implementation of green threads, which means Parrot is getting onto the same level of capability as modern Python or Ruby implementations. You can schedule multiple independent tasks to run and, while they cannot run truly in parallel on separate OS threads or even on multiple processor cores, you can get the illusion of concurrency. It also gives you the ability to run multiple tasks together without having to explicitly schedule them one after the other, or having to manually switch back and forth between them. In a later post I may go into some detail about how this all works, and how it can be used by programs.&lt;br /&gt;&lt;br /&gt;This is a pretty significant step forward, and once a few of the final nits get worked out I think we will be able to merge this into Parrot trunk and start making use of the new functionality immediately. However, green threads is only one half of the promised "hybrid threads" that Chandon's proposal hoped to achieve. The other half is the ability to run Parrot code in true parallel on separate OS threads. This was the larger part of the project and definitely the more difficult piece. Today I would like to talk a little bit about why it didn't get done, and maybe motivate some people to look into help completing the work as we move forward.&lt;br /&gt;&lt;br /&gt;&lt;br /&gt;Let's take a look at a small snippet of normal object-oriented code:&lt;br /&gt;&lt;br /&gt;foo.bar();&lt;br /&gt;foo.bar();&lt;br /&gt;&lt;br /&gt;Extremely simple, we have an object "foo" and we are calling the "bar" method on it. In a very naive staticly-typed language this would be a simple operation. The compiler would determine the type of foo, calculate the location of the bar routine in memory and would simply call that address twice. This would be extremely fast to execute too, which everybody likes. This would basically be converted to this low-level pseudo-code:&lt;br /&gt;&lt;br /&gt;call bar(foo)&lt;br /&gt;call bar(foo)&lt;br /&gt;&lt;br /&gt;Now let's move up to a slightly more complicated example: a statically-typed language which allows subclassing and method overriding. Now, the compiler doesn't necessarily know which function in memory corresponds to "foo.bar()", since foo could be of type Foo, or of type FooSubclass, or even FooSubSubclass, and the location of the appropriate bar function would change with each different type. However, for each type, the value of bar does not change. It can be overridden by subclasses, but it cannot be changed for the class itself. Now, the compiler needs to ask foo to get the appropriate method first:&lt;br /&gt;&lt;br /&gt;method _bar = get_method(typeof(foo), "bar")&lt;br /&gt;call _bar(foo)&lt;br /&gt;call _bar(foo)&lt;br /&gt;&lt;br /&gt;Assuming foo does not change type inside the call to _bar, this code works just fine. Next let's look at a more complicated example still, the case of a dynamic language like Perl or Python. In these languages the class MyFooClass is an object of type Class, and foo is an object of type MyFooClass. Also bar is not a compiled-in constant property of the Class, but is instead a mutable property of it. Inside the call to bar, maybe we change the definition of bar itself inside the class object. Likewise, the definition of routines like "find_method" inside Class can change. Accounting for all this dynamicism,&amp;nbsp; our code changes to look like this:&lt;br /&gt;&lt;br /&gt;class fooclass = find_class(foo)&lt;br /&gt;method _get_method = find_method(fooclass, "bar")&lt;br /&gt;call _bar(foo)&lt;br /&gt;class fooclass = find_class(foo)&lt;br /&gt;method _get_method = find_method(fooclass, "bar")&lt;br /&gt;&lt;br /&gt;&lt;br /&gt;call _bar(foo) &lt;br /&gt;&lt;br /&gt;Keep in mind that everything can change inside each call to bar. We can:&lt;br /&gt;&lt;br /&gt;&lt;ul&gt;&lt;li&gt;Modify the inheritance hierachy of MyFooClass to add or remove parent classes&lt;/li&gt;&lt;li&gt;Add or remove methods in MyFooClass, and any item in it's inheritance hierarchy&lt;/li&gt;&lt;li&gt;Add or remove methods on the object foo itself (like in javascript where foo.bar = function() {...})&lt;/li&gt;&lt;li&gt;Change the class of foo&lt;/li&gt;&lt;/ul&gt;Let's bring this back to the idea of threading before I go too far off topic. Every time we call a method, or attempt to access a field on an object, we need to look those up in the associated Class object first. Those Class objects need to be global, because all code on all threads need to be able to lookup methods on any object, because we can pass objects between threads and we need to be able to work with them.&lt;br /&gt;&lt;br /&gt;If classes are global, and need to be accessible from multiple threads, we inevitably run into the idea of contention: If one thread is changing the definition of foo.bar() at the same time that another thread is attempting to look it up, the thread doing the reading may end up with incomplete, corrupted, or incorrect information.&lt;br /&gt;&lt;br /&gt;Global data is the enemy of good multithreading. Global data is a necessity for highly dynamic object systems like those supported by Parrot. Ergo, multithreading on Parrot is hard.&lt;br /&gt;&lt;br /&gt;&lt;br /&gt;Look at all the global data in Parrot: there's the interpreter object itself, the packfiles, the list of Classes (and the methods/properties that they contain), the list of Namespaces (and the global values they contain), the context graph (and the contents of all registers in those contexts), the opcode table and the MMD signature cache. This is a hell of a lot of globally-available data, and we're going to need to find a sane way to limit corruptive concurrent access to these things. Not only that, but if we have to lock every single access to a global value, which means we need to obtain a lock every time we want to do a method call, our performance is going to be &lt;i&gt;terrible&lt;/i&gt;.&lt;br /&gt;&lt;br /&gt;By convention I think we can all agree that things like the opcode table should really not be modified when we have multiple threads running. But it's harder to make such a convention that class/method definitions should be treated as constant when multiple threads are running.&lt;br /&gt;&lt;br /&gt;What we can do to alleviate some of the performance problems is to create short-term caches for things like Classes and Methods in local threads, with the understanding that without explicit synchronization, the exact ordering of read/write operations between threads cannot be guaranteed and can be scheduled for maximum benefit. If the relative execution times of two operations on separate threads cannot be guaranteed, then it makes no difference to functionality for the user whether those operations happen at random times with respect to each other and whether Parrot manually orders them to improve caching.&lt;br /&gt;&lt;br /&gt;Think about it this way: We have two threads, one is writing a global value and one is reading it. These threads are operating in true parallel on two separate processor cores. If we run this program a million times, sometimes the read will randomly occur before the write, sometimes the write will occur before the read, and sometimes they will happen at the exact same moment and cause corruption. Depending on a simple matter of random timing, we could get three very different results from our program. Now, if Parrot implements a heuristic that if a write and a read happen within a very short period of time, Parrot will manually order them so that the read occurs before the write. And, if the read always happens before the write, maybe we don't read at all, but instead use a cached version. Then, only when the write has complete, we update the cache.&lt;br /&gt;&lt;br /&gt;Another thing to think about is that we could disallow direct lookups of data in global variables, and give each thread a local copy of global values. Considering the principal I mentioned above, we can be a little bit liberal with the way we handle writes and reads. If a thread modifies its copy of a Class, those changes could be propagated to a global reference copy at a convenient time, and then propagated down to the local copies later, only when it's convenient to do so. Remember, if things are happening at random times compared to each other, it makes no substantive difference to the thread whether it's copy of a global variable is exactly up-to-date or whether it's a little bit lagged behind. That is, the reading thread doesn't know whether the writing thread had a legitimate delay before writing, or whether Parrot manually scheduled the write at a different time.&lt;br /&gt;&lt;br /&gt;To get a complete Hybrid Threads implementation in Parrot like what Chandon was envisioning, we are going to have a few steps to take:&lt;br /&gt;&lt;ol&gt;&lt;li&gt;We have to break the &lt;a href="http://trac.parrot.org/parrot/wiki/GSOC_ThreadsInterpreterSplit"&gt;Interpreter structure up into two parts&lt;/a&gt;: One which is truly global and contains data which all threads need, and one which is local for each thread and contains data that the thread needs. The local interpreter may contain a cache or a mirror of data in the global interpreter.&lt;/li&gt;&lt;li&gt;We need to come up with a good sane scheme (which will likely consist of both technical solutions and programmer conventions) to manage access to global values&lt;/li&gt;&lt;li&gt;We need to come up with a good sane scheme for sharing values between threads.Creating a local variable and passing a reference to it to another thread essentially turns it into a global variable and opens up problems with contention. We need a good way to manage this without slowing everything down.&lt;/li&gt;&lt;/ol&gt;All of these problems are solvable, and if we put forward a concerted effort we could have a decent hybrid threading system in Parrot within the next few months. We almost have a complete working Green Threads system ready to merge now, and hopefully that will be enough for our users while we get the rest of the details of the hybrid system worked out and implemented.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-6242513867170954402?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/6242513867170954402/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/08/gsoc-threads-chandons-results.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/6242513867170954402'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/6242513867170954402'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/08/gsoc-threads-chandons-results.html' title='GSoC Threads: Chandon&apos;s Results'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-5121899035634591844</id><published>2010-08-20T17:00:00.000-04:00</published><updated>2010-08-20T17:00:02.297-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='Parrot'/><category scheme='http://www.blogger.com/atom/ns#' term='Parrot-Linear-Algebra'/><title type='text'>PLA: More New Methods</title><content type='html'>In a previous post I discussed some of the &lt;a href="http://wknight8111.blogspot.com/2010/08/pla-status-updates.html"&gt;common behaviors&lt;/a&gt; that the matrix types in Parrot-Linear-Algebra implement. Recently, I've added a few more which I think are pretty cool and worth discussing.&lt;br /&gt;&lt;br /&gt;First and foremost, I've finally added some methods to convert between type. There are now methods for &lt;b&gt;convert_to_number_matrix&lt;/b&gt;, &lt;b&gt;convert_to_complex_matrix&lt;/b&gt;, and &lt;b&gt;convert_to_pmc_matrix&lt;/b&gt;. Every type implements all of these methods, even when it would be a no-op. This is so you can take a PMC and if it "does 'matrix'" you can cast it to the type you want to deal with without worrying about spending too much expense. These methods always return a new matrix, so you can keep a copy of the matrix in it's original form and also have a new copy to play with. In the case where the matrix is already in the target format, I create a clone.&lt;br /&gt;&lt;br /&gt;Unfortunately, these conversion operations can be a little bit expensive when you're actually converting types. The problem is that the data for the matrices is stored internally in a very dense format. For the Number and Complex matrices, the data is stored internally in the format required by the BLAS library routines. For the number matrix, the values are basically stored together in a single large buffer. For complex matrices, the real and imaginary values are stored together also, alternating positions. Converting one to the other is not easy, since I have to allocate a completely new buffer and iterate over each space individually. So, too many conversion operations can get expensive quickly.&lt;br /&gt;&lt;br /&gt;Using these new conversion methods, I have updated some previous methods, like gemm(), the routine which performs the &lt;a href="http://en.wikipedia.org/wiki/GEMM"&gt;GEMM&lt;/a&gt; operation from the &lt;a href="http://en.wikipedia.org/wiki/Basic_Linear_Algebra_Subprograms"&gt;BLAS library&lt;/a&gt;. You can now pass any matrix types to this method, and it will perform conversions internally to ensure the types match. Here's a fun little NQP example that shows some of the capabilities of this library today:&lt;br /&gt;&lt;br /&gt;&lt;pre&gt;INIT { pir::load_bytecode("pla_nqp.pbc"); }&lt;br /&gt;&lt;br /&gt;my $A := NumMatrix2D.new();&lt;br /&gt;$A.initialize_from_args(2, 2, 1, 2.0, "3", 4);&lt;br /&gt;$A.transpose();&lt;br /&gt;pir::say("A: " ~ $A);&lt;br /&gt;&lt;br /&gt;my $B := ComplexMatrix2D.new();&lt;br /&gt;$B.initialize_from_array(2, 2, [1, 2.0, "3+3i", "7+5i"]);&lt;br /&gt;$B.conjugate();&lt;br /&gt;pir::say("B: " ~ $B);&lt;br /&gt;&lt;br /&gt;my $C := PMCMatrix2D.new();&lt;br /&gt;$C.fill(4.4, 2, 2);&lt;br /&gt;pir::say("C: " ~ $C);&lt;br /&gt;&lt;br /&gt;my $D := $B.gemm(0.5, $A, $B, "1+2i", $C);&lt;br /&gt;pir::say("D: " ~ $D);&lt;br /&gt;&lt;/pre&gt;&lt;br /&gt;You can try it yourself, or you can take my word for it that the result is correct. I've verified it this morning in Octave, and the results are the same (though the Octave script to produce this result is considerably shorter).&lt;br /&gt;&lt;br /&gt;PLA is finally starting to get the kind of basic functionality and test coverage that I've been hoping for. With a few more finishing touches on this base, I'm going to start adding new functionality like LAPACK bindings. Specifically, I'm hoping to add in support for some common matrix decompositions, matrix reductions, inverses and eigenvalues. I'm also hoping to get started on the module for Rakudo sometime soon.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-5121899035634591844?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/5121899035634591844/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/08/pla-more-new-methods.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/5121899035634591844'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/5121899035634591844'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/08/pla-more-new-methods.html' title='PLA: More New Methods'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-7369903067145452664</id><published>2010-08-19T22:05:00.000-04:00</published><updated>2010-08-19T22:05:29.941-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='Testing'/><category scheme='http://www.blogger.com/atom/ns#' term='Parrot'/><category scheme='http://www.blogger.com/atom/ns#' term='Parrot-Linear-Algebra'/><title type='text'>PLA Test Suite Redux</title><content type='html'>I've been doing a hell of a lot of work on the Parrot-Linear-Algebra test suite in the past few days, and the results are quite spectacular. I've completely reworked the way the suite works, and broke a small handful of monolithic test files into a series of smaller, focused test files. Instead of describing it, I'll just show the results:&lt;br /&gt;&lt;br /&gt;&lt;pre&gt;t/sanity.t .............................................. ok&lt;br /&gt;t/pmc/pmcmatrix2d.t ..................................... ok&lt;br /&gt;t/pmc/nummatrix2d.t ..................................... ok&lt;br /&gt;t/pmc/complexmatrix2d.t ................................. ok&lt;br /&gt;t/methods/nummatrix2d/convert_to_complex_matrix.t ....... ok&lt;br /&gt;t/methods/nummatrix2d/row_scale.t ....................... ok&lt;br /&gt;t/methods/nummatrix2d/get_block.t ....................... ok&lt;br /&gt;t/methods/nummatrix2d/mem_transpose.t ................... ok&lt;br /&gt;t/methods/nummatrix2d/convert_to_number_matrix.t ........ ok&lt;br /&gt;t/methods/nummatrix2d/initialize_from_args.t ............ ok&lt;br /&gt;t/methods/nummatrix2d/row_swap.t ........................ ok&lt;br /&gt;t/methods/nummatrix2d/iterate_function_inplace.t ........ ok&lt;br /&gt;t/methods/nummatrix2d/fill.t ............................ ok&lt;br /&gt;t/methods/nummatrix2d/initialize_from_array.t ........... ok&lt;br /&gt;t/methods/nummatrix2d/iterate_function_external.t ....... ok&lt;br /&gt;t/methods/nummatrix2d/transpose.t ....................... ok&lt;br /&gt;t/methods/nummatrix2d/item_at.t ......................... ok&lt;br /&gt;t/methods/nummatrix2d/gemm.t ............................ ok&lt;br /&gt;t/methods/nummatrix2d/row_combine.t ..................... ok&lt;br /&gt;t/methods/nummatrix2d/resize.t .......................... ok&lt;br /&gt;t/methods/nummatrix2d/convert_to_pmc_matrix.t ........... ok&lt;br /&gt;t/methods/nummatrix2d/set_block.t ....................... ok&lt;br /&gt;t/methods/complexmatrix2d/convert_to_complex_matrix.t ... ok&lt;br /&gt;t/methods/complexmatrix2d/row_scale.t ................... ok&lt;br /&gt;t/methods/complexmatrix2d/get_block.t ................... ok&lt;br /&gt;t/methods/complexmatrix2d/mem_transpose.t ............... ok&lt;br /&gt;t/methods/complexmatrix2d/convert_to_number_matrix.t .... ok&lt;br /&gt;t/methods/complexmatrix2d/initialize_from_args.t ........ ok&lt;br /&gt;t/methods/complexmatrix2d/row_swap.t .................... ok&lt;br /&gt;t/methods/complexmatrix2d/iterate_function_inplace.t .... ok&lt;br /&gt;t/methods/complexmatrix2d/fill.t ........................ ok&lt;br /&gt;t/methods/complexmatrix2d/initialize_from_array.t ....... ok&lt;br /&gt;t/methods/complexmatrix2d/conjugate.t ................... ok&lt;br /&gt;t/methods/complexmatrix2d/iterate_function_external.t ... ok&lt;br /&gt;t/methods/complexmatrix2d/transpose.t ................... ok&lt;br /&gt;t/methods/complexmatrix2d/item_at.t ..................... ok&lt;br /&gt;t/methods/complexmatrix2d/gemm.t ........................ ok&lt;br /&gt;t/methods/complexmatrix2d/row_combine.t ................. ok&lt;br /&gt;t/methods/complexmatrix2d/resize.t ...................... ok&lt;br /&gt;t/methods/complexmatrix2d/convert_to_pmc_matrix.t ....... ok&lt;br /&gt;t/methods/complexmatrix2d/set_block.t ................... ok&lt;br /&gt;t/methods/pmcmatrix2d/convert_to_complex_matrix.t ....... ok&lt;br /&gt;t/methods/pmcmatrix2d/get_block.t ....................... ok&lt;br /&gt;t/methods/pmcmatrix2d/mem_transpose.t ................... ok&lt;br /&gt;t/methods/pmcmatrix2d/convert_to_number_matrix.t ........ ok&lt;br /&gt;t/methods/pmcmatrix2d/initialize_from_args.t ............ ok&lt;br /&gt;t/methods/pmcmatrix2d/iterate_function_inplace.t ........ ok&lt;br /&gt;t/methods/pmcmatrix2d/fill.t ............................ ok&lt;br /&gt;t/methods/pmcmatrix2d/initialize_from_array.t ........... ok&lt;br /&gt;t/methods/pmcmatrix2d/iterate_function_external.t ....... ok&lt;br /&gt;t/methods/pmcmatrix2d/transpose.t ....................... ok&lt;br /&gt;t/methods/pmcmatrix2d/item_at.t ......................... ok&lt;br /&gt;t/methods/pmcmatrix2d/resize.t .......................... ok&lt;br /&gt;t/methods/pmcmatrix2d/convert_to_pmc_matrix.t ........... ok&lt;br /&gt;t/methods/pmcmatrix2d/set_block.t ....................... ok&lt;br /&gt;PASSED 356 tests in 55 files&lt;br /&gt;&lt;/pre&gt;&lt;br /&gt;I factored out all tests to delegate matrix creation and matrix population routines into a series of type-specific factory objects. With these factories, creation of tests goes extremely quickly:&lt;br /&gt;&lt;br /&gt;&lt;ol&gt;&lt;li&gt;Write tests using a few methods and values from the factory instead&lt;/li&gt;&lt;li&gt;Create a short stub test for each Matrix type, subclassing the common tests and add a proper factory&lt;/li&gt;&lt;li&gt;There is no step 3.&lt;/li&gt;&lt;/ol&gt;It's really become quite simple, and&amp;nbsp; I'm much happier with the test suite now. Plus, I've added plenty of new features lately (complete with tests!) and I'm going to talk about that plenty in a later post.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-7369903067145452664?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/7369903067145452664/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/08/pla-test-suite-redux.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/7369903067145452664'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/7369903067145452664'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/08/pla-test-suite-redux.html' title='PLA Test Suite Redux'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-5024002903939162472</id><published>2010-08-18T08:00:00.000-04:00</published><updated>2010-08-18T08:00:03.556-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='Wiki'/><category scheme='http://www.blogger.com/atom/ns#' term='EmbedVideo'/><category scheme='http://www.blogger.com/atom/ns#' term='Github'/><title type='text'>Github Wikis and EmbedVideo</title><content type='html'>Github is rolling out a &lt;a href="http://github.com/blog/699-making-github-more-open-git-backed-wikis"&gt;new wiki system&lt;/a&gt; which uses a git backend to store data. This is an awesome feature, and one that I've wanted Github to add ever since I started using a similar feature at &lt;a href="http://code.google.com/"&gt;Google Code&lt;/a&gt;. It's great to be able to download all the wiki pages as a repository and work with it locally using my normal editing tools.&lt;br /&gt;&lt;br /&gt;&lt;b&gt;Whiteknight's rule of the internet #47&lt;/b&gt;: Raw TextArea controls in a browser are rarely good enough for anything, and should be avoided for any task besides extremely rare edits of small bits of text.&lt;br /&gt;&lt;br /&gt;Only one of my projects on Github makes use of the wiki feature there: &lt;a href="http://github.com/Whiteknight/mediawiki-embedvideo"&gt;EmbedVideo&lt;/a&gt;. Last night I converted the existing &lt;a href="http://github.com/Whiteknight/mediawiki-embedvideo/wiki"&gt;wiki pages&lt;/a&gt; to the new git-based wiki system, and pulled down the repository. In a wild fit of productivity, I updated much of the documentation and filled in some of the blanks that I had left for too long. It's still not perfect (software documentation rarely is), but it's better than it was and it's probably good enough to get most users moving with some of the new features.&lt;br /&gt;&lt;br /&gt;I love when tools get better, and compared to what they used to be the new Github wikis are certainly much much better.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-5024002903939162472?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/5024002903939162472/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/08/github-wikis-and-embedvideo.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/5024002903939162472'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/5024002903939162472'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/08/github-wikis-and-embedvideo.html' title='Github Wikis and EmbedVideo'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-5434604396199333838</id><published>2010-08-16T20:15:00.001-04:00</published><updated>2010-08-17T07:49:56.717-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='Parrot'/><category scheme='http://www.blogger.com/atom/ns#' term='Parrot-Linear-Algebra'/><category scheme='http://www.blogger.com/atom/ns#' term='Rakudo'/><title type='text'>PLA Status Updates</title><content type='html'>On Friday night we dropped the kid off with his grandparents for a sleepover. With the apartment to ourselves, my wife and I did what we've been wanting to do for months: she went to bed early and I stayed up to program. I started making some real progress in Parrot-Linear-Algebra, and also uncovered some interesting bugs which needed to be fixed.&lt;br /&gt;&lt;br /&gt;I added a short file for adding PLA support to programs written in NQP. The file, &lt;a href="http://github.com/Whiteknight/parrot-linear-algebra/blob/master/src/nqp/pla.nqp"&gt;pla.nqp&lt;/a&gt;, can be included into an NQP program like this once it's installed:&lt;br /&gt;&lt;pre&gt;INIT { pir::load_bytecode("pla.pbc"); }&lt;br /&gt;&lt;/pre&gt;That file loads the PLA binary library, stores a global reference to the library, and registers the type names with P6metaclass. That done, you can start writing more natural-looking programs in NQP. I updated the &lt;a href="http://github.com/Whiteknight/parrot-linear-algebra/commit/5ec59719855c180505526d21148836a14bd6a461"&gt;Gaussian Elimination &lt;/a&gt;example I wrote a few weeks ago to use this new bootstrap file (and some new methods I added), which means the example can now run without Kakapo or any other dependencies.&lt;br /&gt;&lt;br /&gt;Next up, I started prototyping bindings for the library for Rakudo. These are still preliminary, but I also have a working example of using numerical matrices in Rakudo. Within the next few days I will probably create a complete module for Rakudo (Math::LinearAlgebra, or something like that). I'll post more details as I do more work on it.&lt;br /&gt;&lt;br /&gt;I added a new item_at method which returns and optionally sets a value at given coordinates in a matrix. This method is common to all my core matrix types. The list of common methods for these matrix types is:&lt;br /&gt;&lt;br /&gt;&lt;ul&gt;&lt;li&gt;resize() : pre-allocate size for the matrix, growing (never shrinking) it to hold a certain size&lt;/li&gt;&lt;li&gt;fill() : Fill a matrix, or a region of a matrix, with a constant value. Automatically resize if necessary&lt;/li&gt;&lt;li&gt;transpose() : Transpose (swap rows with columns) the matrix lazily&lt;/li&gt;&lt;li&gt;mem_transpose() : Eagerly transpose the actual memory contents&lt;/li&gt;&lt;li&gt;iterate_function_inplace() : Execute a function for every element of the matrix, replacing that element with the function result&lt;/li&gt;&lt;li&gt;iterate_function_external() : Execute a function for every element of the matrix, creating a new matrix with the results. This is similar to Perl's map function.&lt;/li&gt;&lt;li&gt;initialize_from_array() : Insert values into the matrix from an array&lt;/li&gt;&lt;li&gt;initialize_from_args() : Similar to the _from_array variant, but initializes the matrix using elements from a slurpy argument list&lt;/li&gt;&lt;li&gt;get_block() : Return a block, or "submatrix" from the matrix&lt;/li&gt;&lt;li&gt;set_block() : Set a block in the matrix&lt;/li&gt;&lt;li&gt;item_at() : New this weekend, gets or sets a value in the matrix at the specified coordinates&lt;/li&gt;&lt;/ul&gt;There are maybe a handful of other methods I would like to add, but in terms of the core types, this is the standard API (plus a series of VTABLE calls, which are standard). For mathematics types I also have GEMM (the BLAS-based matrix multiply operation), and elementary matrix operations (add rows, swap rows, scale rows). This is not a shabby interface at all, and can start to be used for some real-world applications.&lt;br /&gt;&lt;br /&gt;This weekend I also started seeing some very weird errors in the PLA test suite. Tests were running fine, but I was seeing exceptions (and, in one case, a segfault) occur after all tests had passed. This sounded very &lt;a href="http://wknight8111.blogspot.com/2009/07/bugs.html"&gt;similar to another problem I've seen&lt;/a&gt; in the past. Here's the test that set it off. Can you spot the problem?&lt;br /&gt;&lt;pre&gt;method test_METHOD_iterate_function_inplace_TRANSPOSE() {&lt;br /&gt;    my $m := self.fancymatrix2x2();&lt;br /&gt;    $m.transpose();&lt;br /&gt;    my $n := self.matrix2x2(self.fancyvalue(0) * 2, self.fancyvalue(2) * 2,&lt;br /&gt;                            self.fancyvalue(1) * 2, self.fancyvalue(3) * 2);&lt;br /&gt;    my $sub := -&amp;gt; $matrix, $value, $x, $y {&lt;br /&gt;        return ($value * 2);&lt;br /&gt;    };&lt;br /&gt;    $m.iterate_function_inplace($sub);&lt;br /&gt;    assert_equal($m, $n, "external iteration does not respect transpose");&lt;br /&gt;}&lt;br /&gt;&lt;/pre&gt;&lt;br /&gt;What's maddening is that this test has been a problem for &lt;i&gt;months&lt;/i&gt;, but never caused a failure. It was silently wrong, probably since the day I wrote it.&lt;br /&gt;&lt;br /&gt;See it yet?&lt;br /&gt;&lt;br /&gt;The problem is that return statement. What should happen is this: The pointy block creates an anonymous subroutine, which I pass to the iterate_function_inplace method. The method should loop over ever element in our matrix and call that pointy block for each one. The result should be the same matrix with every element multiplied by two.&lt;br /&gt;&lt;br /&gt;What actually happens is this: the iterate_function_inplace method, in order to call the callback, must recurse on the C stack into a new runloop function. The pointy block executes in this &lt;i&gt;inferior runloop&lt;/i&gt;. However, where things get weird is that return statement. In NQP, returns are performed by constructing and throwing control exceptions. In the case of the pointy block (and I'm not sure whether or not this is a bug), the return statement jumps to the return handler for the test_METHOD_iterate_function_inplace_TRANSPOSE() function, not the anonymous sub.&lt;br /&gt;&lt;br /&gt;The sub executes after having only called the callback once, it never hits the final assertion (which, it turns out, would have failed). Control flow continues inside the inner runloop until the end of the test program, then the C runloop function returns, tries to continue executing from that point, and things go to hell.&lt;br /&gt;&lt;br /&gt;The solution is really quite simple. Change this:&lt;br /&gt;&lt;pre&gt;my $sub := -&amp;gt; $matrix, $value, $x, $y {&lt;br /&gt;    return ($value * 2);&lt;br /&gt;};&lt;br /&gt;&lt;/pre&gt;&lt;br /&gt;into this:&lt;br /&gt;&lt;pre&gt;my $sub := sub ($matrix, $value, $x, $y) {&lt;br /&gt;    return ($value * 2);&lt;br /&gt;}; &lt;br /&gt;&lt;/pre&gt;Problem solved, and now more tests are legitimately passing.&lt;br /&gt;&lt;br /&gt;It's been a productive couple of days in PLA, and I'm hoping I can keep this momentum up in the days ahead. I need to finish implementing my GEMM wrapper for ComplexMatrix2D, and then get started on the Linear Algebra module for Rakudo.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-5434604396199333838?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/5434604396199333838/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/08/pla-status-updates.html#comment-form' title='1 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/5434604396199333838'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/5434604396199333838'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/08/pla-status-updates.html' title='PLA Status Updates'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>1</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-7655421144280678942</id><published>2010-08-12T07:50:00.000-04:00</published><updated>2010-08-12T07:50:41.368-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='Testing'/><category scheme='http://www.blogger.com/atom/ns#' term='Parrot'/><category scheme='http://www.blogger.com/atom/ns#' term='Kakapo'/><category scheme='http://www.blogger.com/atom/ns#' term='Parrot-Linear-Algebra'/><title type='text'>Kakapo Fixed!! (Mostly)</title><content type='html'>The lovable Austin Hastings has gotten Kakapo mostly working as of yesterday. It's not 100%, but I'm able to use it for some purposes such as getting the PLA test suite passing again.&lt;br /&gt;&lt;br /&gt;When I tried to build PLA last night, for the first time in a long time, the build failed. Some of the string handling functions I was using in the get_string VTABLEs were using old, deprecated functions in the string API. I took this as my opportunity to &lt;a href="http://github.com/Whiteknight/parrot-linear-algebra/commit/444238a4a1746b3f5291f8687a11aed6660888a4"&gt;rewrite a lot of the logic&lt;/a&gt; to use the new &lt;a href="http://trac.parrot.org/parrot/browser/trunk/src/pmc/stringbuilder.pmc"&gt;StringBuilder&lt;/a&gt; type instead. This should improve performance and bring PLA up to some of the modern best-practices of the Parrot community.&lt;br /&gt;&lt;br /&gt;When I tried to run the tests, however, KABLAMO. Kakapo wasn't happy and the tests wouldn't run. I tracked down a small series of problems, and put together a hasty patch. Austin pointed out that this wasn't the real solution, so he didn't want me to push it to the master at Gitorious. Instead, &lt;a href="http://github.com/Whiteknight/kakapo/commit/a249c7709ee3fc21bb1431faf24a9693ee0ca982"&gt;I pushed to my mirror at Github&lt;/a&gt;. This fix out of the way, the vast majority of the PLA test suite runs, and most tests were passing. This morning, I pushed another patch to my Kakapo mirror. After that, I'm proud to say, the PLA test suite passes 100% of important tests. That means I can start making preparations for &lt;a href="http://wknight8111.blogspot.com/2010/07/kakapo-and-parrot-linear-algebra.html"&gt;an official release&lt;/a&gt;, and then begin development on cool new features.&lt;br /&gt;&lt;br /&gt;Some people have asked me why I've gone through all this hassle; if I had just rewritten my testsuite without Kakapo it could have been working a long time ago. The answer to that question is pretty simple: I want the abstraction and insulation that Kakapo provides.&lt;br /&gt;&lt;br /&gt;To look at it another way, consider this: NQP has a test suite of it's own that needs to pass before it can make a release. If I'm using NQP, I know that the behavior NQP provides will be a little more stable than the underlying Parrot VM. However, NQP isn't a perfect overlay for Parrot: There are some features that Parrot provides that NQP doesn't cleanly wrap. To get an abstraction layer over those features, I need a framework like Kakapo. This is all not to mention that several other high-profile projects use NQP, so even if it's test suite isn't all-encompassing, the test suites of projects which build on top of NQP probably cover more ground.&lt;br /&gt;&lt;br /&gt;Kakapo likewise has a test suite, and that needs to pass before Kakapo gets released. If I'm using Kakapo, I can be certain that the API that it and NQP provide together should be relatively stable and are tested to work before they are released. Since we are talking about &lt;a href="http://wknight8111.blogspot.com/2010/06/parrot-linear-algebra-change-of-course.html"&gt;tracking Parrot trunk&lt;/a&gt; with PLA development, this stability is very welcome.&lt;br /&gt;&lt;br /&gt;What we've run into recently is a rare confluence of events where highly disruptive changes in Parrot caused major breakages in Kakapo. When Kakapo got fixed, the PLA test suite ran perfectly without any changes to the suite itself. Considering the magnitude of changes to Parrot recently, the ability for me to continue running the test suite without alterations is a pretty big and important aspect. Sure there was some down-time, but once our dependencies were fixed, our stuff worked again like magic.&lt;br /&gt;&lt;br /&gt;Kakapo isn't 100% yet, but Austin is still working on it and I'll be lending a hand. I'm also going to start making some progress in PLA, and see what I can do to help out with some of the performance improvements and other big tasks that are going on in Parrot too. Expect to see many more updates on things in the coming days and weeks.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-7655421144280678942?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/7655421144280678942/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/08/kakapo-fixed-mostly.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/7655421144280678942'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/7655421144280678942'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/08/kakapo-fixed-mostly.html' title='Kakapo Fixed!! (Mostly)'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-581863388005288717</id><published>2010-07-30T20:02:00.001-04:00</published><updated>2010-07-30T20:31:51.666-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='Work'/><category scheme='http://www.blogger.com/atom/ns#' term='VisualBasic'/><title type='text'>Sic Semper VisualBasicus</title><content type='html'>Something I overheard today at work:&lt;br /&gt;&lt;br /&gt;&lt;blockquote&gt;We should hold a big VisualBasic conference, and once everybody is inside we gas the place.&lt;/blockquote&gt;&lt;br /&gt;It still boggles my mind that so much code is written in VisualBasic.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-581863388005288717?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/581863388005288717/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/07/sic-semper-visualbasicus.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/581863388005288717'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/581863388005288717'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/07/sic-semper-visualbasicus.html' title='Sic Semper VisualBasicus'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-7915154731754148775</id><published>2010-07-29T18:00:00.000-04:00</published><updated>2010-07-29T18:00:00.289-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='Prism'/><category scheme='http://www.blogger.com/atom/ns#' term='Work'/><category scheme='http://www.blogger.com/atom/ns#' term='Silverlight'/><category scheme='http://www.blogger.com/atom/ns#' term='VisualStudio'/><title type='text'>Silverlight Website</title><content type='html'>&lt;a href="http://wknight8111.blogspot.com/2010/07/visualstudio-2010.html"&gt;Yesterday I mentioned&lt;/a&gt; that I'm writing a new web application with Silverlight. I also expressed a few misgivings about the current level of support for Silverlight in VisualStudio and other tools. Today, I'm going to talk a little bit about the design of this new website and some of the decisions that we made to arrive at this design.&lt;br /&gt;&lt;br /&gt;Our current web application is written in ASP.NET/C# and uses a series of third-party plugin controls to add improved aesthetics, performance, and capabilities. This strategy works reasonably well, though I've personally found it to not be maintainable as-is. I've found that complexity is growing much more quickly than capability does, and that our limited development resources quickly get strained. This is especially true when we start running into impossible-to-debug errors in hundreds of lines of generated javascript. Our entire team has ground to a halt on more than one occasion while one intrepid developer suffered through long sessions of phone tech-support to resolve these kinds of issues.&lt;br /&gt;&lt;br /&gt;If we were using a combination of PHP and JavaScript, with a comprehensive client-side framework like jQuery things might be a little different. But, we're not. That ship has long-since sailed and there is no going back now.&lt;br /&gt;&lt;br /&gt;The fact is that my office is a C# house. All our developers know C#, all our projects are written in C#. Trying to switch our main web application to use PHP instead would require major retooling time and effort, retraining for several developers, lots of lost time and productivity, etc. C# is as good a language as any, and if we can get moving with the right tools and the right technologies it should work just as well as anything else.&lt;br /&gt;&lt;br /&gt;The problem I have with ASP is that it doesn't run directly, it generates HTML, CSS, and JavaScript. The result is that the source of the actual page which the client receives bears only the vaguest resemblance to the code which the developer wrote. Find a bug somewhere in the CSS? Unhandled JavaScript exception? Unless it is painfully obvious where the problem is coming from, it might be impossible for anybody to trace that problem directly back to the offending ASP code. Producing a robust and portable solution may be even harder.&lt;br /&gt;&lt;br /&gt;This is not to mention that there are major differences in JavaScript, HTML, and CSS implementations between browsers and there's no guarantee that anything you see on your development machine will look the same when the client gets it. I can guarantee that Microsoft tries hardest to make things look spiffy in Internet Explorer, but even then I have found some severe problems with ASP websites not rendering properly in various versions of IE. For modern browsers this usually isn't a huge problem anyway, but we deal with several clients who use IE6 by corporate mandate and refuse to update for "security concerns". Go figure.&lt;br /&gt;&lt;br /&gt;[Side Note: If your IT personnel claim that upgrading &lt;i&gt;from&lt;/i&gt; IE6 to something newer is a security concern, &lt;b&gt;fire them&lt;/b&gt;. This is the first sign, and not a trivial one, that the person in charge of your IT department is incompetent.] &lt;br /&gt;&lt;br /&gt;JavaScript differences between such a huge range of browsers is really a pretty serious concern. There's a reason why I like working with VMs, which provide a standard interface across multiple platforms. &lt;a href="http://www.parrot.org/"&gt;Parrot&lt;/a&gt; is my favorite, of course, but .NET/Mono earns a strong honorable mention too. The more we can write in a managed language like C#, and the less it gets converted to something less reliable, the better. Generated code is the enemy here, and HTML/CSS/JavaScript is really a terrible choice for target "language".&lt;br /&gt;&lt;br /&gt;These are the reasons why, in a nutshell, that I picked Silverlight for the new website. Toolchain support is sophomoric at best, but it does give us what we need otherwise: We can write everything in C#, we don't have to worry about differences in rendering, performance, or capability between different browsers. Even the poor IE6 folks have a Silverlight plugin installed. We can leverage the strengths of our existing team too, which is deadline-friendly. We have a good opportunity here to mature with our tools, and I've already filed a few bug reports and sent in some feedback to Microsoft about some of the issues I've found. All we need now is to wait for service pack 1 without killing ourselves or each other.&lt;br /&gt;&lt;br /&gt;The website requirements are a pretty nebulous beast, as I'm sure they are for any sufficiently-usable web application. Different customers all want to see different things and have different tools at their disposal. What we needed was a framework that would allow us to create a pluggable interface and support/enforce encapsulation and abstraction like any modern &lt;a href="http://en.wikipedia.org/wiki/Model_View_Presenter"&gt;MVP&lt;/a&gt; application wants. To do all these things, we decided to use &lt;a href="http://compositewpf.codeplex.com/"&gt;Prism&lt;/a&gt;. Prism is part framework and part "guidance" which provides tools, lessons, and examples to follow for building modular MVP or &lt;a href="http://en.wikipedia.org/wiki/MVVM"&gt;MVVM&lt;/a&gt; applications. Basically, it's a nice booster to make sure your team is following some modern best-practices when it comes to scalable and maintainable application design.&lt;br /&gt;&lt;br /&gt;Like any methodology, adhering too strictly to MVP or MVVM design can put you in an uncomfortable place, but if you take the lessons to heart and use the tools that you need, I've found that it works out pretty well. One particular nit that I do have about it is that aggressive decoupling of your classes leads&amp;nbsp; to referring to modules by name with strings, or System.Type objects, and you miss out on entire classes of analysis and error-detection tools that your compiler's type system would provide otherwise. "Be really careful" is a strategy that &lt;a href="http://en.wikipedia.org/wiki/Rythm_method"&gt;doesn't really work well in life&lt;/a&gt; and doesn't really work well in programming either. There's nothing worse than trying to track down an error which raises no compiler warnings and fails silently at runtime too without throwing an exception. Combine this with my inability to debug Silverlight from VisualStudio and I've run into a few situations where I almost pulled some hair out. Once I get my debugging issues sorted out this won't nearly be so much of a problem, but right now it makes for long days.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-7915154731754148775?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/7915154731754148775/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/07/silverlight-website.html#comment-form' title='2 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/7915154731754148775'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/7915154731754148775'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/07/silverlight-website.html' title='Silverlight Website'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>2</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-2814449459857351</id><published>2010-07-28T18:00:00.000-04:00</published><updated>2010-07-28T18:00:01.804-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='Parrot'/><category scheme='http://www.blogger.com/atom/ns#' term='Kakapo'/><category scheme='http://www.blogger.com/atom/ns#' term='Parrot-Linear-Algebra'/><title type='text'>Kakapo and Parrot-Linear-Algebra</title><content type='html'>&lt;a href="http://www.parrot.org/news/2010/Parrot-2.6.0"&gt;Parrot's 2.6 release is out&lt;/a&gt; and now everybody is holding their breath in anticipation of the new &lt;a href="http://github.com/rakudo/star"&gt;Rakudo Star &lt;/a&gt;release tomorrow. I'm still fighting with my own release project: &lt;a href="http://www.gitorious.com/kakapo/kakapo"&gt;Kakapo&lt;/a&gt;.Kakapo got broken pretty severely when Parrot was changed to &lt;a href="http://trac.parrot.org/parrot/ticket/389"&gt;no longer store methods in namespaces&lt;/a&gt;, and I have been unable in several weeks to make it work again. Austin has been busy with other projects during this time too, so it's been me by my lonesome to try and get this thing working again.&lt;br /&gt;&lt;br /&gt;The problems are three-fold: First, Austin's plan for Kakapo was ambitious and the framework does a hell of a lot of things, not all of which I fully understand. Second, NQP and P6object have progressed in some significant ways since Kakapo was originally inked out, and much of the workaround code that Austin wrote to bypass some missing features no longer seems necessary. Third, we have this issue with methods not working nicely with NameSpaces anymore, which makes all of Kakapo's method-injection logic break.Without method injection happening, and happening early enough, all the code in Kakapo which relies on the new methods is broken.&lt;br /&gt;&lt;br /&gt;Tene put together a &lt;a href="http://github.com/perl6/nqp-rx/commit/96bc19114a3962c356304c11b20768ef11fe91ba"&gt;patch for NQP&lt;/a&gt; which allows methods marked "our" to be stored in NameSpaces again. This should simplify some of the new logic in Kakapo though I've been too busy in the past few days to test it. Hopefully this patch of his represents a large portion of the fix to get Kakapo working again. I doubt it will be everything that's necessary, but it will be a nice start.&lt;br /&gt;&lt;br /&gt;Without working Kakapo I haven't done any work whatsoever on &lt;a href="http://github.com/Whiteknight/parrot-linear-algebra"&gt;Parrot-Linear-Algebra&lt;/a&gt;. PLA uses Kakapo to implement all it's unit tests, and I'm happy enough with the new testing framework that I do not want to move it to use anything besides Kakapo. In the worst case I would consider trying to break out a testing sub-framework from Kakapo if we can't fix the entire framework, but I haven't gotten to that point yet and likely won't have the time for it in the near future. I would much rather fix the framework as a whole first, and then maybe talk about splitting out the unit test stuff into a separate sub-library later, if I still feel like that's a beneficial path to take.&lt;br /&gt;&lt;br /&gt;What I would really like to be doing is getting back into PLA development. I had hoped to get some wrappers together to make it work with Rakudo Star, but without working tests I haven't cut a suitable release and without a release it won't be included in the Rakudo Star bundle anyway. &lt;br /&gt;&lt;br /&gt;My hope is that within the next few days I can get Kakapo working again. At least I need it working enough that I can get the unit tests for PLA going. With Tene's fix, some hard work, and some trial-and-error I think it will be possible.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-2814449459857351?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/2814449459857351/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/07/kakapo-and-parrot-linear-algebra.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/2814449459857351'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/2814449459857351'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/07/kakapo-and-parrot-linear-algebra.html' title='Kakapo and Parrot-Linear-Algebra'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-6883717111267678028</id><published>2010-07-27T21:13:00.001-04:00</published><updated>2010-07-28T08:12:13.788-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='Work'/><category scheme='http://www.blogger.com/atom/ns#' term='Silverlight'/><category scheme='http://www.blogger.com/atom/ns#' term='VisualStudio'/><title type='text'>VisualStudio 2010</title><content type='html'>The new web application at my work is going to be Silverlight-based, which gives me an opportunity to learn a fancy new technology and add another line of gibberish to my resume.&lt;br /&gt;&lt;br /&gt;As anybody who's worked with Silverlight will tell you, VisualStudio 2008 really doesn't help you at all. I don't have many nice things to say about XAML, and VS2008 requires you to write it by hand. We could use Expression Blend, but without a paid license you're using a crippled product, and that doesn't make any sense when you already have an expensive license to VisualStudio. Luckily, the beautiful IT people got us licenses for &lt;a href="http://en.wikipedia.org/wiki/Microsoft_Visual_Studio#Visual_Studio_2010"&gt;VisualStudio 2010&lt;/a&gt;, which boasts massively-improved Silverlight support.&lt;br /&gt;&lt;br /&gt;Notice that I said "Massively-improved", not "perfect" or even "good". Yes, VS2010 is head-and-shoulders above VS2008 in terms of Silverlight support. However, I'm having a hard time really appreciating those improvements in light of the other glaring deficiencies. And there are &lt;i&gt;many&lt;/i&gt; deficiencies still.&lt;br /&gt;&lt;br /&gt;In the marketing and documentation VisualStudio 2010 is sold as having "Silverlight support". What does that even mean? Yes there is a new WYSIWYG editor to help improve Silverlight GUI design which produces mostly-correct XAML code. It's far better than nothing, but I find it to still be ridiculously frustrating to do even simple operations. Working with grids is tiresome. Working with controls which need to dynamically size is even worse. I find myself several times an hour wishing I could use some CSS to add style information. I find myself wishing I could specify height in percent, instead of a fixed number, the ambiguous "Auto" or as a weighted ratio. None of these are even among the most damning criticisms I have.&lt;br /&gt;&lt;br /&gt;The set of built-in controls available to the Silverlight developer is paltry at best. If you want to write anything more complicated than a tic-tac-toe game, you're going to need to download some kind of add-in or third party package. Even some of the most basic controls that a Windows developer has grown accustomed to in the past 20 years are missing with no replacements in sight. All the best documentation, even documentation from Microsoft, recommends you download the &lt;a href="http://silverlight.codeplex.com/"&gt;Silverlight Toolkit&lt;/a&gt; to get back some of these basics. You have to wander why some of these necessary and recommended controls weren't just included in the standard libraries.&lt;br /&gt;&lt;br /&gt;Silverlight has support for several powerful graphical features like transforms, animations, storyboards, and projections, but no good ones are available by default. Get within 20 feet of PowerPoint and some knucklehead is going to abuse the star-wipe, but the basic Silverlight library doesn't even provide you with a basic wipe, fade, slide or expand/shrink. Combine that with the fact that VisualStudio2010 doesn't really provide you with any decent tools to create these things yourself, and you find a huge new source of frustration.&lt;br /&gt;&lt;br /&gt;And I've left my biggest complaint for last: Debugging. Let me add some emphasis here for clarity: &lt;b&gt;You cannot debug a Silverlight application in VisualStudio 2010 without downloading and installing a separate, and finicky, add-on. &lt;/b&gt;You cannot debug Silverlight applications, by default, in an IDE which boasts "Silverlight support". If you want to debug your Silverlight application (or do other advanced operations like "unit testing"), you need to download the &lt;a href="http://www.microsoft.com/downloads/details.aspx?FamilyID=40ef0f31-cb95-426d-9ce0-00dcfabf3df5&amp;amp;displaylang=en"&gt;Silverlight Tools for VisualStudio 2010&lt;/a&gt; package. I can't even imagine why they wouldn't have included this functionality directly in VisualStudio itself, if every Silverlight developer is going to need to download and install it separately for even the most basic tasks. That is a decision that boggles the mind and leaves me completely flabbergasted.&lt;br /&gt;&lt;br /&gt;But wait! The problems have not ended once you install those tools. If you install your add-ons in the wrong order, or pick the wrong settings in VisualStudio or &lt;a href="http://forums.silverlight.net/forums/t/188434.aspx"&gt;even your browser&lt;/a&gt;, or &lt;a href="http://boxbinary.com/2010/04/debugging-silverlight-in-visual-studio-breakpoints-not-being-hit/"&gt;unknowingly anger the gods of programming&lt;/a&gt;, you still &lt;a href="http://social.msdn.microsoft.com/Forums/en/Vsexpressvcs/thread/5224a8ba-df86-4f3c-8e72-41229c64ca62"&gt;won't be able to debug&lt;/a&gt;. I know, it happened to me. I haven't been able to set a breakpoint since I started this web application two days ago, and it is seriously hampering my coding mojo.&lt;br /&gt;&lt;br /&gt;I like Silverlight, generally. The reliance on XAML is a pretty lousy and short-sighted design decision, in my opinion, but isn't a killer. You do get rich client-side applications without having to write them in ActionScript or Java. I wish that VisualStudio's touted "support" was a little bit more impressive and comprehensive.&amp;nbsp; In short, I really hope that this platform and toolchain matures significantly in the coming years, because we have a growing web application that we're going to need to be expanding and maintaining for the foreseeable future.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-6883717111267678028?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/6883717111267678028/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/07/visualstudio-2010.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/6883717111267678028'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/6883717111267678028'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/07/visualstudio-2010.html' title='VisualStudio 2010'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-1502440153493497713</id><published>2010-07-10T18:30:00.001-04:00</published><updated>2010-07-10T18:42:09.228-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='Parrot'/><category scheme='http://www.blogger.com/atom/ns#' term='Lorito'/><title type='text'>Lorito: A Design</title><content type='html'>Cotto has been putting together a really great &lt;a href="http://trac.parrot.org/parrot/wiki/LoritoRoadmap"&gt;checklist for Lorito&lt;/a&gt;, the new microcoding approach that the Parrot developers are hoping to implement soon. The first step on the checklist, creating a proper compiler for our ops code &lt;a href="http://en.wikipedia.org/wiki/Domain_Specific_Language"&gt;DSL&lt;/a&gt; is already complete. Today at #parrotsketch &lt;a href="http://irclog.perlgeek.de/parrotsketch/2010-07-06#i_2523352"&gt;Allison also mentioned some things &lt;/a&gt;about Lorito, and how we should start focusing more effort on it. I, for one, am very happy about that.&lt;br /&gt;&lt;br /&gt;In this post I'm going to play architect and spell out a vision for Lorito. At the very least, this will be a good play-excercise. I don't expect everybody to like all the things I say here, but I do expect this post to generate some thought and discussion.&lt;br /&gt;&lt;br /&gt;On the ground floor, Lorito needs to be able to do what C code does now. Actually, it really needs to be able to do much of what the underlying hardware machine does now. C is just a useful abstraction over the hardware, one that people are familiar with. What we don't want to be doing is creating our own &lt;a href="http://en.wikipedia.org/wiki/Application_binary_interface"&gt;ABI&lt;/a&gt;, or creating new low-level calling conventions or anything like that. Those kinds of problems are already resolved and if we want to be able to tap into the myriad of existing libraries we will want to stay compatible with existing conventions.&lt;br /&gt;&lt;br /&gt;We want Lorito to interact with raw integers and pointers, including pointer dereferences and pointer arithmetic. This is important so that we can easily work with C-level arrays and pointers. I can't think of a modern architecture where pointers and integers are different sizes and are treated differently, but I can't say I'm familiar with every architecture in current use. I don't foresee any huge problems with creating an opset that works transparently with integers and pointers.&amp;nbsp; We also want Lorito to be able to call &lt;a href="http://en.wikipedia.org/wiki/X86_calling_conventions#cdecl"&gt;C-level functions&lt;/a&gt;, including indirect calls through a function pointer. There's no hard requirement yet stated that Lorito needs to be binary compatible with compiled C code, but I think we can all agree that such would be a major benefit. In fact, if we had a utility that could compile Lorito directly into C code, that would be the best intermediate step we could take. As I will discuss next, such a tool would make the conversion of Parrot to using Lorito easier in the long run. We probably don't want Lorito-to-C conversion to be a permanent part of our build, but it does get us moving now.&lt;br /&gt;&lt;br /&gt;The PASM ops are currently written in C. If Lorito does everything that C can do, eventually they could all be rewritten in Lorito instead. That raises the interesting idea that groups of Lorito ops can be composed into more complex higher-level ops. PASM then becomes little more than a set of predefined convenience compositions, though it certainly does not represent the complete set of possible compositions. Dynops would just be custom predefined Lorito compositions and, since they wouldn't be defined in machine-code libraries, their definitions could be included directly in bytecode for later use.&lt;br /&gt;&lt;br /&gt;In fact, while we are moving down this though path, we get to the idea that we could identify repeated patterns of Lorito ops at compile time, and define custom compositions on the fly. This allows us to compress packfiles for size without really adding all the overhead of general-purpose compression algorithms. When optimizing code, if we can apply an optimization to any composed op, even those identified on the fly, those optimizations can be immediately used anywhere the composition op is. This allows us to prune branches out of the parse tree when optimizing quickly.&lt;br /&gt;&lt;br /&gt;Any language for which a compiler exists to convert it to Lorito can be considered an "overlay" language for Lorito. We can then use any overlay language any place where we can use Lorito. For instance, if NQP is converted to output Lorito (and composition ops) directly, we can then use NQP to define all the PASM ops, all the core PMCs, and several parts of Parrot's core. That would be quite the turnaround from how NQP is used currently.&lt;br /&gt;&lt;br /&gt;Conversely, almost any C code can be rewritten in Lorito, so long as we have a step in the build to preprocess Lorito back into valid C code. In fact, I see no reason why we cannot interleave code written in both languages, in a manner analogous to how the Q:PIR construct allows PIR code to be written in NQP source. If we had a preprocessor that scanned code files and converted Lorito ops into equivalent C code line-by-line, we could start slowly rewriting much of Parrot, if not all of it, in Lorito. Then, once we have converted all the code over that we need, we could change the build process to convert those code files to bytecode instead of machine code, and run large parts of Parrot's code through the runcore.&lt;br /&gt;&lt;br /&gt;Alternatively, instead of writing core code in Lorito directly, we could write core code in any Lorito overlay language. NQP comes to mind here. &lt;br /&gt;&lt;br /&gt;The huge benefit to be had is that if user programs are written in a Lorito overlay, and Parrot itself is written primarily in Lorito or an overlay, our eventual JIT can record traces across the API boundary, and optimize hot sections of the Parrot core on the fly at runtime. &lt;br /&gt;&lt;br /&gt;Here ends the part of the blog post concerned with overarching architecture discussions. Let's get down to the nitty-gritty details.&lt;br /&gt;&lt;br /&gt;Lorito is going to want some basic arithmetic operations: addition, subtraction, multiplication and division on integers (pointers) and floating point numbers. We're also going to want modulus, address-of (C &amp;amp;) and pointer dereference (C unary *) for integer/pointer types, int-to-float and float-to-int cast operations. Logical operations are probably essential as well: and, or, xor, not. Logical inversion (C unary !) would probably be necessary as well. Assuming we can find a way to share ops for int/pointer and float operands (which I doubt we can do in an elegant way), that's still about 20 ops we're going to want to even perform basic manipulations on numbers. I can't imagine any modern, robust, mature virtual machine which doesn't perform these operations readily.&lt;br /&gt;&lt;br /&gt;On top of basic mathematical operations, we're going to need some important operations for calling subroutines: Passing and retrieving arguments, calling subroutines, returning with values. I don't know how low-level Lorito intends to get, but let's face the reality of the world: Hardware machines are stack-based. Every library function we want to call is probably going to be passing arguments on the system stack. If we want to be defiant and call these ops "pass_arg" and "retrieve_arg" to hide the fact that Parrot is using the system stack, that's fine by me. The "stacks are evil" mantra, while occasionally misguided, is definitely firmly ingrained in Parrot culture.&lt;br /&gt;&lt;br /&gt;In a minimalist world, the only ops we would really need are ops to call functions with passed arguments. We could implement every other op as a huge library of functions to call. Of course this would be extremely sub-optimal, but it does open a new possible avenue: Any operation which is sufficiently uncommon could be turned into a function call instead of having a dedicated op. We could encapsulate the call in a composite op. There are lots of options here, we have to weigh the desire to have a smaller op set against the need to have ready access to a huge myriad of low-level operations.&lt;br /&gt;&lt;br /&gt;Having ops to do basic ops is necessary but not sufficient. I don't think we can sit back on our laurels and be happy with an op set that only does a subset of what C can do. That's worthless. Parrot needs to provide access to it's PMC and STRING types, and also to its interpreter. We want ops to load registers with values from the constants table in the bytecode file. We want to treat PMCs and STRINGs as opaque pointers at the user-level. They aren't like other pointers which can be manipulated, they are GCable objects that Parrot treats specially. We don't, for instance, want to be passing a PMC pointer willy-nilly to an arithmetic operation. We also don't want such a mistake to imply a call to one of the addition VTABLEs.&lt;br /&gt;&lt;br /&gt;That said, we want to be able to get the VTABLE structure from a PMC, introspect information about that, and be able to call the various VTABLE interface functions there without having to calculate pointer offsets. This might make a good use of composite ops, where we can still do the pointer derefs in Lorito, but hide the nonsense behind some PASM-levle composite ops. We already have most of these ops already, but I think we're going to want to rework some of them. We definitely need to radically reduce the number of VTABLE interface functions, or else interacting with all of them from Lorito will be extremely ungainly.&lt;br /&gt;&lt;br /&gt;So far as we are talking about a major redesign of the system, maybe it's time to start considering an idea chromatic has been talking about with making all vtables into methods so we can share a common lookup mechanism, common dispatch mechanism, common MMD behaviors, common inheritance behaviors, etc. That's not a bad idea, but we either need dramatic improvements to PCC performance, or we need to create a PCC fast-path for calls which are made without constructing a call object and without doing too many marshalling operations to get arguments where they need to be.&lt;br /&gt;&lt;br /&gt;In fact, on a tangentially-related topic, I definitely think PCC should have a fast-path for functions which do not participate in MMD and which have fixed argument lists containing only positional args (no :optional, no :slurpy, no :named, no :call_sig, etc). This is, I think, an extremely common case and there are plenty of optimizations to be had here if we want them.&lt;br /&gt;&lt;br /&gt;To round out the low-level opset, we need ops to move data between registers, between registers and memory, and maybe even ops to move data between memory locations directly, without moving them to a register first.&lt;br /&gt;&lt;br /&gt;&lt;br /&gt;That's my conception for Lorito: probably 64 ops or less, a capability to add composite ops, treating PASM as a series of common composite ops, and the ability to write low-level code interchangably in C or Lorito. I think this plan for it provides a great development path without too many abrupt stops or course changes. We are quickly going to find ourselves in desperate need of a robust, modern JIT. We are going to be able to move to a proper precise GC since all the Lorito code will be using Parrot's registers to store working values, not relying on stack space which will need to be traced.&lt;br /&gt;&lt;br /&gt;I am highly interested in hearing what other people have to say about Lorito, and what changes Parrot should be considering so long as we are writing this blank check for a massive refactor.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-1502440153493497713?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/1502440153493497713/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/07/lorito-design.html#comment-form' title='5 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/1502440153493497713'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/1502440153493497713'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/07/lorito-design.html' title='Lorito: A Design'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>5</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-7781256503090597883</id><published>2010-06-25T13:30:00.000-04:00</published><updated>2010-06-25T13:30:45.299-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='MediaWiki'/><category scheme='http://www.blogger.com/atom/ns#' term='EmbedVideo'/><category scheme='http://www.blogger.com/atom/ns#' term='Release'/><title type='text'>EmbedVideo Version 1.0 Released</title><content type='html'>I'm pleased to announce the 1.0 release of the &lt;a href="http://www.mediawiki.org/wiki/Extension:EmbedVideo"&gt;EmbedVideo extension for Mediawiki&lt;/a&gt;.&lt;br /&gt;&lt;br /&gt;EmbedVideo is a highly capable and configurable extension for embedding flash video content from hosting sites such as YouTube. The extension is being &lt;a href="http://github.com/Whiteknight/mediawiki-embedvideo"&gt;developed on Github&lt;/a&gt; and is available for &lt;a href="http://github.com/Whiteknight/mediawiki-embedvideo/downloads"&gt;download from there&lt;/a&gt; as well. Information is also available on the &lt;a href="http://www.mediawiki.org/wiki/Extension:EmbedVideo"&gt;extension page at mediawiki.org&lt;/a&gt;.&lt;br /&gt;&lt;br /&gt;Major new features include:&lt;br /&gt;&lt;ul&gt;&lt;li&gt;Completely rewritten code base to follow current MediaWiki best practices&lt;/li&gt;&lt;li&gt;i18n internationalization support&lt;/li&gt;&lt;li&gt;Support for new hosting services&lt;/li&gt;&lt;ul&gt;&lt;li&gt;&lt;a href="http://www.interia.pl/"&gt;Interia&lt;/a&gt;&lt;/li&gt;&lt;li&gt;&lt;a href="http://www.edutopia.org/"&gt;Edutopia&lt;/a&gt;&lt;/li&gt;&lt;li&gt;&lt;a href="http://teachertube.com/"&gt;TeacherTube&lt;/a&gt;&lt;/li&gt;&lt;li&gt;&lt;a href="http://www.divshare.com/"&gt;Divshare&lt;/a&gt; &lt;/li&gt;&lt;/ul&gt;&lt;li&gt;Backwards-compatibility support for EmbedVideoPlus&lt;/li&gt;&lt;li&gt;Thumbnailing, floating alignment, and attached descriptions&lt;/li&gt;&lt;li&gt;Support for custom embedding clauses&lt;/li&gt;&lt;li&gt;Support for per-service default width and width/height ratios&lt;/li&gt;&lt;/ul&gt;Contributions and patches, especially those that add support for new video hosting services, will be much appreciated.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-7781256503090597883?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/7781256503090597883/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/06/embedvideo-version-10-released.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/7781256503090597883'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/7781256503090597883'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/06/embedvideo-version-10-released.html' title='EmbedVideo Version 1.0 Released'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-164633329631455407</id><published>2010-06-23T21:10:00.000-04:00</published><updated>2010-06-23T21:10:10.422-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='Parrot'/><category scheme='http://www.blogger.com/atom/ns#' term='Parrot-Linear-Algebra'/><title type='text'>Parrot-Linear-Algebra: Change of Course</title><content type='html'>I've added a lot of functionality to PLA in the last few days, including a large boost to the test suite this evening. Since Kakapo doesn't have a release that targets Parrot 2.3.0, I decided to refocus my efforts on Parrot 2.6.0. However, with all the changes in Parrot since 2.3.0 came out, the upgrade path was much harder than I anticipated. Luckily I have some superstars like &lt;a href="http://github.com/Whiteknight/parrot-linear-algebra/commit/1f0d9f839cf3060fe5b4befe98206401c29cc507"&gt;darbelo and NotFound&lt;/a&gt; on my side, otherwise I would still be fighting with things.&lt;br /&gt;&lt;br /&gt;After talking to darbelo I've decided PLA is not going to attempt to track supported releases, at least not for normal development. It's too much of a pain in the ass, and nobody else is doing it. Sure it's best practice, but it's the kind of thing the whole ecosystem really needs to do together. If the HLL compiler is tracking Parrot HEAD because of a desperate need of new features and improved performance, then it doesn't make any sense for libraries not to be keeping up pace.&lt;br /&gt;&lt;br /&gt;Darbelo made one particularly good point:&lt;br /&gt;&lt;br /&gt;&lt;blockquote&gt;If we track HEAD the we'll run on 2.6 when it's released. We branch on the release day and do whatever 'release engineering' we need to do on the branch and tag a release when we're done.&lt;/blockquote&gt;&lt;br /&gt;That's it, in a nutshell: If we track trunk closely enough, we will always be ready with a release following Parrot's release. Spend a few days to prepare the necessary accouterments, then cut a tag and continue on with normal development. Our releases can be tied to Parrot's supported releases, but our ongoing development will target trunk as closely as possible. I don't anticipate that we will want to cut a PLA release every time Parrot does, but if we follow a 4-month schedule and follow supported releases that should be just fine. At least, fine for now.&lt;br /&gt;&lt;br /&gt;PLA is going to track Parrot trunk and hopefully be prepared to cut a release following Parrot 2.6.0. That gives us about 4 weeks to get all our ducks in a row (and our Kakapos too!).&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-164633329631455407?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/164633329631455407/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/06/parrot-linear-algebra-change-of-course.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/164633329631455407'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/164633329631455407'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/06/parrot-linear-algebra-change-of-course.html' title='Parrot-Linear-Algebra: Change of Course'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-5412529727912985922</id><published>2010-06-22T17:00:00.001-04:00</published><updated>2010-06-22T17:00:00.654-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='Parrot'/><category scheme='http://www.blogger.com/atom/ns#' term='Parrot-Linear-Algebra'/><title type='text'>BLAS, LAPACK, and Parrot-Linear-Algebra</title><content type='html'>Last night I got back to work on Parrot-Linear-Algebra, my matrix package for Parrot. I'm really starting to get happy with the functionality it provides and am looking to start moving the project to the next level of functionality and usability.&lt;br /&gt;&lt;br /&gt;I hadn't touched it in a while, for a variety of reasons. Austing Hastings has been very busy with other projects and hasn't cut an "official" release of &lt;a href="http://www.gitorious.com/kakapo/kakapo"&gt;Kakapo&lt;/a&gt; that works with the last supported Parrot release, 2.3. I've applied some hotfixes that make Kakapo build and pass tests on 2.3, but that's not quite good enough. When I make a PLA release I don't want "clone the Kakapo repository from Gitorious" in the instructions, because then the instructions get out of date when Kakapo is updated to be incompatible. What I want is to say "Download Kakapo version X from this URL", which will be invariant. &lt;br /&gt;&lt;br /&gt;Last night I added some prototype new features to the NumMatrix2D PMC, the basic numeric matrix type. I'm going to mirror the majority of those changes to the two other general-purpose types; ComplexMatrix2D and PMCMatrix2D. I &lt;a href="http://github.com/Whiteknight/parrot-linear-algebra/blob/471954c9223f57cdedc4aa48cab5b68a908ef58c/src/pmc/nummatrix2d.pmc#L1025"&gt;added methods&lt;/a&gt; to do basic &lt;a href="http://en.wikipedia.org/wiki/Elementary_row_operations"&gt;elementary row operations&lt;/a&gt; (swap two rows, add a multiple of one row to another, and multiply the contents of a row by a non-zero constant), and used those operations to put together an example program to &lt;a href="http://github.com/Whiteknight/parrot-linear-algebra/commit/471954c9223f57cdedc4aa48cab5b68a908ef58c#diff-0"&gt;calculate both the Row Echelon and Reduced Row Echelon&lt;/a&gt; forms of a matrix. Those methods, combined with block manipulation methods I added previously and the new &lt;a href="http://en.wikipedia.org/wiki/GEMM"&gt;GEMM&lt;/a&gt; method I added&lt;a href="http://github.com/Whiteknight/parrot-linear-algebra/commit/c84ff7596f5dc2bae4d1bdf5bb5a8124ea913cbc"&gt; last night as well&lt;/a&gt;, create a basis for us to create a comprehensive library of linear algebra routines.&lt;br /&gt;&lt;br /&gt;But that brings me to my next concern: How to implement the remainder of the fundamental algorithms a linear algebra pack is going to want to provide? Obviously I would love to wrap the LAPACK library up and call the routines it provides directly. LAPACK provides a huge number of routines for doing things like singular value decomposition, QR, LU, and other types of decomposition, solving the eigen probem, solving systems of equations in two variables, etc. In fact, LAPACK provides almost all of the functionality I would want PLA to have in the next few months.&lt;br /&gt;&lt;br /&gt;The problem, however, is that LAPACK is not nearly as accessible from C code as I would like. In fact, there is no "standard" for C-bindings to the library, and several lame attempts are available that tend to be incompatible with each other. The reference version, and only reliably-available version, of LAPACK is written in FORTRAN. The standard CLAPACK library is just a machine-lead translation of the FORTRAN sourcecode, with a few points in the code needing to be manually tweaked after conversion. It has a few problems, including the fact that every single function parameter (even basic ints and floats) must be passed by reference. The ATLAS library, which PLA currently prefers to provide BLAS bindings, provides some LAPACK C bindings of it's own, but only a very very small subset of all LAPACK functions are provided, and the ones it does have hardly support all the operations PLA is going to want to provide.&lt;br /&gt;&lt;br /&gt;CLAPACK, being more or less the standard could be made to work, except that it doesn't come as any sort of user-friendly package. There are no CLAPACK packages for Debian (Ubuntu) or RedHat that I have seen, and that raises the barrier to entry significantly, something that I want to avoid.&lt;br /&gt;&lt;br /&gt;I could use the LAPACK library directly, and&lt;a href="http://www.yolinux.com/TUTORIALS/LinuxTutorialMixingFortranAndC.html"&gt; twist my code to match the FORTRAN calling conventions&lt;/a&gt; and data alignments. That's not unthinkable, though it would require some more work on the part of PLA developers than any other solution.&lt;br /&gt;&lt;br /&gt;I could skip LAPACK entirely, and instead rely on something like GSL with proper C bindings built-in. GSL does provide &lt;a href="http://www.gnu.org/software/gsl/manual/html_node/Linear-Algebra.html"&gt;several important matrix operations and decompositions&lt;/a&gt;, though I would need to do more research into the capabilities of that library.. What I don't want is to lose focus and have this project grow to try and become a general wrapper for all of GSL. I want PLA to stay focused on Linear Algebra only. We could maybe create sister projects to encapsulate more of the GSL functionality and use PLA under the hood to implement the basic data structures, of course.&lt;br /&gt;&lt;br /&gt;Maybe we will get lucky, and the new NCI system will have support for calling FORTRAN routines from shared libraries. I don't think we can just anticipate this kind of functionality, at least not during the GSoC program. &lt;br /&gt;&lt;br /&gt;A "nuclear" option, perhaps, would be to not rely on any external library for these things and instead brew up all the basics myself. I'm not against such work &lt;i&gt;per se&lt;/i&gt;, but it would be a huge investment in time and the case cannot be made that it's the best use of my limited development time. I did put together a Gauss-Jordan elimination routine last night, it wouldn't be too too much effort to put together a generalized QR decomposition algorithm and a singular-value decomposition algorithm, followed by routines to calculate eigenvalues, eigenvectors, and matrix inverses from those things. If PLA had a larger team of active developers who wanted to participate in this kind of work it would be an easier decision to make, but if it's primarily me and a handful of occasional-contributors, this really isn't a doable task.&lt;br /&gt;&lt;br /&gt;My plan for PLA in the near future is this: After the 2.6 release I want to push for a stable release of Kakapo, and then using that I want to cut a release of PLA. From that point forward, PLA will target the 2.6 version of Parrot at least until 2.9 and maybe later. The first release of PLA is going to provide three basic matrix types: A 2D matrix of floats, a 2D matrix of complex numbers, and a 2D matrix of PMCs. These three matrix types will have a large base of common functionality and each type will have some additional functionality too, as required by that type. Everything provided will be well-tested (including error cases, which I haven't exercised nearly enough so far) and some example programs will be provided in both PIR and NQP.&lt;br /&gt;&lt;br /&gt;There are a few projects I am envisioning to start in the future that will rely on PLA, so I really am hoping to create a nice, stable, functional release within the next few months. I'll post more information about any other projects as they arise.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-5412529727912985922?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/5412529727912985922/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/06/blas-lapack-and-parrot-linear-algebra.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/5412529727912985922'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/5412529727912985922'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/06/blas-lapack-and-parrot-linear-algebra.html' title='BLAS, LAPACK, and Parrot-Linear-Algebra'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-7302838359471348057</id><published>2010-06-19T09:24:00.000-04:00</published><updated>2010-06-19T09:24:00.357-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='Policy'/><category scheme='http://www.blogger.com/atom/ns#' term='Deprecation'/><category scheme='http://www.blogger.com/atom/ns#' term='Parrot'/><title type='text'>Parrot's Deprecation Policy</title><content type='html'>Parrot user kthakore sent a &lt;a href="http://lists.parrot.org/pipermail/parrot-dev/2010-June/004423.html"&gt;very interesting email&lt;/a&gt; to the Parrot developers list this week to criticize the current deprecation policy. After taking some time off from development, he returned to find that his extension no longer built on Parrot HEAD and he couldn't quite figure out why. There &lt;a href="http://trac.parrot.org/parrot/changeset/47190/trunk/DEPRECATED.pod"&gt;was a deprecation notice for the feature that changed&lt;/a&gt;, but the notice was so vaguely worded and so short on explanation that it offered very little help to the beleaguered extension developer.&lt;br /&gt;&lt;br /&gt;When we're talking about code, there are only a handful of things that we really need to worry about: utility, performance, security, stability and maintainability. Well-written code, for the most part, can satisfy all these requirements. Sometimes trade-offs are made, such as trading a certain amount of maintainability to optimize for performance, but in those cases some well-placed comments can help alleviate or minimize any regressions. Code, while technical and deep, is often pretty easy: we follow rules and policies, make changes, measure results, wash, rinse, repeat.&lt;br /&gt;&lt;br /&gt;Not so easy are the softer sides of open source software: the people. People come in varieties: core developers, extension developers, testers, documenters, end-users and well-wishers of varying levels of technical competency. Keeping all these groups of people working together nicely and happily can be quite a difficult challenge, and there is likely no way for all groups to be 100% happy 100% of the time. The tradeoffs here are harder to understand, harder to manage, and a misstep can have huge negative consequences for the project.&lt;br /&gt;&lt;br /&gt;I've long been a detractor of Parrot's &lt;a href="http://trac.parrot.org/parrot/browser/trunk/docs/project/support_policy.pod"&gt;current deprecation policy&lt;/a&gt;: It's too rigid, too narrow, and doesn't really do anything to help the people who need helping. It also doesn't really take into account Parrot's current stage in the development life cycle. Parrot, as I've mentioned on occasion, has &lt;a href="http://wknight8111.blogspot.com/2010/05/bright-blue-yonder.html"&gt;plenty of warts&lt;/a&gt; and has suffered many &lt;a href="http://wknight8111.blogspot.com/2010/05/fitness-of-parrot-as-target-platform.html"&gt;growing pains&lt;/a&gt;. It is folly to think that we should be making blanket guarantees about what &lt;a href="http://wknight8111.blogspot.com/2010/01/parrot-20-personal-retrospective.html"&gt;will or will not be present in various releases&lt;/a&gt;, or how quickly we can or cannot make changes.&lt;br /&gt;&lt;br /&gt;When there's a problem or a bug or a misfeature, people want those things fixed quickly. A good example of this were the &lt;a href="http://wknight8111.blogspot.com/2009/10/pcc-branch-lands.html"&gt;PCC refactors a few months ago&lt;/a&gt;. Even though those refactors created some backwards-incompatible behavior our users (who the deprecation policy was at least nominally designed to protect) were trying to &lt;a href="http://wknight8111.blogspot.com/2009/10/pcc-hackathon-day.html"&gt;rush them through&lt;/a&gt;. Rakudo developers specifically were blocking on the PCC improvements and having to wait for months and months would have been bad for them. The PCC refactors were high priority and high need.&lt;br /&gt;&lt;br /&gt;The deprecation of several core ops and their conversion to dynops recently is an example from the other end of the spectrum (and the source of kthatkore's frustration). While we followed the letter of the deprecation policy, these things didn't &lt;i&gt;need&lt;/i&gt; to be removed with haste and created a bit of hassle for users who weren't expecting it. I'm not saying they shouldn't have been removed (I'm always &lt;a href="http://wknight8111.blogspot.com/2010/03/lean-and-mean-parrot.html"&gt;a proponent of removing cruft&lt;/a&gt;), but it does expose some short-comings of our deprecation policy and process.&lt;br /&gt;&lt;br /&gt;What we have is a series of conflicting motivations, even for individual groups. Consider:&lt;br /&gt;&lt;ul&gt;&lt;li&gt;&lt;b&gt;Core Developers&lt;/b&gt;: Core developers want to remove bad features, want not to maintain bad features, want to add new good features and want to create the best software for the users. Developers need to work on fun new features, but also need their work to be used and appreciated. Take away either of those pieces, and many volunteer developers will simply walk away. Moving too quickly alienates the users and creates a huge disconnect between what the developers are developing and what the users are actually using. Moving too slowly is boring and developers start to leave for greener pastures.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Extension Developers&lt;/b&gt;: Want to add new extensions to the Parrot ecosystem for the benefit of users, but have to deal with binary compatibility at the C level which changes much more frequently than the "normal" user-visible interfaces like PIR. Parrot has a lousy extension API right now, so necessary improvements there require extension developers to stay up-to-date with current core development. At the same time, all sorts of other changes break things in new versions, even when necessary features are fixed. &lt;/li&gt;&lt;li&gt;&lt;b&gt;Users&lt;/b&gt;: For stability, it's good for users not to upgrade. To fix bugs and get new features, it's good t upgrade. Upgrading brings rewards but also hassles:&amp;nbsp; Core features disappear, new bugs are added, and extensions are broken by binary incompatibilities. Upgrading Parrot means needing to upgrade (or, at least, rebuild) extensions too.&lt;/li&gt;&lt;/ul&gt;It's difficult to tell an extension developer to stick to the supported releases because the extending API is so lousy and incomplete. Having to wait 3 months or more for a necessary change is hard for these small projects, and their developers can quickly lose interest and motivation. Until we reach some basic level of usability, we have to expect that extension developers are going to be tracking Parrot HEAD more or less closely. I think it's a little disingenuous to simultaneously expect fully that developers will be tracking the repository HEAD but also write in our deprecation policy that they should only track the stable releases, and you really need to ask who exactly that policy is designed to protect in this case.&lt;br /&gt;&lt;br /&gt;We really need to account for different levels of user and different levels of feature. End-users shouldn't be using Parrot directly. Joe Schmoe at home is not and definitely &lt;a href="http://wknight8111.blogspot.com/2010/01/problem-with-pir.html"&gt;should not be writing his tools in PIR&lt;/a&gt;. If he's writing his code in a higher-level language like Rakudo Perl 6, NQP, or Winxed, he's buffered from disruptive changes made to the Parrot core VM. It's the developers of HLL compilers and extensions that need to worry about these kinds of changes.&lt;br /&gt;&lt;br /&gt;Likewise, we need to differentiate between issues of multiple severities.When a big issue is blocking development in extensions and HLL compilers, it behooves Parrot to ignore the mandatory wait period and to make those fixes post haste. Alternatively, a change which is not necessary and would cause a block or a slow-down for extension developers should be put off for longer and made with more care.&lt;br /&gt;&lt;br /&gt;What we need, in a nutshell, is a policy that actually does what we claim the current deprecation policy does now: &lt;b&gt;Protect our users from disruptive changes&lt;/b&gt;, but also &lt;b&gt;enable forward progress to be made without being forever tied to every bad decision ever made&lt;/b&gt;. I suggest these changes to policy and process:&lt;br /&gt;&lt;br /&gt;&lt;ol&gt;&lt;li&gt;Deprecation should be added before a disruptive change is made &lt;/li&gt;&lt;li&gt;The deadline for the deprecation shouldn't be blindly tied to the next stable release, but &lt;b&gt;intelligently selected with input from the affected parties&lt;/b&gt;. We do have weekly planning meetings where these kinds of things can be decided. If we need to regularly schedule additional meetings with other parties (HLL compiler devs, extension devs, etc) we should do that as well.&lt;/li&gt;&lt;li&gt;Deprecations should be well-documented and publicized. Information about the deprecation should include what exactly is changing, how users of those features can work around the changes, and who to contact when/if problems arise.&lt;/li&gt;&lt;li&gt;Information about the deprecation should be sent to the users, not just dumped in DEPRECATED.pod where we expect people to be looking regularly. An email list for this purpose was suggested and I like that idea (other ideas were also suggested that I also like). Any way to send the information directly to the user is a good thing. &lt;/li&gt;&lt;li&gt;Where possible, both old and new versions should &lt;a href="http://lists.parrot.org/pipermail/parrot-dev/2010-June/004430.html"&gt;be provided simultaneously&lt;/a&gt; for a period of time while users transition. This is most important in the C-level API where function wrappers can easily be provided to translate old calls into new ones.&lt;/li&gt;&lt;/ol&gt;I'm still a big proponent of the idea that the deprecation policy should be opt-in, in the sense that only features that we've put a stamp of approval onto should be covered under the deprecation policy and anything else should not be relied upon. You &lt;i&gt;can&lt;/i&gt; use a feature that we haven't approved, but then you're responsible for paying the price when that feature changes or disappears completely. You would also be responsible for sending the core developers feedback about which features you would like to see be added and approved.&lt;br /&gt;&lt;br /&gt;Having a deprecation policy is a good thing and a necessary part of a mature software project. However, the policy we have currently fails on a number of counts and requires some serious re-thinking if we want to make it better. I sincerely hope, and I know several other people also hope, that we do make it much better in the future.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-7302838359471348057?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/7302838359471348057/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/06/parrots-deprecation-policy.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/7302838359471348057'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/7302838359471348057'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/06/parrots-deprecation-policy.html' title='Parrot&apos;s Deprecation Policy'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-8448603728564444292</id><published>2010-06-18T09:54:00.000-04:00</published><updated>2010-06-18T09:54:28.732-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='Parrot'/><category scheme='http://www.blogger.com/atom/ns#' term='Threads'/><title type='text'>ParrotTheory: Locks and Synchonization</title><content type='html'>I've talked a good amount in the past and recently about threading, so today I'm going to talk about some of the small bits that make threading and concurrency work. I'm thinking that we're also going to need to implement some of these primitives in Parrot eventually, so consider this blog post an extremely verbose way of planning for that.&lt;br /&gt;&lt;br /&gt;Let's start the discussion by looking at two small code snippets; a structure definition and a routine that uses that structure:&lt;br /&gt;&lt;pre&gt;typedef struct _array {&lt;br /&gt;  int length;&lt;br /&gt;  int *values;&lt;br /&gt;} array;&lt;br /&gt;&lt;br /&gt;void push( array *ary, int x) { &lt;br /&gt;  array-&amp;gt;length++; &lt;br /&gt;  realloc(array-&amp;gt;values, newlen * sizeof(int));&lt;br /&gt;  array-&amp;gt;values[array-&amp;gt;length - 1] = x;&lt;br /&gt;}&lt;br /&gt;&lt;/pre&gt;This isn't nearly as contrived an example as it may seem initially, though I purposefully made the code a little bit naive. It's worth noting here that the idea of a structure which contains both an array and the integer length of that array is very common. It's how Parrot implements it's STRING type and several of its array PMC types as well. In fact, the push_int VTABLE method for the ResizableIntegerArray PMC probably looks extremely similar to this example code.&lt;br /&gt;&lt;br /&gt;Astute observers, and veterans of threading trench warfare will see a problem with this code: There are no locks and no concurrency safeguards, so this code isn't thread safe. Let's take a short walkthrough of this code where we have a preemptive thread switch in the middle to another thread also attempting to access this same method on the same object:&lt;br /&gt;&lt;ol&gt;&lt;li&gt;We have an array object with length 5 and items {1, 2, 3, 4, 5}&lt;/li&gt;&lt;li&gt;Thread A attempts to push the number 6 to the array, Thread B attempts to push the number 7.&lt;/li&gt;&lt;li&gt;Thread A enters the function. We set length to 6 and realloc to gain a sixth slot. The array now contains values {1, 2, 3, 4, 5, ?}, because the last value isn't initialized by realloc.&lt;/li&gt;&lt;li&gt;At this point, there is a preemptive thread switch and Thread B takes over control.&lt;/li&gt;&lt;li&gt;Thread B enters the function and sets length to 7 and reallocs again. The array now contains values {1, 2, 3, 4, 5, ?, ?}.&lt;/li&gt;&lt;li&gt;Thread B sets the sixth element to 7. The array is now {1, 2, 3, 4, 5, ?, 7}&lt;/li&gt;&lt;li&gt;Thread B gets preempted, Thread A takes over.&lt;/li&gt;&lt;li&gt;In thread A, the value of length is still 7, so [ary-&amp;gt;length - 1] is still 6. When we add x to the array, we now have {1, 2, 3, 4, 5, ?, 6}&lt;/li&gt;&lt;/ol&gt;&amp;nbsp;There are some things we could try here, such as saving the values we need from the structure to local variables, so that even if we preempt in the middle of the functions the length field will be an accurate index. Or, we can try to rearrange the order of operations so some errors appear less frequently or less obviously.&amp;nbsp; The real problem here is that we have three operations that need to stay coupled: Increasing the known length of the array, reallocating array storage, and assigning an item to that storage. Any break between a set of coupled operations causes a problem. We call these areas of code where operations are sensitive to cross-thread interference &lt;b&gt;critical sections&lt;/b&gt;.&lt;br /&gt;&lt;br /&gt;As another example of a very small critical section, consider this one line of code:&lt;br /&gt;&lt;pre&gt;foo = i++;&lt;br /&gt;&lt;/pre&gt;This seems pretty simple, but in fact it is not. This line of code requires several assembly language instructions, especially when we are talking about a non-optimized build:&lt;br /&gt;&lt;ol&gt;&lt;li&gt;Fetch the value of i from memory into a processor register&lt;/li&gt;&lt;li&gt;Copy the value of i to the variable foo&lt;/li&gt;&lt;li&gt;Increment the register containing the value for i&lt;/li&gt;&lt;li&gt;Copy the value of the register back to memory where i is located.&lt;/li&gt;&lt;/ol&gt;A preemptive thread switch can happen in between any of these steps. Consider the case where we break between steps 2 and 3 to another thread performing the same operation: If i is 5, Thread 1 foo is 5, Thread 2 Foo is 5, and at the end of the code snippet i is 7! If this code snippet is, for instance, in an SQL database generating unique integer keys for rows in a particular table, we've just generated non-unique keys and created a world of hurt for ourselves.&lt;br /&gt;&lt;br /&gt;To get around these kinds of problems, one solution is to use a &lt;b&gt;synchronization primitive&lt;i&gt;. &lt;/i&gt;&lt;/b&gt;A synchronization primitive is any of a class of algorithms and objects that are designed to synchronize and limit access to shared resources. In this sense, a &lt;b&gt;resource&lt;/b&gt; is anything that multiple threads might want to access: An IO stream, a global variable, a shared pointer, even a sensitive sequence of instructions,&amp;nbsp; etc. Any time we have a critical section of code that is sensitive to sharing we want to find a way to limit access to a finite number of simultaneous threads (usually one). There are several ways to do this.&lt;br /&gt;&lt;br /&gt;A &lt;b&gt;mutex&lt;/b&gt;, short for "mutual exclusion", object is a type of &lt;b&gt;lock&lt;/b&gt; that helps to prevent access to a critical section. To pick a pertinent example, think of a mutex like a basketball. Only the one person in the game with the basketball can do things: shoot, pass and dribble. Other players on the team can do other stuff like running, covering, or posting, but they cannot do ball stuff without the ball. You cannot shoot the ball if you do not have the ball, you cannot pass the ball if you do not have the ball. This is a convention of the sport. If we were playing Calvinball instead, maybe we could do these things without looking preposterous. By convention also, if we as programmers declare that a certain shared resource can only be accessed by a thread (player) with the mutex (ball), the those are the rules for our system (game) and things can move along smoothly. Here's an example of that convention in action:&lt;br /&gt;&lt;pre&gt;Mutex *m;&lt;br /&gt;AQUIRE_MUTEX(m);&lt;br /&gt;// critical section code&lt;br /&gt;RELEASE_MUTEX(m);&lt;br /&gt;&lt;/pre&gt;The power in this code is that the AQUIRE_MUTEX() function will attempt to gain ownership of the mutex, and will wait indefinitely for the mutex to become available if some other thread already owns it. ACQUIRE_MUTEX is like waving your arms in the air, shouting "I'm open" until the current ball carrier passes the ball to you. Until you get the ball, you just have to stand with your arms in the air until you get it. Because of that behavior, no two threads can enter the same critical section, assuming of course that the programmer (you) has properly protected that critical section with the mutex. Keep in mind that there is no intrinsic property of the critical section itself that prevents multiple threads from running it simultaneously and corrupting data. The exclusion comes from the proper and pervasive use of locks like our mutex to keep the critical section safe. Here's another example:&lt;br /&gt;&lt;pre&gt;Mutex m;&lt;br /&gt;&lt;br /&gt;int pop(array* a) {&lt;br /&gt;  ACQUIRE_MUTEX(m);&lt;br /&gt;  int item = a-&amp;gt;values[a-&amp;gt;length - 1];&lt;br /&gt;  a-&amp;gt;length--;&lt;br /&gt;  RELEASE_MUTEX(m);&lt;br /&gt;  return item;&lt;br /&gt;}&lt;br /&gt;&lt;br /&gt;void push(array* a, int item) {&lt;br /&gt;  a-&amp;gt;values[a-length] = item; &lt;br /&gt;  a-&amp;gt;length++;&lt;br /&gt;}&lt;br /&gt;&lt;/pre&gt;In this example we can see that we aren't properly using the mutex everywhere, so we can't guarantee that we won't get corrupt data. Multiple threads could just as easily enter the push function simultaneously as could attempt to enter the pop function. If you don't use mutexes everywhere, it's almost as good as not using them anywhere. This is a convention that the coder must decide upon beforehand and follow diligently.&lt;br /&gt;&lt;br /&gt;&lt;br /&gt;There are multiple ways to implement locks and mutexes. One idea is a &lt;b&gt;spinlock&lt;/b&gt;, which attempts to access a flag and enters an endless while-loop until it can. An empty while-loop can be very inefficient on a processor, but if we call a sleep command inside the loop to allow other threads to run while we wait it isn't such a big problem. Spinlocks implemented by the OS inside the kernel event loop can be very efficient indeed. In fact, as a general rule, if the OS implements locking primitives they tend to be much better to use than anything you can write in userspace.&lt;br /&gt;&lt;br /&gt;Another type of lock primitive is a &lt;b&gt;semaphore&lt;/b&gt;, though it is subtly different. A semaphore allows a finite number of threads to access a finite number of shared resources at a time. Where a normal mutex, like a spinlock, allows only one thread to enter at a time the semaphore may allow one or more. Consider a case where we have five worker threads in a web server, and 100 incoming connections. A semaphore uses a first-come-first-served method to assign incoming connections to available threads. Each incoming connection attempts to access the semaphore. As requests are completed, threads signal their availability and the semaphore assigns the next connection in the list to that thread. A semaphore with only one shared object acts like a normal mutex or spinlock.&lt;br /&gt;&lt;br /&gt;The &lt;b&gt;overhead&lt;/b&gt; of a lock is the amount of effort it takes to acquire and manage the lock. In a uniprocessor system the lock may be very simple to obtain: First disable interrupts so we cannot be preempted by another thread, check the status of the lock, obtain the lock if it's available, and re-enable interrupts. In a multiprocessor system, especially one with shared memory, the overhead and error-checking involved can be much higher. In these systems the performance gain from using threads can be much higher too, so it's a trade-off.&lt;br /&gt;&lt;br /&gt;&lt;b&gt;Granularity&lt;/b&gt; is the amount of stuff in your critical section protected by a lock. &lt;b&gt;Course Granularity&lt;/b&gt; means that we have lots of code inside our critical section. This is good because we need fewer locks and therefore experience lower overhead. Plus, it's easier as a programmer to make sure we acquire fewer locks over large swaths of our program. The downside is that the larger our protected critical section is, the more likely other threads are going to be blocked waiting to enter it. This, in turn, can create problems like high &lt;b&gt;latency.&lt;/b&gt; &lt;b&gt;Fine Granularity&lt;/b&gt; is the opposite, where we lock as little code as possible. The upside is that we don't have to worry about multiple threads blocking for long on small bits of code. The downside is that acquiring more locks means more lock overhead, and more programmer effort to implement all the locks consistently and safely. Fine granularity can also lead to &lt;b&gt;deadlock&lt;/b&gt;, where multiple threads are stuck waiting for locks that other threads own.&lt;br /&gt;&lt;br /&gt;The Python interpreter, as an example, implements a single &lt;b&gt;Global Interpreter Lock&lt;/b&gt;, which is a lock to govern the entire Python interpreter. Only one operating system thread can be running the interpreter at once, to prevent corruption of global data. I think new versions of Ruby do this too.&lt;br /&gt;&lt;br /&gt;There are other methods of synchronizing access to shared resources. One method is to make all data immutable; If you can't modify data, you can't corrupt it. Since Parrot's strings are immutable, you shouldn't ever need a lock when working with them. You may still need to worry about playing with a mutable container PMC which holds strings, or the mutable registers which point to strings, however.&lt;br /&gt;&lt;br /&gt;Parrot is definitely going to want to make use of OS-supplied locks in some fashion. Maybe we want to make a PMC wrapper around system lock primitives, or we want to create some kind of lock manager that uses a single system mutex to distribute a series of immutable tokens to worker threads. The exact details of locking are certainly up for debate, but the fact that we don't want to brew our own should be obvious.&lt;br /&gt;&lt;br /&gt;Since locks need to be used consistently for them to be of use at all strongly hints at the fact that Parrot should probably do the locking internally. We probably don't want to apply locks to every single operation, since the common case programs are single-threaded applications and we don't want to apply the performance penalty of lock overhead to programs which don't need it. If Parrot can identify only those PMCs which are shared, it can apply locks selectively to those PMCs only, limiting overhead to only the places where it is necessary. For instance, if we add a synchronize op:&lt;br /&gt;&lt;pre&gt;$P0 = syncronize $P1&lt;br /&gt;&lt;/pre&gt;We can create some kind of wrapper PMC type whose vtables enter a lock, call the vtable of the synchronized PMC, and then release the lock. In this example, if everybody used $P0 when they wanted to modify $P1, all operations would be safe. The onus would be on the programmer to explicitly mark the PMC as synchronized, of course, and many programmers will probably forget to do that.&lt;br /&gt;&lt;br /&gt;Maybe instead of passing PMC references between threads directly we create and pass clones and modify them separately on different threads. Then, when we want our changes from one thread appear in another thread, we would call some kind of propagate op:&lt;br /&gt;&lt;pre&gt;propagate thread, obj&lt;br /&gt;&lt;/pre&gt;This would pause the specified thread, update the object contents, and then unpause the thread. This would be very similar to the message passing that languages like Erlang use (not exactly the same, because Erlang wouldn't pause the recipient for this, but you get the idea).&lt;br /&gt;&lt;br /&gt;Maybe we have a system where we only share read-only copies. So thread A would own the PMC, but thread B could get a read-only copy of it. This would completely obviate the need to lock the PMC since only one thread could write to it, but then we need some kind of mechanism where thread B could make modifications back if necessary, or maybe B could gain the writable copy and make A's copy read-only. This system could get very complicated very quickly, however.&lt;br /&gt;&lt;br /&gt;We could also avoid most locks if we used a transactional memory system to avoid memory corruption, but that could still add overhead to the single-threaded common case and then we would still want a system of locks for other operations that don't require locking a PMC.&lt;br /&gt;&lt;br /&gt;These are only a handful of the many potential options that Parrot has ahead of it, and I can go into greater detail about any of them that people are interested in thinking about. I think Parrot is going to want at least some kind of locking mechanism, so we could start prototyping those things immediately if we wanted. How these mechanisms get implemented and applied within Parrot is obviously the bigger issue that we can ignore for now.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-8448603728564444292?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/8448603728564444292/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/06/parrottheory-locks-and-synchonization.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/8448603728564444292'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/8448603728564444292'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/06/parrottheory-locks-and-synchonization.html' title='ParrotTheory: Locks and Synchonization'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-8273805344092505115</id><published>2010-06-08T17:00:00.001-04:00</published><updated>2010-06-08T17:00:02.901-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='lost'/><title type='text'>Lost Finale: Redux</title><content type='html'>I wasn't planning to write any more about the disappointing Lost series finale, but I received too many comments on &lt;a href="http://wknight8111.blogspot.com/2010/05/lost-finale.html"&gt;my last post&lt;/a&gt; to ignore. I started writing a reply and, as happens so frequently, my reply grew to the size of a full blog post. So I'm posting it here.&lt;br /&gt;&lt;br /&gt;Wednesday evenings were a time that I kept blocked off for Lost. Watching that show is what I did on those evenings and now my schedule has a gaping 1-hour hole. Looking back on the past few years of television, I don't feel the kind of fondness I should. Instead, I feel an utter disappointment and emptiness because of the way that damn series ended. 6 years of loyal TV watching is now nothing more than wasted time and bad memories. For 6 years I would tell myself "Sure things are crazy now, but there are more episodes to come and if I follow this through to the end everything else will make sense". Well, we made it to the end. I still have that feeling like the show is a crazy mishmash of weird ideas and disjointed storytelling, but there aren't any more episodes to look forward to for these things to be resolved.&lt;br /&gt;&lt;br /&gt;I sincerely can't tell if the writers were simply being lazy and disconnected, tired of the grind for 6 years and already focused on their next big projects. I can't tell if maybe the writers are just bad at writing, not able to resolve, interpret, or make sense of the complex predicaments and scenarios they created. Every time the situation became too complex, every time the scenario became too crazy and unmanageable, they simply abandoned ship and moved on to the next mystery. Night after night, season after season they wrote themselves into holes that they either had no intentions or no capability to write themselves out of. They introduced new characters and new subplots with no explanation, no development, and no lasting effect on the overarching plot. In fact, I can't say with certainty that there was any single overarching plot to the whole show. They used throwaways to distract the audience from the real plot, and they abandoned so many story elements without so much as a passing apology.&lt;br /&gt;&lt;br /&gt;Who would you say was the main antagonist of the show? For 5 seasons, the smoke monster was described as being a protective force, or a "security system" for the island, not it's main antagonist until the final season. Maybe you could say Widmore was the antagonist, but he had a tangential effect at best on most of the characters throughout, and later was revealed to be acting on Jacob's behalf (right before he died). Was Ben the antagonist? He certainly appeared so for about 2 seasons, though once we started seeing into his past and his actions we realize that he's a force for the good of the island, even if his means-justify-the-end &lt;i&gt;modus operandi&lt;/i&gt; is a little questionable. And who would be the main protagonist? Jack would appear so, though for the first few seasons he butted heads against Locke and frequently was overshadowed by him. Also, his "go with the flow" attitude in the later seasons is hardly reconcilable with the notion of the series' primary hero. Maybe we could say Locke was the protagonist, but he died halfway through the series.&lt;br /&gt;&lt;br /&gt;Let's look at some other characters too. Sawyer was one of the few people to actually escape from the island alive, though he is still the same angry, vengeful conman he was at the beginning of the series. He had some liberating moments early on when Kate learned about his backstory and when he was finally able to kill the man who caused the death of his parents. He even had some moments of heroism after his transformative time in the Dharma Initiative, though the death of Juliet appeared to drive him back to his old self in Season 6. Kate was an interesting and likable character, but we can't really say that she transformed as a character. The role she mostly appeared to play was a catalyst for Jack and Sawyer, and a suspense-building element in their weird love triangle. Sayid's character was interesting because he was constantly being pulled across the line between good and evil, torn between needing to do whatever was necessary, and needing to be a good person. Jin and Sun were basically non-participants, spending the entire show involved primarily with each other. In the beginning their characters were defined by their language barrier and weird marital problems.&amp;nbsp; As the show progressed they found true love together, but then they died leaving behind little more than an orphaned child and motivation for Kate to catalyze Jack to kill the Man in Black. Desmond was also an interesting character, though he didn't really make any appearance in Season 1, and was missing entirely in Season 5 and the first half of Season 6. In the end, Desmond was used as little more than a tool to help Jack.&lt;br /&gt;&lt;br /&gt;It's hard to say that Lost really had any central antagonists, protagonists, or even a single overarching plot. It ended up being a mishmash of unresolved subplots which just happened to feature a common set of characters. Sometimes they seemed to do good, sometimes they seemed to do evil, but the results of their actions were never shown and in the end almost all of them simply died or were forgotten.&lt;br /&gt;&lt;br /&gt;In the end it's really all about the audience, and nearly two weeks later the audience that I have talked to is still unhappy. It's worse than the infamous ending to the Sopranos where the screen simply cut to black. That was remarkably fulfilling for most audience members, but at least some meaning could be drawn from it: Tony Soprano was continuing his life as usual, was still an unremorseful gangster, and after everything that happened to himself and his family and his friends we was still carrying on as he had always carried on.&amp;nbsp; Any time that the series ended would have been the same, you could just as easily cut out an episode prior or an episode later with no effect. After all, we weren't seeing and weren't expecting to see the complete demise of organized crime, or even the salvation of one of it's prominent figures. The Sopranos was simply a short window into that life, nothing more and nothing less.&lt;br /&gt;&lt;br /&gt;The Lost season finale offered no such message. You can't say that things would continue as they always had. You can't say that the troubled characters were changed for the better. You can't say that anything of importance happened at all.&lt;br /&gt;&lt;br /&gt;That's one of my biggest problems with that show. Looking back on it, it's hard to say with certainty that anything happened. The net result, that some people crashed on an island and only a handful of people survived (though, notably, not all of the final survivors were on the original crashed plane) is a pretty boring story. Look back at some of the big subplots: Babies and expecting mothers dieing? Not addressed at all. Going back in time to change the future? Didn't happen. Lessons about "what happened, happened" and "dead is dead"? Casually dismissed without any explanation. Greater lessons about good and evil? Mired in too much ambiguity and mystery to draw any meaningful conclusions. Greater lessons about fate and free will? Absolutely ignored.&lt;br /&gt;&lt;br /&gt;We could maybe say that the whole island was some crazy nonsense, but the important part were the characters and their development. That would be fine if the whole show was about a group of troubled people who were put in an absurd situation and learned important lessons and ultimately grew to transcend their own problems and limitations. Or maybe the show could have focused on the relativity of "good" and "evil" and show how a lot of people ended up senselessly dieing to protect ideals (especially unverified oral traditions) that ended up being false or irrelevant. That would have been extremely deep, and would have made for a satisfying conclusion, but it was not so. It might also have been nice to compare and contrast the killings of the Temple people by the Man in Black with the killings of the Dharma initiative by the followers of Jacob. Trust me, there is tons of unmined gold in this show that, instead of being polished and crafted, was ignored in favor of giving the audience a huge piece of coal.&lt;br /&gt;&lt;br /&gt;And it really is all about the audience. My other biggest problem with this finale was the complete lack of regard and respect for the audience on the part of the writers. This is a show which thought nothing of naming characters after famous philosophers, theologians, and scientists. This is a show which was filled with obscure cultural and literary references. This is a show which used Latin writing and Egyptian hieroglyphs for the fans to painstakingly translate and interpret. This was a show that appeared to have been targeted towards an intelligent and educated audience. Why then were we treated like dogs; so easily distractable by the next shiney object, loud noise, or scrap of food from the table? Why, in the presence of so many fantastic and occasional absurd scenarios did the writers simply choose to ignore the story we were following and distract us with the next mystery, the next suspense-building cliffhanger? Were we, the audience, just the observers in the Pearl station, tasked with observing something we did not understand, not realizing that &lt;i&gt;we were the experiment&lt;/i&gt;, and &lt;i&gt;we were being observed&lt;/i&gt;? What is the lesson here, that if you pretend to treat your audience as if they were intelligent, that you can treat them just like every other stupid audience, make a boatload more money, and call it revolutionary TV?&lt;br /&gt;&lt;br /&gt;The lesson I learned, ultimately, is that TV is a huge load of shit. Lost over-promised and severely under-delivered, and I've been burned by that. I do like to watch Burn Notice, when I can find it on at a reasonable time. I do like to watch NCIS too, sometimes. But, I absolutely refuse to get invested in any new show the way I invested myself into Lost.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-8273805344092505115?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/8273805344092505115/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/06/lost-finale-redux.html#comment-form' title='1 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/8273805344092505115'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/8273805344092505115'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/06/lost-finale-redux.html' title='Lost Finale: Redux'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>1</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-3016104020802019622</id><published>2010-06-06T19:33:00.001-04:00</published><updated>2010-06-06T19:35:55.303-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='MediaWiki'/><category scheme='http://www.blogger.com/atom/ns#' term='EmbedVideo'/><title type='text'>EmbedVideo Extension Pre-Release</title><content type='html'>Last Tuesday I got confirmation that I now had a commit bit to the MediaWiki subversion repository. I don't have access to the whole thing, I only needed access to the extensions section. Today&lt;a href="http://www.mediawiki.org/wiki/Special:Code/MediaWiki/67470"&gt; I checked in a snapshot of the EmbedVideo extension&lt;/a&gt; &lt;a href="http://github.com/Whiteknight/mediawiki-embedvideo"&gt;I've been working on&lt;/a&gt;, in preparation for the first release of the extension under new management.&lt;br /&gt;&lt;br /&gt;Since &lt;a href="http://wknight8111.blogspot.com/2010/05/embedvideo-mediawiki-extension.html"&gt;I started working on the extension&lt;/a&gt; I've made a number of changes, most of them internal. Here's a short list of things I've done:&lt;br /&gt;&lt;br /&gt;&lt;ol&gt;&lt;li&gt;Broken the extension up into multiple files and created a proper directory structure, following current best practices&lt;/li&gt;&lt;li&gt;Added i18n support, though I've only added English-language messages so far.&lt;/li&gt;&lt;li&gt;Added support for the website teachertube.com.&lt;/li&gt;&lt;li&gt;Added the ability to float the video left or right, and add a caption&lt;/li&gt;&lt;li&gt;Added an {{#evp:}} tag, to be fully compatible with the EmbedVideoPlus extension syntax.&lt;/li&gt;&lt;/ol&gt;The biggest changes have been the internal cleanup tasks. The code base has been almost entirely rewritten from the ground-up to follow best practices, although there is still a little bit more cleanup work to do.&lt;br /&gt;&lt;br /&gt;I'm planning a formal release of the extension to happen sometime within the next two or three weeks. I will be trying to test the release against a small handful of releases: Probably 1.12, 1.13, 1.14, 1.15, and the betas of 1.16. I'm not sure how I want to handle releases in the future, since I do plan to add new features and support for new sites, but I don't want to spend huge amounts of effort backporting every feature I add to all sorts of old MediaWiki versions. I will probably try to tag revisions for old versions, but will not worry about backporting new features myself. If somebody wants to do the hard work of backporting features, they can submit patches (or ask for a commit bit!) and do it themselves.&lt;br /&gt;&lt;br /&gt;I'm not planning to add any major new features before I release in the next few weeks, so from here on out it's all testing and &lt;a href="http://wiki.github.com/Whiteknight/mediawiki-embedvideo/"&gt;documenting&lt;/a&gt;. As soon as it's tested to my liking, I'll put out a new release.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-3016104020802019622?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/3016104020802019622/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/06/embedvideo-extension-pre-release.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/3016104020802019622'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/3016104020802019622'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/06/embedvideo-extension-pre-release.html' title='EmbedVideo Extension Pre-Release'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-1873078366056520426</id><published>2010-06-02T09:30:00.000-04:00</published><updated>2010-06-02T09:30:17.205-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='Parrot'/><category scheme='http://www.blogger.com/atom/ns#' term='ParrotTheory'/><title type='text'>ParrotTheory: Green Threads</title><content type='html'>This summer I'm mentoring GSoC student Chandon in his project to bring Hybrid Threads to Parrot. Today I'm going to talk a little bit about that, and maybe give some information about what Parrot needs to do to properly support a full and robust threading system in the long term. I've &lt;a href="http://wknight8111.blogspot.com/2010/02/parrottheory-threading.html"&gt;written about threads previously on this blog&lt;/a&gt;, so I won't cover all the nitty-gritty details again.&lt;br /&gt;&lt;br /&gt;At the risk of over simplicity there are basically two types of threads to consider: &lt;a href="http://green%20threads/"&gt;OS Threads&lt;/a&gt; and &lt;a href="http://en.wikipedia.org/wiki/Green_threads"&gt;Green Threads&lt;/a&gt;. OS Threads or "native threads" are what most people are probably familiar with: These are the threads managed by your operating system kernel. Native threads represent a flow of execution of native machine code on a single processor core, and multiple native threads can run simultaneously on multiple processor cores. On a single core, they run sequentially but in time slices so short it appears to the human observer that things are happening at the same time. This appearance of simultaneity is extremely important for computer users, especially for graphical interfaces, because people don't like waiting and don't like it when the computer "freezes" or "locks up" during periods of heavy computation.&lt;br /&gt;&lt;br /&gt;A native thread represents a system context: A separate call stack and a snapshot of the processor state at any given time. By saving the processor state and call stack away, we can switch to a different task and later resume our original task as if nothing has happened.&lt;br /&gt;&lt;br /&gt;I don't know where they were originated, although Green Threads were probably most famously implemented in early versions of the &lt;a href="http://en.wikipedia.org/wiki/JVM"&gt;JVM&lt;/a&gt;.&amp;nbsp; Instead of being managed by the OS, they are managed by the VM. Green threads are scheduled onto OS threads in the same way that OS threads are scheduled onto processor hardware. The VM saves it's current state into a VM context object, moves to a different task, and then reinvokes that context object to resume the original task.&lt;br /&gt;&lt;br /&gt;Green threads are very interesting tools, when they are available. They tend to be extremely inexpensive compared to native OS threads. They are managed by the VM, and if they all execute sequentially on a single OS thread the risk of lock contention and data corruption drops almost to zero. The down-side to green threads is that they are all multiplexed onto a single OS thread, so they each get a smaller share of the processor's time.&lt;br /&gt;&lt;br /&gt;To help enforce that green threads in the VM may only execute on a single OS thread at a time, the VM may implement a global interpreter lock (GIL). Famous examples of this occur in Python and Ruby. A GIL helps with integration to C libraries which are not thread-safe, and also helps to prevent things like callback functions from executing in a different OS thread and causing unprotected data corruption. The downside to the GIL approach is that&amp;nbsp; extra overhead must be spent to perform locking/unlocking and checking of the GIL.&lt;br /&gt;&lt;br /&gt;Chandon's Hybrid Threads idea tries to combine the two models of OS and Green threads. Parrot will maintain a pool of OS "worker" threads, based on user settings and the number of available processor cores. Parrot will then be able to schedule a queue of green threads onto the available OS threads. Yesterday on his blog he actually posted an example of using Parrot's existing continuations to form a system of cooperative green threads. Instead of happening at any time, green thread switches in his example are manually triggered by calling the functon th_resched(). If we move the call to th_resched into the interpreter's scheduler object so it can be called when the scheduler runs, we gain a proper preemptive green threads system without too much effort. &lt;br /&gt;&lt;br /&gt;There are some difficulties with this model, especially since Parrot currently has no mechanism to manage data contention across OS threads, and also considering that each OS thread ill have it's own interpreter object, which will need to be created at some runtime expense. Parrot is also going to need to add new mechanisms and policies for dealing with data which is considered "global": namespaces and class definition metaobjects, etc.&lt;br /&gt;&lt;br /&gt;We're also going to need to start answering some very difficult questions: If a continuation created in interpreter A is passed to interpreter B and invoked, what do we do? If an exception is thrown by interpreter A and is caught by a handler defined in interpreter B, what do we do? If an object's vtable override is preempted by a green thread switch and the object is in an inconsistent state when it is attempted to be accessed in a different green thread, what do we do? How do we even detect when any of these things happen? What mechanism are we going to need to write to allow sharing PMCs between OS threads? Do we use &lt;a href="http://en.wikipedia.org/wiki/Software_transactional_memory"&gt;STM&lt;/a&gt; again? If so, do we write STM ourselves or try to use a pre-existing and proven library? Considering that we now have immutable strings (which don't need to be locked because they can't be written), do we maybe want to consider a system where shared PMCs are either read-only in other threads or maybe are &lt;a href="http://en.wikipedia.org/wiki/Copy-on-write"&gt;COW&lt;/a&gt; to prevent corruption? How do we deal with long-running operations, such as blocking IO and calls to long-running library functions?&lt;br /&gt;&lt;br /&gt;For Chandon's project to be a success I think we need just two things: a robust green threads system, and a prototype OS threads system with a scheduler that can intelligently assign green threads to OS threads. Once we have that system in place, along with some rules or algorithms against sharing writable data until we sort out the details for how that becomes possible, we can add new mechanisms piece-wise to enable new features and functionality. If he has time sure he can do more, but if we can get the basics in place we will be in a much better state than we were before he started.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-1873078366056520426?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/1873078366056520426/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/06/parrottheory-green-threads.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/1873078366056520426'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/1873078366056520426'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/06/parrottheory-green-threads.html' title='ParrotTheory: Green Threads'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-4832539926026353662</id><published>2010-05-29T07:50:00.000-04:00</published><updated>2010-05-29T07:50:48.720-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='JIT'/><category scheme='http://www.blogger.com/atom/ns#' term='Parrot'/><category scheme='http://www.blogger.com/atom/ns#' term='Optimization'/><title type='text'>The AOT Advantage</title><content type='html'>In my previous blog post I briefly mentioned PHP, &lt;a href="http://github.com/facebook/hiphop-php"&gt;HipHop&lt;/a&gt;, and AOT. For readers coming late to the party, HipHop is a native code compilation system for PHP, created by Facebook. The idea is that compiling down a relatively large subset of PHP (basically everything besides eval) to C++ code and passing it to an optimizing compiler can produce a much more efficient program than interpretation alone or even compiling to bytecode in advance and executing that.&lt;br /&gt;&lt;br /&gt;Let's look at this same idea, but in relation to Parrot's intended LLVM backend system.&lt;br /&gt;&lt;br /&gt;We can start off with a basic PHP compiler. The compiler doesn't need to be particularly fast since compilation happens once in advance. Sure, we don't want compilation to take forever because that slows down development, but it doesn't have the same kind of speed mandate we would have in an interpreted environment where we compile the script every single time we run it.&lt;br /&gt;&lt;br /&gt;Our basic, feature-light compiler compiles the PHP code down to PAST. From there we kick in a series of optimization stages. Again, if we know we're doing ahead-of-time compilation, we can add more optimization stages up front and take a little more time to compile. For a release build of a high-throughput application, maybe we really want to swing for the fences with this one and enable every optimization in the book. Let's hope it's a long, well-written book.&lt;br /&gt;&lt;br /&gt;We compile the PAST down to Parrot bytecode, after all the optimizations have run their course. At this point, we hand off the bytecode to the LLVM backend. LLVM converts the Parrot bytecode into LLVM IR, and passes that through it's own impressive suite of optimizations. At the end, LLVM spits out a native executable which is screaming fast.&lt;br /&gt;&lt;br /&gt;The converse side is a case where we want to shorten the turnaround time, such as when we are doing direct interpretation, or even making executables for debugging purposes and don't want our developers sitting around for a long time &lt;a href="http://xkcd.com/303/"&gt;waiting for the thing to compile&lt;/a&gt;. We don't want to completely avoid optimizations, because after compilation the developers have huge and expensive test suites to run which can take much much longer.&lt;br /&gt;&lt;br /&gt;What we do in this case is compile the code to PAST, maybe run a handful of "good bang for the buck" (GBFTB) optimizations like constant folding, and compile the PAST down to bytecode. We'll avoid the last step and just execute the bytecode directly, since this is a debug build. When we run it, the JIT engine will kick in to do basic runtime compilation of detected hot spots, giving us nice speedups for programs with tight loops. The JIT compiler will likewise use a few of the GTFTB optimizations, but won't go crazy; we're doing this at runtime now and we have a user waiting.&lt;br /&gt;&lt;br /&gt;There are some people in the world of scripting languages who view the complete lack of native executable support as a virtue. After all, they say, the nature of these languages is fast turnaround; We can make changes and run them directly without needing to invoke a separate compiler to produce a separate executable. Plus, the immediate availability of human-readable source code furthers the common goals of open source software and knowledge sharing.&lt;br /&gt;&lt;br /&gt;Fast turnaround, from script to execution, certainly is a virtue--in development. Once we're ready to cut that release however, things change pretty dramatically. At my job, I write some applications in C#. During development, I keep things in "Debug" mode, and do lots of tests very quickly. When it's time to cut that release I change the VisualStudio configuration to "Release" and then click "Build All". Where I was doing incremental debug builds, now I'm doing a full release build with optimizations turned on and debugging symbols stripped out. I also build a separate installer project, and to test everything I need to install the software first and run it from it's installed location. This is a lot of extra cost, but I don't do it frequently. I go from about a 10 second compile time turnaround to almost 10 minutes worth of compilation and installation work. The net result is a faster, better executable for my end users.&lt;br /&gt;&lt;br /&gt;In the case of Facebook, using HipHop to compile PHP down to native machine code improves throughput performance by 30%, which translates into lower server load, lower wait times for users and, at the end of the day, real money. Say what you want to about obfuscation of the source code, 30% is nothing to ignore.&lt;br /&gt;&lt;br /&gt;I like to think that the Parrot infrastructure will support any workflow that our end users want. Having speedy interpretation around for people who want that is a very good thing, and we do it already. But, having the ability to employ tiered libraries of optimizations and create native machine binaries for fast execution is a very very good thing as well. This is why I keep pushing for our eventual JIT system to also support AOT as well. LLVM will already handle all the nitty-gritty, so the amount of scaffolding we'll have to provide in Parrot is minimal in comparison.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-4832539926026353662?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/4832539926026353662/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/05/aot-advantage.html#comment-form' title='2 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/4832539926026353662'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/4832539926026353662'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/05/aot-advantage.html' title='The AOT Advantage'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>2</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-3914014113871240240</id><published>2010-05-28T17:56:00.000-04:00</published><updated>2010-05-28T17:56:29.793-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='Parrot'/><category scheme='http://www.blogger.com/atom/ns#' term='Optimization'/><title type='text'>Compilers Targetting Parrot</title><content type='html'>I got to thinking today about Parrot and came to an interesting conclusion: Writing our own parsers for various existing languages is nice, but if we could get the language communities involved it would be even better. There's a symbiotic relationship that could form from such an arrangement and I would like to talk about that here for a little bit.&lt;br /&gt;&lt;br /&gt;Let's take PHP as an interesting example. The PHP community has been talking about some kind of rewrite of their core engine to support Unicode from the ground-up. I'm not really involved in those discussions, so I don't know what the current state of the discussion is. I couldn't even tell you if work has started on such a rewrite. However, I do know one thing: If the next version of the PHP compiler targeted Parrot as the backend, it would have built-in Unicode support for free.&lt;br /&gt;&lt;br /&gt;There are some reasons why PHP people might not like this option, considering PHP's niche in the web server world and Parrot's current issues with performance and memory consumption. But, as I'll describe here, those are relatively small issues.&lt;br /&gt;&lt;br /&gt;Pulling an extremely wrong number out of a hat, let's say that it takes 10,000 man-hours to construct a modern, though bare-bones, dynamic language interpreter from the ground-up. You would have some features like GC, but wouldn't have others like JIT. Unicode support but a small or even non-existant library of built-in optimizations. You build a compiler, you build the interpreter runtime, and then you build a language library runtime so the programs you write don't have to reinvent every wheel from scratch. It's a daunting task, but it's very doable and has been done many times before.&lt;br /&gt;&lt;br /&gt;Now, let's say you take that same developer pool of 10,000 man-hours and instead build a compiler on top of Parrot instead of writing a new VM from the ground up. You do get a whole bunch of the really tricky features immediately: GC, Exceptions, IO, JIT (to come), Threads (to come), Unicode, Continuations, Closures, a library of Optimizations (to come), and all sorts of other goodies. You also get a nicer environment to write your parser in than bare lex/yacc (though, I think, you could still use lex/yacc if you really wanted). You get some libraries already, though you would likely need to write wrappers at least to get things into the proper semantics of your language. You really do get a hell of a lot and you could easily have a working compiler, at least the basics of one, up and running within a few weeks.&lt;br /&gt;&lt;br /&gt;Let's even be conservative and say that your team can't build the compiler front-end in less than 1,000 man-hours on Parrot.  Even then, you still have 9,000 man-hours left. You could spend some of that making a gigantic runtime library, but I like to think you could devote some of that effort back to Parrot too. If you spent even 5,000 hours making Parrot faster, or adding new features, or whatever you need, you'll still be spending less effort overall and getting a comparable result, if not a better one.&lt;br /&gt;&lt;br /&gt;Keep in mind, on Parrot, you're going to get all the features that other people add, in addition to the ones you added yourself. Parrot really is a symbiosis: You hack Parrot to add what you need, and in turn Parrot gives you all sorts of other things for free.&lt;br /&gt;&lt;br /&gt;Parrot has plenty of stuff around to get you started, even if it's not perfect right now. Remember that Parrot is extremely actively developed. You can do your development now, build your compiler and get your language ready and grow your regression test suite. When you decide that you need things improved, you can make targetted improvements incrementally. If you want a better GC, for instance, you can go in and just add that part. But, you have the benefit of knowing that Parrot already has one and your test suite team doesn't have to wait for your GC team to finish writing their first GC. That's good, because those damned GCs can take a while.&lt;br /&gt;&lt;br /&gt;My point, in a nutshell, is this: Parrot has some really cool features, but the coolest is that it provides a standard base level, a standard toolbox, that you and your compiler team can take and use immediately if you want it. You can get started right now with the kinds of high-powered features and tools that it would take you years to get right by yourselves. Yes there are improvements needed and some features yet to be desired, but adding them to Parrot represents smaller, more incremental improvements than trying to write an entire interpreter yourself.&lt;br /&gt;&lt;br /&gt;Parrot is adding some really cool features this year too. I fully expect by the 3.0 release we will these new things:&lt;br /&gt;&lt;ol&gt;&lt;li&gt;A new, fully-featured robust threading system&lt;/li&gt;&lt;li&gt;A pluggable optimization library, with optimizations able to be written (likely with some bootstrapping) in the high-level language of your choice&lt;/li&gt;&lt;li&gt;A new GC with radically improved performance characteristics&lt;/li&gt;&lt;li&gt;An extremely powerful instrumentation and analysis framework (again, with tools able to be written in the language of your choice, possibly with some bootstrapping)&lt;/li&gt;&lt;li&gt;A flexible native function call interface for interacting with existing libraries&lt;/li&gt;&lt;/ol&gt;If you can wait a little later, I fully expect by 3.6 or 3.9 that we will be rolling out a powerful LLVM-based native code compilation core which will support JIT and (I hope!) AOT. You don't have to look much further than Facebook's HipHop project to see that compilation of dynamic languages down to native executables will be a big benefit, especially if we can aggressively optimize all the way down. I'll talk about that particular issue in a later post.&lt;br /&gt;&lt;br /&gt;Parrot does have some problems now, like performance. I can certainly understand the criticism on that point. However, for a fraction of the effort it will take you to write a new interpreter or virtual machine from the ground up you can target Parrot instead, get a bunch of free features, and maybe spend some of your free time helping us add the optimizations and improvements that you need. I really and truly believe that it's a win-win proposition for everybody involved.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-3914014113871240240?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/3914014113871240240/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/05/compilers-targetting-parrot.html#comment-form' title='7 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/3914014113871240240'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/3914014113871240240'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/05/compilers-targetting-parrot.html' title='Compilers Targetting Parrot'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>7</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-3754457734977407588</id><published>2010-05-24T20:11:00.000-04:00</published><updated>2010-05-24T20:11:45.367-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='Personal'/><category scheme='http://www.blogger.com/atom/ns#' term='lost'/><title type='text'>Lost Finale</title><content type='html'>I know it's not my normal topic of conversation, but the day after the big finale I'll damned if I don't write at least something about Lost. For 6 years now that show has been my absolute favorite hour of TV each week, and for some time it's been the only hour of TV that I will watch on a regular basis. Sure, there are a few other shows I will watch if they are on when I am looking, but Lost was the only show that I would arrange my schedule around.&lt;br /&gt;&lt;br /&gt;Last night was the big, long-awaited finale and it completely soured the rest of the series. I don't know if I will ever be able to watch any of it again, knowing now that the whole thing was some huge farce. My questions were not answered, as was promised to me in the advertising blitz this year, and I was left completely unrewarded for all the time and effort I spent following along with this story.&lt;br /&gt;&lt;br /&gt;And I followed. I watched almost every episode when it aired. I read lostpedia cover-to-cover. I talked with other fans on the internet and tracked down clues and easter eggs in the various guerrilla media campaigns that ABC ran. I did it all.&lt;br /&gt;&lt;br /&gt;The rest of this blog post contains spoilers. Normally I don't like to be the one to spoil the surprise for other people, but I feel that the show spoiled itself last night. There's nothing left for me to ruin.&lt;br /&gt;&lt;blockquote&gt;&lt;br /&gt;In the normal "island" timeline Jack, newly crowed as protector of the island and his team meet up with Locke, Linus, and Desmond on their way to the glowing source in the center of the island. Desmond lowers into the hole, uncorks the source, and the island starts to sink. Without the power of the light, Locke is human again and is killed by Jack and Kate. Jack enters the source himself to re-cork the island and bring the light back, but dies in the process. Hurley becomes the new protector, with Linus as his assistant. Everybody else (read: like 5 people) make it off the island in the Ajira plane.&lt;/blockquote&gt;&lt;br /&gt;This part is fine. Maybe it's a little unimaginative, but it's fine. It works.&lt;br /&gt;&lt;blockquote&gt;&lt;br /&gt;In the "sideways" alternate timeline, All the stars are meeting up with each other and having flashbacks about their time on the island. They all go to a concert, where Clair has the baby. Jack shows up late, meets Kate, and they go to the funeral home to attend the funeral of Jack's father. But lo! Jack's father is up and walking around. They have an emotional conversation, Jack realizes that they are all already dead. Christian says something about leaving but won't say to where, there is a white light, and the series ends.&lt;/blockquote&gt;&lt;br /&gt;That's right: in the big "flash sideways" timeline, everybody is dead and it's apparently some sort of purgatory. Purgatory, you know, that thing that the writers said early in the series &lt;a href="http://www.usatoday.com/life/television/news/2006-05-09-lost_x.htm"&gt;that it definitely wouldn't be&lt;/a&gt;. Everything in the flash-sideways timeline was basically a dream, which makes this one of the largest and most-watched usages of the "&lt;a href="http://en.wikipedia.org/wiki/Dream_world_%28plot_device%29"&gt;...and then they woke up&lt;/a&gt;" cop-out gimmick ever. According to &lt;a href="http://en.wikipedia.org/wiki/Dream_sequence"&gt;Wikipedia&lt;/a&gt;:&lt;br /&gt;&lt;br /&gt;&lt;blockquote&gt;Similar to a dream sequence is a plot device in which an entire story  has been revealed to be a dream. As opposed to a segment of an otherwise  real scenario, in these cases it is revealed that everything depicted  was unreal. Often times this is used to explain away inexplicable  events. Because it has been done, in many occasions, to resolve a  storyline that seemed out of place or unexpected, it is often considered  weak storytelling; and further, in-jokes are often made in writing (particularly  television scripts) that refer to the disappointment a viewer might  feel in finding out everything they've watched was a dream.&amp;nbsp; &lt;/blockquote&gt;&lt;br /&gt;If I learned nothing new about the show last night it was this: The writers were lazy and got themselves into more trouble than they could handle. They created a situation so complex and unbelievable that they couldn't deal with it, so they wrote it all away as a figment of imagination. It's not that they didn't have the time or the advanced notice they would have needed to wrap things up cleanly, they simply chose the lazy, easy way out of a story that they allowed to get out of control. &lt;br /&gt;&lt;br /&gt;The vast majorty of the finale was actually very good. We see Jack solving the mystery of how to kill the smoke monster, and then sacrificing himself for the sake of the island and ultimately the whole world. Hurley becomes the new protector of the island, Linus finds redemption for his transgressions, and a handful of people from the island were able to leave for good. Everything was going fine until the last 10 minutes. At 11:20 last night they could have kept things going, ended on a strong note, and enshrined it as one of the best possible finales for one of television's greatest shows. But, they didn't.&lt;br /&gt;&lt;br /&gt;Let's give one possible alternate ending that I think would have been far better:&lt;br /&gt;&lt;br /&gt;&lt;blockquote&gt;Desmond uncorks the island and dies. Jack and Locke fight each other to the death. Locke dies, Jack is mortally wounded. The plane with the rest of the survivors takes off. Jack closes his eye as the island sinks below the water. That universe blinks out of existance, with the "sideways" timeline becoming the new main timeline. All the lost survivors have their memories of their time on the island and, through their bonds of comradery and new life experiences they learn to start resolving some of their personal problems to become better people.&lt;/blockquote&gt;&lt;br /&gt;That ending would have been much better in my opinion because it isn't a cop-out and we see the character development actually reach a conclusion for many characters. Another possible ending:&lt;br /&gt;&lt;br /&gt;&lt;blockquote&gt;Desmond uncorks the island. Jack and Locke fight, Locke dies (Jack is wounded).&amp;nbsp; Jack enters the cave to re-cork the island, and when the light comes back, Jack is transformed into the new smoke monster (though, he is obviously a benevolent one). Hurley becomes the island protector. We see Jack and Hurley at some time in the future, fast friends, vowing together to protect the island with their newfound powers. Hurley allows all the remaining people from the island to leave, the island flashes and disappears. In the sideways timeline, all the survivors have their memories of the time on the island, get over many of their personal problems, and form a pact together to help protect what the island stands for.&lt;/blockquote&gt;&lt;br /&gt;A little touchy-feely and feel-good for my taste, but still far superior to the "real" ending we saw. At least we would have known that this ending was real and we weren't just watching some hastily-written dream sequence. Here's something even better:&lt;br /&gt;&lt;blockquote&gt;&lt;br /&gt;It's the end of season 5. Juliet is in the well. She hits the nuke with the stone, the screen goes white. We see Oceanic 815 land in LAX as planned, the survivors get off the plane. While they aren't explicitly aware, they have been subtly changed because of the experiences on the island, and every one of them goes on to live long and fulfilling lives.&lt;/blockquote&gt;&lt;br /&gt;This ending cuts off a whole damn season, and yet would have been much more rewarding and would have left fewer issues unresolved. We wouldn't have gotten explanations for any of the weird things that happened on the island, but that wouldn't really have been so important. We could chalk it all up to the island being weird and magical, and now that the island is sunk under water and everybody is home safe, we can basically forget all about it. Maybe, we could even leave it ambiguous as to whether the events on the island even happened at all, or whether it was some sort of collective mass hallucination that changed the lives of all the people on the plane for the better. I would have been much happier with that ending, personally, even though we would have lost a whole year of the show. Hell, we could have pushed that ending back a year, and spent all of season 5 really digging into some of the mysteries of the island, the role of Jacob, the workings of the Dharma initiative, etc. That would have been cool too. And, it would have been much better than what we got.&lt;br /&gt;&lt;br /&gt;And while I am rambling, I was promised that all my questions would be answered, but in reality very few of them were. If anything, too many problems and dangling plot threads were left completely unaddressed. I would have been completely willing to ignore some of these issues if I thought we were hurtling towards a cataclysmic ending that would shock and awe us all into a happy stupor, but with the main plotline being written off as a work of bad creative fiction, the other ommisions becomes more striking and infuriating. Some of the more pressing points I would like to enumerate here:&lt;br /&gt;&lt;br /&gt;What was the problem with pregnant women and babies on the island? This is a pretty serious question, considering almost two full seasons were devoted to this issue alone. Juliet, being a fertility specialist was on the island for no other reason than to address this problem, seems to become a pretty pointless character without a satisfactory answer to it. With mothers dieing in child birth, Linus torchering his daughter's boyfriend to prevent her from getting pregnant, Ethan kidnapping Clair at night to inject her with technicolor drugs, Roussou having her infant daughter kidnapped from her and raised--on the very same island--without her knowledge, etc. There were so many important plot points tied to this phenomina, and it would have been absolutely easy to tie it up. Here's an example bit of dialog that, while short and vague, would have resolved the issue nicely:&lt;br /&gt;&lt;blockquote&gt;&lt;br /&gt;&lt;b&gt;Jack&lt;/b&gt;: When I get to the source, I'm going to kill you.&lt;br /&gt;&lt;b&gt;Locke&lt;/b&gt;: And how do you plan to do that? Guns and knives can't kill me. I can rip trees out of the ground and prevent women from bearing children. How do you possibly plan to kill me?&lt;br /&gt;&lt;b&gt;Jack&lt;/b&gt;: I have a plan. Wait and see.&lt;/blockquote&gt;&lt;br /&gt;Seriously, it wouldn't have taken much effort, just a passing reference by someone in the know and the issue would have disappeared. Maybe it wasn't the fault of the Man in Black, but was instead one of Jacob's "rules" for the island:&lt;br /&gt;&lt;blockquote&gt;&lt;br /&gt;&lt;b&gt;Locke&lt;/b&gt;, to Linus early in the season: Why would you be upset about killing Jacob? He's a man who never cared about you after everything you gave. He's a man who let mothers die because he made a rule against children being born on this island. His island. He created so much hardship for you and your people, even let your daughter die in front of you.&lt;/blockquote&gt;&lt;br /&gt;With this statement, Linus's conversation at the end with Hurley about how Hurley could be a better leader and make better rules would be much more poignant. Seriously, we could turn the whole notion of good and evil upside down and start to wonder whether Jacob was really good, whether the smoke monster was really bad, and what would be the real ramifications for everybody getting on the plane together and heading back to reality. It would have created a much more interesting metaphysical dilemma for our antagonists, tasked to decide whether Jacob was really the side of good when he did things that seemed so evil; for the sake of the island, of course. We also might have gained a lot of insight into Linus, the quintessential character who seemingly does evil to pursue good ends.&amp;nbsp; If that was the modus operandi of Jacob and even the island at large, we could have seen a lot of things in a different light.&lt;br /&gt;&lt;br /&gt;What the hell did the atomic bomb accomplish at the end of Season 5? If the flash sideways in season 6 were just a view of purgatory where all the losties were already dead, then how could Juliet claim that detonating the bomb "worked"? If everybody died in the atomic bomb explosion, and sent everybody on the island to purgatory, then what do we make of the people still on the island in the "main" timeline? And if, considering what I said about Juliet above, the bomb didn't save anybody and didn't keep the plane from crashing, what purpose did Juliet's character really serve in the show? Sure, she created a bit of a weird and dramatic love triangle with Jack and Kate, and she did get romantically involved with Sawyer, though her death would only have reaffirmed his sense that his loved ones are always taken from him, and would have prevented any further growth of Sawyer's character. The main problem that brings her to the island is simply ignored, and the big self-sacrifice that she made didn't have any effect. If anything, apparently it created a separate reality where everybody either died or was already dead. Go Juliet!&lt;br /&gt;&lt;br /&gt;If the atomic bomb didn't do anything worthwhile, what was the point of the survivors traveling back in time to 1977 in the first place? We didn't really learn anything new about the Dharma initiative, or about the power of the island that they were studying. We already knew that the island had the power to transport things through time and space, and those were the most important things that the Dharma people appeared to have been working on. I know that there were other things of significance that Dharma was supposed to be involved in, from other media like the Lost video games or other guerilla media, but none of those were mentioned even in casual passing in the show so they don't matter. &lt;br /&gt;&lt;br /&gt;Before I let the issue with Juliet drop completely, one of the biggest problems with this show that has been gnawing at me (though it isn't really an unanswered mystery of the show) involves Juliet's death. When she died they were on their way to the Temple to heal Sayid. They put him into the pool of water and he was magically healed. Juliet's still-warm dead body is laying outside the temple, and inside the temple is a magical pool that can bring people back to life. Put two and two together and GO OUTSIDE AND GET JULIETS BODY AND PUT HER IN THE DAMN POOL. How is it that the entire damn braintrust is sitting inside the temple simultaneously mourning the death of Juliet, witnessing the resurrection of a dead Sayid and not once questioning whether somebody else recently-deceased could be brought back as well? That seems to me to be a remarkably stupid oversight on their part. I'm not saying that the idea would even have worked, since nobody has told me any of the details of the workings of that magic pool, but you can be damn certain that if I were there I would have at least given the idea some consideration. There's certainly no harm in trying, if you can toss any old corpse into the water and some of them come back to life and the rest just stay dead. Again, this is just an annoyance and not a real pressing mystery of the show. If anything it just shows how lazy the writers were.&lt;br /&gt;&lt;br /&gt;What the hell was the deal with the cabin, the ash circles, the temple, and Dogan? Dogan apparently had the ability to keep the monster out of the temple, but how? Did anybody else have that power? Was it just an example of Dogan being "special", like several other people were listed as being "special", or was it some kind of gift given to him by Jacob? And what was going on with Jacob's cabin? I think we're lead to assume that the man in black was trapped inside the ring of ash around the cabin, but that doesn't really make sense considering the fact that we see the monster outside the ring in the very first episode of the season. How can he be outside, terrorizing the crash survivors and simultaneously trapped inside whispering "help me" to any passers-by? I also don't think Jacob was the one in the cabin, considering that we never see Jacob having a problem with ash lines in the ground, and we also never see Jacob being invisible like a ghost or needing help from somebody like John Locke. What is the significance of Horace Goodspeed having built the cabin? That's the kind of detail that could just as easily have been left out if they weren't going to explore it. There are lots of details like that throughout the show that they should have just left out if they were going to be too lazy to give an even cursory examination. Maybe the spirit in the cabin is a manifestation of the sentient island, and it was the will of the island itself that needed help from John Locke. It could also be this spirit force that the ash line was trying to protect from the smoke monster. Sure it's all speculation, but there is a lot that they could have done with this incident to gain a further understanding of the island and the relationship between it and Jacob, but they didn't.&lt;br /&gt;&lt;br /&gt;Ilana and Brahm seemed to have expected to find Jacob inside the cabin, but were alarmed when they saw that the ring of ash was broken, and then set the house on fire. Maybe Jacob lived inside the cabin, protected by the ash ring that kept the monster out, and therefore kept Jacob safe from harm. But if Jacob was essentially barricaded in there, how would Jacob have gotten out to go make visits to the various characters on the main land? And, if Jacob were inside that cabin and Ben knew how to find it, why hadn't Ben ever met Jacob? In fact, why was Ben surprised when things inside the cabin started shaking and going crazy? If that was Jacob, why do we see Christian Shepard and Claire in there, people who are associated with the smoke monster? People call it "Jacob's Cabin". Linus didn't expect to see anything in there, Ilana and Brahm expected Jacob to live there, but the only other-worldly being we ever see there is the smoke monster, who we also see outside the ash ring. Forgive me for harping on this particular point, but it is very confusing.&lt;br /&gt;&lt;br /&gt;Why did the smoke monster kill some people, like Mr Eko, but couldn't kill others, like Jack? Where were the bloody goddamn rules when Mr Eko was getting bludgeoned to death, or when the frenchman's arm was getting ripped off, or when Locke was being dragged by the leg into a pit? Everybody seems to know these rules, and despite a complete lack of stated repurcussions everybody seems to follow them blindly to the death, yet they appear to be completely porous and permissive when convenient. More information about the rules would have been much appreciated, but was never given.&lt;br /&gt;&lt;br /&gt;Why wouldn't Ben have known a little bit more about the smoke monster? Conceivably Richard would have known all about him, having met him in person, or could have asked Jacob at least once over the course of hundreds of years. Richard knew that the smoke monster was the arch nemesis of Jacob and the ultimate evil force on the island, yet Ben seemed to view it as some sort of protective force that was, at least, benign and at best benevolent. &lt;br /&gt;&lt;br /&gt;The smoke monster was able to kill Jacob through some elaborate loophole in some kind of rule. Seriously, look at the steps the smoke monster had to take to accomplish his goal: He convinced Linus to move the island (under the guise of Jacob, and nobody knew the wiser) which caused the wheel to fall off it's axis. He then convinced Locke to fix the wheel and also leave the island. Off island, somehow he influenced Ben to bring all the survivors back, along with the body of John Locke, at which point he assumed the form of John's corpse. Using his new form, and the form of his dead daughter, he convinced Ben to finally kill Jacob. This is awefully convoluted to be a plan, and you have to seriously wonder what kind of "rule" this solution was a workaround for. If Ben was the necessary killer, why go through the whole production of killing the mercenaries and sending Ben and Locke off the island only to bring them back? Or, maybe the "rules" stated that only a person who left the island and returned could kill Jacob, but it seems silly for Jacob to build that kind of loophole into the rules and then be surprised when it actually happened.&lt;br /&gt;&lt;br /&gt;And while we're on the topic, why was the smoke monster trying to save the island at that point anyway? Why would the smoke monster kill the mercenaries and tell Linus to move the island? Why doesn't he convince the mercenaries to kill Jacob themselves? It would have been easy to do:&lt;br /&gt;&lt;br /&gt;&lt;blockquote&gt;&lt;b&gt;Locke&lt;/b&gt;: Here's a man named Jacob. If you kill him for me, I will tell you where Benjamin Linus is and I will help you kill him.&lt;br /&gt;&lt;b&gt;Mercenaries&lt;/b&gt;: Okay. Enough said. BANG BANG.&lt;/blockquote&gt;&lt;br /&gt;Much less convoluted and much less prone to errors than the plan he finally settled on. Or, better yet, He could have offered to save Linus' daughter in exchange for Linus killing Jacob. There are any number of other ways that the smoke monster could have asked another person to kill Jacob, but when put in the situation the smoke monster protected Ben and friends from the foreign invaders and actively tried to save the island. If the smoke monster really wanted to destroy the island, why would he help John Locke fix the broken wheel? Constant time flashes would have eventually killed all the surviving candidates on the island. Other uses of that light energy could have been used to destroy the island completely. Why didn't the monster pursue any of these alternatives earlier in the story? &lt;br /&gt;&lt;br /&gt;What makes people special? The smoke monster apparently had the special ability to understand how things work. He was so intuitive that he turned observations about the magnetic attraction of a knife to the ground into a working time machine. How was Hurley about to see dead people, and how was Miles able to communicate with the recently-deceased? What the hell was the deal with Walt? What was really so special about John? How was Desmond able to withstand the electromagnetism of the hatch explosion, and how was he able to see the future (for a short while)? And, for that matter, when did Desmond lose the ability to see the future and why? Are these gifts that Jacob gave or, in the case of the smoke monster, that the current protector of the island can grant? Are these innate abilities that some people just have? Are these powers that the island grants to people to aide in it's own protection? And, if so, wouldn't that imply that the island had intelligence?&lt;br /&gt;&lt;br /&gt;What the hell are the rules? Who makes the rules? How are they enforced? What would have happened if the smoke monster tried to kill Jacob? We heard a whole hell of a lot about the rules throughout the show, but none of the rules were ever explicitly stated. We know that some things didn't seem to be allowed, but we don't know why or what the consequences would have been. We also know that some rules had some pretty big damn loopholes in them that could be exploited in such a way that the rule appears to be completely broken. &lt;br /&gt;&lt;br /&gt;We were told that the light on the island was inside all people, and we were also told that if the light went out that it would go out everywhere, implying that the light going out would cause people to die. News flash: Desmond put the light out, and nothing bad appeared to happen from it. Sure, the island shook a little bit, but that was hardly undoable. Similarly, what would really have happened if the smoke monster left the island? He started off as just a man and we have seen several men, including Jacob and Richard, leave the island. Why couldn't the man in black do the same? What was so bad about him leaving the island that the Mother had to kill all his people, fill in his well, and knock him unconscious to prevent it? I can understand that things would have been different after he turned into the smoke monster (the smoke is tied to the light, if the smoke leaves the island, the light goes out, etc), but what would have been the harm before that?&lt;br /&gt;&lt;br /&gt;This certainly isn't an exhaustive list, there are millions and millions of small questions that have been raised but never addressed. Like I said, we could have easily glossed over all these things if we were given a proper ending. But, the ending as bad as it was made all the other omissions, contradictions, and unresolved issues all the more glaring.&lt;br /&gt;&lt;br /&gt;The fact is this: Lost was supposed to be better than this. It's fans were supposed to be smarter and more dedicated. It's writers more learned, daring, and uncompromising. Lost could have and should have been so much more, but the ending it received cheapened the entire thing. I wish, sincerely, that things had ended differently. Maybe then I wouldn't have to write such a long and rambling blog post about it.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-3754457734977407588?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/3754457734977407588/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/05/lost-finale.html#comment-form' title='3 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/3754457734977407588'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/3754457734977407588'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/05/lost-finale.html' title='Lost Finale'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>3</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-2802260404555382380</id><published>2010-05-21T09:55:00.001-04:00</published><updated>2010-05-21T10:05:37.291-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='Parrot'/><title type='text'>The Fitness of Parrot as a Target Platform</title><content type='html'>In response to the &lt;a href="http://wknight8111.blogspot.com/2010/05/bright-blue-yonder.html"&gt;blog post I wrote the other day&lt;/a&gt;, I received &lt;a href="http://wknight8111.blogspot.com/2010/05/bright-blue-yonder.html?showComment=1274442384238#c4576637928891075936"&gt;a very poignant question&lt;/a&gt; regarding Parrot's stability as a target for HLL compilers. Since the comment was anonymous and I can't ask permission, I'm going to repost a portion of it here:&lt;br /&gt;&lt;br /&gt;&lt;blockquote&gt;However to be honest, its scary that over the last 10 years so much was  done and discarded and is being rewritten. I agree with the "throw away  the first" reasoning but that is something I would expect in a 2-3 year  project not a 10 year old project.&lt;br /&gt;&lt;br /&gt;Could you comment on the risk  associated with for a HLL project that starts to build on Parrot with an  ETA of say 1 year? Can it be said with a reasonably level of certainity  that whatever sub-projects (JIT, treads etc.) are added to Parrot now,  they will take an evolutionary approach rather than being scrapped and  rewritten?&lt;br /&gt;&lt;br /&gt;My intent is not to put down Parrot as I really want  to use it but I want to voice my fears and hope to understand the  parrot-dev perspective on making Parrot a stable (for some definition of  stable) platform to build on.&lt;/blockquote&gt;This really is two questions rolled into one:&lt;br /&gt;&lt;ol&gt;&lt;li&gt;Parrot has been in development for about 10 years, why are these big changes being made so late in the project?&amp;nbsp;&lt;/li&gt;&lt;li&gt;Are these big changes going to affect the stability of the external API, and is Parrot a stable, reliable platform for HLL designers?&lt;/li&gt;&lt;/ol&gt;I'll try to tackle these two separately.&lt;br /&gt;&lt;br /&gt;The 0.0.1 release of Parrot was cut in September 2001, according to &lt;a href="http://trac.parrot.org/parrot/browser/trunk/docs/parrothist.pod"&gt;documentation in our repository&lt;/a&gt;. I don't know much about the early history of the project; I've only been a Parrot contributor for about &lt;a href="http://wknight8111.blogspot.com/2008/02/yay-im-being-helpful.html"&gt;two and a half years now&lt;/a&gt;, about one fifth of the total life of the project. Looking through the &lt;a href="http://trac.parrot.org/parrot/browser/trunk/NEWS"&gt;NEWS&lt;/a&gt; entries starting back at 0.0.2, it does look like things were moving at a rapid pace back in 2001 and so maybe Parrot should be a lot further along than it is now.&lt;br /&gt;&lt;br /&gt;There are a few responses to this. The obvious ones: Volunteer developers are a limited resource, and Parrot is an enabling technology so many developers have been focusing their efforts outside the core (HLL compilers, libraries, programs written using both of those, etc). Core development effort and HLL compiler development effort are  certainly not directly transferable, but in the early years there was a  lot of cross-work among the central developers, and this did play a factor.  There are less obvious reasons, such as attempts to support various platforms (Parrot has supported or attempted to support several architecture and compiler combinations, some of which were exotic even 10 years ago), attempts to improve performance throughout (sometimes prematurely), and the fact that the NEWS file is misleading because the early "action-packed" releases that seem to show a high development velocity were not time-based releases and often represent several months of work. Hell, there were only 6 releases in all of 2003 and 2004 combined, but the NEWS entries for those releases don't look any more impressive than the &lt;a href="http://wknight8111.blogspot.com/2010/05/parrot-240-sulfur-crest-released.html"&gt;news release for 2.4&lt;/a&gt;. The pace of development is certainly not constant (though I would say it is increasing) and while 10 years looks like a lot, it's 10 years of inconsistent development time from contributors with other things going on as well. &lt;br /&gt;&lt;br /&gt;The biggest factor to keep in mind is that Parrot development from 2001 until 2010 &lt;i&gt;didn't happen in a straight line&lt;/i&gt;. Parrot started out as simply the internals engine for the Perl6 compiler and grew to become larger than that. Along the way a lot of designs and plans were hammered out, then thrown away entirely. As I mentioned last time however, there isn't a whole hell of a lot of prior art in the area of dynamic language VMs; guesses were made and some of them ended up being wrong. &lt;br /&gt;&lt;br /&gt;We could have declared Parrot more or less "complete" some time ago and shipped a program that was known to have some serious flaws. Instead what we've been doing, and what I think is the more responsible course of action: fix it until it is correct. So to answer the first question: yes it's been 10 years, but Parrot isn't perfect yet and we are going to continue working on it until it is, &lt;i&gt;however long that takes&lt;/i&gt;.&lt;br /&gt;&lt;br /&gt;The second question can itself be decomposed into several parts: Will things that I rely on change? If so, how quickly will they change? Will the work I do now need to be thrown out when the next big feature set lands? Is Parrot mature enough for me to build on top of it a software project that I want to lead into maturity? These are all good questions, and very concerning for HLL developers. The answer isn't clear-cut, but I will try to explain it as well as I can.&lt;br /&gt;&lt;br /&gt;The most important point I can raise when tackling this question is to mention &lt;a href="http://trac.parrot.org/parrot/browser/trunk/docs/project/support_policy.pod"&gt;our support policy&lt;/a&gt;. Our support policy, which I have complained about as being too strong, goes something like this:&lt;br /&gt;&lt;ol&gt;&lt;li&gt;We have 4 supported releases per year, every 3 months. We support these releases, including bug fixes, until the next supported release comes out. HLL compilers and other external projects are highly encouraged to target these supported releases (many do not, but they do so at their own risk).&lt;/li&gt;&lt;li&gt;We have a defined external API which cannot be changed without notice. A deprecation notice for any change to any externally-visible feature or behavior must be included in a supported release before that interface can be changed or removed. Assuming you target supported releases as suggested in #1 above, this means you have 3 months of prior warning to prepare your project before disruptive changes to Parrot are made.&lt;/li&gt;&lt;li&gt;New features that are added will be tagged "experimental" so projects can examine them, provide feedback, and get an idea of where things will be going in the future. If people start relying on experimental features and new problems aren't created, they tend to stay around. &lt;/li&gt;&lt;li&gt;Not quite part of the policy, but still relevant: When we cut releases, or change features, we try to do extensive testing in HLLs and external projects. Where problems are found, we typically try to either offer fixes or workarounds.&lt;/li&gt;&lt;/ol&gt;It's interesting to note that many of our most disruptive changes in recent months were actually driven by the HLLs and external projects themselves, as fixes to bugs or longstanding problems. It's also worth mentioning that when the question is raised most HLL developers seem to want Parrot to make fixes and improvements more rapidly than it has been doing.&lt;br /&gt;&lt;br /&gt;Another part of the question is whether the new features and things that we will be adding/changing in coming months will be gradual and be able to be inserted non-disruptively into existing software projects. Let's give a quick rundown of some systems that could be seeing major changes in the next couple months:&lt;br /&gt;&lt;ol&gt;&lt;li&gt;&lt;b&gt;GC&lt;/b&gt;: GC is a very internal thing, when it works properly, you don't even need to know it exists. Any changes here, so long as they don't introduce bugs, will be invisible to the HLL developer, and will only serve to improve performance (or, improve it under certain workloads, depending on algorithm).&lt;/li&gt;&lt;li&gt;&lt;b&gt;JIT&lt;/b&gt;: In the past the JIT system was separate, and you needed a separate command-line switch to activate it. In the future, I'm hoping we get a trace-based system that kicks in automatically when a need is detected. JIT, like GC, shouldn't change execution behavior, only performance, so changes here should be invisible to the HLL developer.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Threads&lt;/b&gt;: We don't really have a good, working, reliable threads implementation now and HLLs are generally not using them. Anything we add/change/fix will be as good as a newly-implemented feature and can be added post-facto by HLL developers with no stress.&lt;/li&gt;&lt;li&gt;&lt;b&gt;NCI&lt;/b&gt;: This mostly affects users of external libraries, and writers of interfaces to those projects. Improved NCI gives us improved access to more libraries. The interface here may change in a disruptive way but, I hope, this won't be a huge issue to most projects&lt;/li&gt;&lt;li&gt;&lt;b&gt;Packfiles&lt;/b&gt;: Packfiles aren't really portable now, so people haven't been using them to their potential. In any case, Packfile structure and handling are mostly transparent to the HLL developer. In the future what we will see are better usage patterns, improved performance and decreased bug volume, at no expense to the HLL developer.&lt;/li&gt;&lt;li&gt;&lt;b&gt;PCC&lt;/b&gt;: We have a pretty extensive library of tests for PCC behavior and a defined standard inferface that won't be changing any time soon. Sure, the internals may change (hopefully become faster) and new features will be added, but exising code will not notice.&lt;/li&gt;&lt;li&gt;&lt;b&gt;PASM&lt;/b&gt;: We're looking to add a suite of optimizations to PASM. These optimizations will likely be opt-in initially, so absolutely nothing changes for HLL developers who don't want them. If you do want them, your programs will probably only get faster at runtime at the cost of some additional processing in the compiler. This is a standard and non-exciting trade-off that most good compilers offer to their users.&lt;/li&gt;&lt;/ol&gt;In summary, Parrot is a good, stable platform for HLL developers to use. Yes there are big changes planned to the internals of the VM, but most of them are going to bring big improvements to the compilers and library projects that run on top of Parrot, without affecting the externally-visible interface much and without bringing new problems. That really is one of the driving benefits of a VM system like Parrot: Write the improvement once and get the benefits everywhere. The interface is in flux, but changes happen slowly and with plenty of prior warning (and support, where support is needed). As the big systems in Parrot get fixed, things will get even more stable and the benefits will become even more apparent.&lt;br /&gt;&lt;br /&gt;I hope that helps answer some concerns.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-2802260404555382380?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/2802260404555382380/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/05/fitness-of-parrot-as-target-platform.html#comment-form' title='3 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/2802260404555382380'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/2802260404555382380'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/05/fitness-of-parrot-as-target-platform.html' title='The Fitness of Parrot as a Target Platform'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>3</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-4114836344040099802</id><published>2010-05-19T21:04:00.000-04:00</published><updated>2010-05-19T21:04:09.898-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='Parrot'/><title type='text'>Bright Blue Yonder</title><content type='html'>Yesterday &lt;a href="http://wknight8111.blogspot.com/2010/05/parrot-240-sulfur-crest-released.html"&gt;I released 2.4, "Sulfur Crest"&lt;/a&gt;. It's a cooler, more succinct name than "&lt;a href="http://en.wikipedia.org/wiki/Sulfur-crested_Cockatoo"&gt;Sulfur-Crested Cockatoo&lt;/a&gt;". Every time I do a Parrot release I find myself searching through Wikipedia to look for interesting Parrot names (and colorful pictures too!).&lt;br /&gt;&lt;br /&gt;I picked an &lt;i&gt;interesting&lt;/i&gt; quote for the release announcement. Amid the broken grammar and colorful imagery was a key point that I wanted to express: Parrot is getting smarter.&lt;br /&gt;&lt;br /&gt;If you had told me two years ago, or even a year ago, that Parrot would have the features it has now, or that it would be on the verge of adding the features that we are currently planning, I might have been incredulous. Parrot has matured a lot recently, and I really think all that effort is going to start paying off. &lt;br /&gt;&lt;br /&gt;Parrot had a lot of features when I first joined the project. Many of these were first drafts and prototypes which have since been ripped out or reimplemented. This is a normal, natural, healthy part of the software development process, and nobody should be surprised or upset about it. Pop quiz: Name for me one other dynamic language virtual machine, open-source or otherwise, that aims to support the same number of programming languages and runtime environments that Parrot aims to support? Smalltalk comes to mind as a virtual machine for a dynamic language, but that VM backend was never intended to support anything besides Smalltalk. At least, not well. .NET supports a few languages, but is certainly not dynamic.&lt;br /&gt;&lt;br /&gt;There's a great rule in programming that we always throw the first one away. The first draft is never the final manuscript, the first stab in the dark never hits the target. The original Parrot designers and developers didn't have a whole hell of a lot of prior art to copy from. They made some guesses, but there was no way they could have gotten everything right the first time, no way they could have forseen all the problems that the project would run into over time. But that lack of information and that uncertainty can't hold people back. You grit your teeth, you write a prototype, and you resolve to yourself that &lt;i&gt;you always throw the first one away&lt;/i&gt;.&lt;br /&gt;&lt;br /&gt;Let's take JIT for an example. In a basic JIT system, you have a snippet of code with a runtime cost of &lt;i&gt;E&lt;/i&gt;. A JIT attempts to compile that code into a piece of machine code with a runtime:&lt;br /&gt;&lt;br /&gt;&lt;i&gt;OE &amp;lt; E&lt;/i&gt;&lt;br /&gt;&lt;br /&gt;Where &lt;i&gt;O&lt;/i&gt; is some sort of optimization factor. Of course, there is some kind of overhead &lt;i&gt;H&lt;/i&gt;, in compiling the code at runtime:&lt;br /&gt;&lt;br /&gt;&lt;i&gt;H + OE &amp;lt; E&lt;/i&gt;&lt;br /&gt;&lt;br /&gt;&lt;i&gt;H&lt;/i&gt; involves constructing the machine code and performing optimizations. The more optimizations we perform, the larger &lt;i&gt;H&lt;/i&gt; gets, but the smaller &lt;i&gt;O&lt;/i&gt; can get. If &lt;i&gt;E&lt;/i&gt; is large enough and if we tune &lt;i&gt;H&lt;/i&gt; properly, eventually we cross a threshold and the left side of the inequality becomes smaller than the right side of the equation. At this point, JIT is a net performance win for the application. Simple, right?&lt;br /&gt;&lt;br /&gt;Anyway, I'll skip the rest of the theory lesson. The point is this: Parrot's original JIT didn't really have anything a modern compiler system would recognize as "optimization passes", so it wasn't really tunable. It also didn't readily support code generation on any platform besides x86. These things weren't going to be easy (or, even possible) to add either. When the burden of maintenance became too high, and when it was clear that the old prototype system was doing more harm than good, &lt;a href="http://wknight8111.blogspot.com/2009/09/jit-first-project-challenge.html"&gt;we ripped it out&lt;/a&gt;.&lt;br /&gt;&lt;br /&gt;There were a number of such systems in early Parrot, prototype code that got Parrot up to the first plateau but needed to be redesigned and reimplemented for Parrot to move up to the next one.&amp;nbsp; There are several projects either going on right now, or about to get started to work on these systems. When you look at the list, you realize that while it's ambitious, with the current team and the current state of the VM it is entirely plausible that they will all succeed. A quick overview:&lt;br /&gt;&lt;br /&gt;Allison, Bacek and chromatic are talking about several ideas for implementing new, more efficient GC algorithms to replace our first GC prototype.&lt;br /&gt;&lt;br /&gt;Plobsing has been hard at work redoing our freeze/thaw serialization system and, along with NotFound and others is working to fix several long-standing bugs in the PBC system.&lt;br /&gt;&lt;br /&gt;Nat Tuck is preparing a GSoC project to&lt;a href="http://www.parrot.org/content/hybrid-threads"&gt; implement a new threading system&lt;/a&gt; to replace the first prototype version of that system.&lt;br /&gt;&lt;br /&gt;Tyler Curtis is preparing a GSoC project to implement a first prototype of a &lt;a href="http://www.parrot.org/content/past-optimization"&gt;PAST optimization system&lt;/a&gt;.&lt;br /&gt;&lt;br /&gt;Daniel Arbelo is preparing his GSoC project to implement a first prototype of the &lt;a href="http://www.parrot.org/content/what-nfg-why-you-want-parrot-have-it"&gt;NFG string normalization form&lt;/a&gt;.&lt;br /&gt;&lt;br /&gt;Muhd Khairul Syamil Hashim Is putting together a GSoC project to implement an &lt;a href="http://www.parrot.org/content/instrumenting-parrot"&gt;instrumentation framework&lt;/a&gt; for Parrot so we can get some high-quality analysis and debugging tools for Parrot.&lt;br /&gt;&lt;br /&gt;John Harrison is preparing his GSoC project to develop a new NCI frame builder with LLVM, which will replace the old prototype frame builder and (hopefully) lay the groundwork to start replacing our prototype JIT system. &lt;br /&gt;&lt;br /&gt;...and there are even other projects that I can't think of off the top of my head right now too.&lt;br /&gt;&lt;br /&gt;If you had told me a year or two ago that all these projects would be on the table, or that we would have such high chances of them all succeeding, I would have called you crazy. But after all the work we've put in between then and now, and considering the high caliber of our development team, I'm feeling pretty confident that we will be successful and Parrot is going to gain some of these awesome new features. Cross&amp;nbsp; your fingers!&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-4114836344040099802?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/4114836344040099802/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/05/bright-blue-yonder.html#comment-form' title='2 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/4114836344040099802'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/4114836344040099802'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/05/bright-blue-yonder.html' title='Bright Blue Yonder'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>2</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-5061561537336116781</id><published>2010-05-18T14:55:00.001-04:00</published><updated>2010-05-21T09:56:01.022-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='Parrot'/><category scheme='http://www.blogger.com/atom/ns#' term='Release'/><title type='text'>Parrot 2.4.0 "Sulfur Crest" Released!</title><content type='html'>&lt;blockquote&gt;"So there me was beating boulder into powder because me couldn't eat it, and magic ball land in lap. Naturally me think, "All right, free egg." because me stupid and me caveman. So me spent about three days humping and bust open with thigh bone so me could eat it good. Then magic ball shoot Oog with beam, and next thing me know me go out and invent wheel out of dinosaur brain. Magic dino wheel rolls for three short distance until me eat it. The point is, me get smarter. Soon me walk upright, me feather back dirty, matted hair into wings for style, and me stop to use bathroom as opposed to me just doing it as me walk. " -- Oog, Aqua Teen Hunger Force&lt;/blockquote&gt;&lt;br /&gt;On behalf of the Parrot team, I'm proud to announce Parrot 2.4.0 "Sulfur Crest." &lt;a href="http://parrot.org/"&gt;Parrot&lt;/a&gt; is a virtual machine aimed at running all dynamic languages.&lt;br /&gt;&lt;br /&gt;Parrot 2.4.0 is available on &lt;a href="ftp://ftp.parrot.org/pub/parrot/releases/devel/2.4.0/"&gt;Parrot's FTP site&lt;/a&gt;, or &lt;a href="http://parrot.org/download"&gt;follow the download instructions&lt;/a&gt;.  For those who would like to develop on Parrot, or help develop Parrot itself, we recommend using &lt;a href="http://subversion.apache.org/"&gt;Subversion&lt;/a&gt;  on &lt;a href="https://svn.parrot.org/parrot/trunk/"&gt;our source code repository&lt;/a&gt; to get the latest and best Parrot code.&lt;br /&gt;&lt;br /&gt;Parrot 2.4.0 News:&lt;br /&gt;&lt;br /&gt;&lt;br /&gt;&lt;pre&gt;- Core&lt;br /&gt;  + Various long-standing bugs in IMCC were fixed&lt;br /&gt;  + STRINGs are now immutable.&lt;br /&gt;  + use STRINGNULL instead of NULL when working with strings&lt;br /&gt;  + Fixed storage of methods in the NameSpace PMC&lt;br /&gt;  + Added :nsentry flag to force method to be stored in the NameSpace&lt;br /&gt;  + Added StringBuilder and PackfileDebug PMCs&lt;br /&gt;  + Added experimental opcodes find_codepoint and unroll&lt;br /&gt;- Compilers&lt;br /&gt;  + Fixed reporting of line numbers in IMCC&lt;br /&gt;  + Removed deprecated NQP compiler, replaced with new NQP-RX&lt;br /&gt;  + Removed NCIGen compiler&lt;br /&gt;- Deprecations&lt;br /&gt;  + Tools to distribute on CPAN were removed&lt;br /&gt;  + Deprecated dynpmcs have been removed to external repositories&lt;br /&gt;  + Removed RetContinuation PMC&lt;br /&gt;  + Removed CGoto, CGP, and Switch runcores&lt;br /&gt;- Tests&lt;br /&gt;  + Many tests for the extend/embed interface were added&lt;br /&gt;  + done_testing() is now implemented in Test::More&lt;br /&gt;- Tools&lt;br /&gt;  + The fakexecutable tapir is renamed parrot-prove&lt;br /&gt;  + Performance fixes to the pbc_to_exe tool&lt;br /&gt;  + Fix data_json to work outside of trunk&lt;br /&gt;  + The dynpmc GzipHandle (zlib wrapper) was added&lt;br /&gt;  + The library Archive/Tar.pir was added.&lt;br /&gt;  + The library Archive/Zip.pir was added.&lt;br /&gt;  + The libraries LWP.pir, HTTP/Message.pir &amp;amp; URI.pir were added.&lt;br /&gt;- Miscellaneous&lt;br /&gt;  + Six Parrot-related projects accepted to GSoC&lt;br /&gt;  + Improve use of const and other compiler hints&lt;br /&gt;&lt;/pre&gt;&lt;br /&gt;Thanks to all our contributors for making this possible, and our sponsors for supporting this project.  Our next release is 15 June 2010.&lt;br /&gt;&lt;br /&gt;Enjoy!&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-5061561537336116781?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/5061561537336116781/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/05/parrot-240-sulfur-crest-released.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/5061561537336116781'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/5061561537336116781'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/05/parrot-240-sulfur-crest-released.html' title='Parrot 2.4.0 &quot;Sulfur Crest&quot; Released!'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-9195218358735026361</id><published>2010-05-12T09:04:00.001-04:00</published><updated>2010-05-13T08:35:11.125-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='Parrot'/><category scheme='http://www.blogger.com/atom/ns#' term='Strings'/><title type='text'>Immutable Strings branch performance</title><content type='html'>Parrot now uses immutable strings internally for it's string operations. In a lot of ways this was a real improvement in terms of better code and better performance for many benchmarks. However, many HLL compilers, specifically NQP and Rakudo suffered significant performance &lt;i&gt;decreases&lt;/i&gt; with immutable strings. Why would this be?&lt;br /&gt;&lt;br /&gt;It turns out that Immutable strings are great for many operations. Copy operations are cheap, maybe creating a new STRING header to point to an existing buffer. There's no sense to actually copy the buffer because nobody can change it. Substrings are likewise very cheap, consisting of only a new STRING header pointing into the middle of an existing immutable buffer.&lt;br /&gt;&lt;br /&gt;Some operations however are more expensive. A great example of that is string appends. When we append two strings, we need to allocate a new buffer, copy both previous buffers into the new buffer (possibly translating charset and encoding to match) and creating a new header to point to the new buffer. With the older system of COW strings, appends were far less expensive, and many pieces of code--especially NQP and PCT code generators--used them in large numbers.After the switch to immutable strings, any code that was optimized to use lots of cheap appends began to take huge amounts of time and waste lots and lots of memory.&lt;br /&gt;&lt;br /&gt;The solution to these problems is not to use many bare append operations on native strings, but instead to create a new StringBuilder PMC type. StringBuilder stores multiple chunks of strings together in an array or tree structure, and only coalesces them together into a single string buffer when requested. This allows StringBuilder to calculate the size of the allocated string buffer only once, only perform a single set of copies, not create lots of unnecessary temporary STRING headers, etc.&lt;br /&gt;&lt;br /&gt;Several contributors have worked on a branch, "codestring", for this purpose, and some results I saw this morning are particularly telling about the performance improvements it brings. Here's numbers from the benchmark of building the Rakudo compiler:&lt;br /&gt;&lt;pre&gt;JimmyZ: trunk with old nqp-rx real:8m5.546s user:7m37.561s sys:0m10.281s&lt;br /&gt;JimmyZ: trunk with new nqp-rx real:7m48.292s user:7m11.795s sys:0m10.585s&lt;br /&gt;JimmyZ: codestring with new nqp-rx real:6m58.873s user:6m22.732s sys:0m6.356s&lt;br /&gt;&lt;/pre&gt;&lt;br /&gt;The "new nqp-rx" he's talking about there is nqp-rx modified to make better use of the new CodeString and StringBuilder PMCs in the branch. The numbers are quite telling, build time is down about 12%. I think the finishing touches are being put on, and the branch will be merged back to trunk soon.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-9195218358735026361?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/9195218358735026361/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/05/immutable-strings-branch-performance.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/9195218358735026361'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/9195218358735026361'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/05/immutable-strings-branch-performance.html' title='Immutable Strings branch performance'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-2520367210159872885</id><published>2010-05-11T13:07:00.000-04:00</published><updated>2010-05-11T13:07:07.219-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='MediaWiki'/><category scheme='http://www.blogger.com/atom/ns#' term='EmbedVideo'/><title type='text'>EmbedVideo MediaWiki extension</title><content type='html'>I've been doing a little bit of contract work lately on a mediawiki-based learning system. One of the features that was requested of this site was the ability to embed flash videos, primarily from &lt;a href="http://youtube.com/"&gt;YouTube&lt;/a&gt;, but also from other kid- and school-friendly hosting sites such as &lt;a href="http://teachertube.com/"&gt;TeacherTube.com&lt;/a&gt; and &lt;a href="http://kidstube.com/"&gt;KidsTube.com&lt;/a&gt;. KidsTube has since announced that they will be going offline soon, so support for them isn't particularly important now, but the idea still stands.&lt;br /&gt;&lt;br /&gt;I did a search on &lt;a href="http://mediawiki.org/"&gt;mediawiki.org&lt;/a&gt; to find a suitable embedding extension for flash videos. A few options pop up at the top of the results list, but I notice some strange similarities: EmbedVideo, EmbedVideo++, EmbedVideoPlus, all of which appear to be forks of the same original codebase. I also found at least two other independently-developed variants of the same extension in my searches.&lt;br /&gt;&lt;br /&gt;The original &lt;a href="http://www.blogger.com/www.mediawiki.org/wiki/Extension:EmbedVideo"&gt;EmbedVideo extension&lt;/a&gt; was developed by mediawiki hacker &lt;a href="http://jimbojw.com/wiki/index.php?title=EmbedVideo_Extension"&gt;jimbojw&lt;/a&gt; some time ago, but he has completely stopped maintaining it. In his absence, the EmbedVideo++ extension was forked off his work to add some new features. However, that extension has since been abandoned as well. Add another fork from yet another developer--EmbedVideoPlus--which has already not been modified in over a year. I saw two other variants as well which are almost identical minus some small additions but which weren't listed in the released extensions list. Neither of these look to be actively developed, at least not towards a larger goal of a generally more-useful extension.&lt;br /&gt;&lt;br /&gt;Where the story starts to get funny is that I had to make some of my own modifications to the extension, and was about two button clicks away from hosting my own version publicly when I had an insight: Why add another version, when I could try to take over maintainership of some (or all!) of the existing versions? Two of them are listed as "abandoned" on the mediawiki.org website. One other hasn't been modified in over a year. Two more are slightly more recent but by no means under active development. Each developer has done just enough work to get their own changes added, but stopped active development as soon as that was complete. I think I can do a little better than that. At the very least, I can put the project into some kind of public source control and make sure it gets exposure and some level of development sanity.&lt;br /&gt;&lt;br /&gt;So, Sunday afternoon I sent out a handful of emails to the involved developers: I would like to take over the project, merge all the variants together into a single extension, throw it up on github, do some new development, and actively maintain it. I've only received a reply back from Jimbojw so far, but he seemed enthusiastic.&lt;br /&gt;&lt;br /&gt;Yesterday I created &lt;a href="http://github.com/Whiteknight/mediawiki-embedvideo"&gt;a project on Github&lt;/a&gt; and uploaded some initial files. The original EmbedVideo.php file with some modest modifications, LICENSE, CREDITS, and README files, and some other related stuff. I've got a few big TODO items that I want to tackle in the coming weeks and months:&lt;br /&gt;&lt;ol&gt;&lt;li&gt;Refactor the code to follow current best practices&lt;/li&gt;&lt;li&gt;Test the code to work with newest development versions of MediaWiki. I've heard some reports that there are some incompatibilities.&lt;/li&gt;&lt;li&gt;Add proper i18n support.&lt;/li&gt;&lt;li&gt;Use, where possible, a dedicated javascript library for the purpose such as &lt;a href="http://code.google.com/p/swfobject/"&gt;SWFObject&lt;/a&gt;. &lt;/li&gt;&lt;li&gt;Add in some support for new video hosting websites&lt;/li&gt;&lt;li&gt;Add in some other bits and pieces from the other extension forks, as possible.&lt;/li&gt;&lt;/ol&gt;I'm hoping to get some nice cleanup work started, and maybe get a few items ticked off this list soon so I can cut a new public release of the extension.&lt;br /&gt;&lt;br /&gt;I'll post more news and updates as development progresses.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-2520367210159872885?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/2520367210159872885/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/05/embedvideo-mediawiki-extension.html#comment-form' title='2 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/2520367210159872885'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/2520367210159872885'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/05/embedvideo-mediawiki-extension.html' title='EmbedVideo MediaWiki extension'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>2</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-4152059276838835007</id><published>2010-05-06T08:51:00.000-04:00</published><updated>2010-05-06T08:51:01.754-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='Git'/><category scheme='http://www.blogger.com/atom/ns#' term='Parrot'/><title type='text'>The Merits of a Distributed Workflow</title><content type='html'>Long-time Parrot contributor &lt;a href="http://wknight8111.blogspot.com/2010/05/svn-or-git.html?showComment=1273113716110#c856831411754526027"&gt;kid51 posted a nice comment&lt;/a&gt; on &lt;a href="http://wknight8111.blogspot.com/2010/05/svn-or-git.html"&gt;my previous post about Git and SVN&lt;/a&gt;. He had some issues with the proposed distributed workflow I suggested. As my reply started to grow far too long for the stupid little blogger comment edit box, I realized I should turn it into a full blog post.&lt;br /&gt;&lt;br /&gt;I've been pretty conflicted myself over the idea of a release manager. On one hand releases are boring which means anybody can do them and they don't require a huge time commitment. On the other hand, there is hardly a lot of "management" that goes on in a release: The release manager has some authority to declare a feature freeze and has some ability to select which revision becomes the final release; but that's it. Sure release managers also try to motivate people to perform the regular tasks like updating the NEWS and PLATFORM files, but my experience is that the release manager ends up doing the majority of those tasks herself. I really feel like we need more direction and vision than this most months, and the release manager is a good person (though not the only possible person) to do it.&lt;br /&gt;&lt;br /&gt;Our releases, especially supported releases, have had certain taglines stemming back to PDS in 2008. The 1.0 release was famously labeled "Stable API for developers" and the 2.0 release was labeled "&lt;a href="http://www.mail-archive.com/parrot-dev@lists.parrot.org/msg00234.html"&gt;ready for production use&lt;/a&gt;", when even a &lt;a href="http://wknight8111.blogspot.com/2010/01/parrot-1x-months-and-beyond.html"&gt;cursory review&lt;/a&gt; shows that these two releases hardly met their respective goals. A release manager with more authority to shape that release, and more authority to shape previous releases as well might have done more to change that. A development focus, be it for weekly development between #ps meetings or for a monthly release, only matters if somebody is focusing on it and motivating the rest of the community to focus as well. That person either needs to be the architect or some other single authority (though playing cheerleader and chief motivator constantly would be quite a drain on that person) or the release manager. The benefit to using the release manager to motivate the team and shape the release is--even though it's more of a time commitment for the release manager--that we share the burden and no one person gets burnt out month after month.&lt;br /&gt;&lt;br /&gt;A tiered development system has a number of benefits. Bleeding edge development can occur unfettered (as it happens now in branches). From there we can pull features into integration branches where we assure all the assorted changes and new additions work well together. Development releases can be cherry-picked to represent the stable baseline features that we want people to play with and rely on, and long-term supported releases would represent only those features which are tested, documented, and worthy of being included under our deprecation policy. I don't think end-users of our supported releases should ever be exposed to features marked "experimental", for instance, or any feature at all that we don't want covered by our long-term deprecation policy. If any feature included in a supported release must be deprecated and cannot be removed or fundamentally altered for at least three more months, we should be particularly careful about including new things in a supported release, and there should be some level of gatekeeper who is able to say "this thing isn't ready for prime time yet". That person, I think, should be the release manager.&lt;br /&gt;&lt;br /&gt;Compare our system for dealing with new experimental features (Goes into trunk, maybe with mention in #ps but with very little fanfare, and is then automatically included in the next release unless somebody removes it), to a system where features are added to a development branch, vetted, tested, documented, and then pulled into a release candidate branch only when it's known by the community to pass muster. I think that's a much better system and would lead us to releases with higher overall quality and stability.&lt;br /&gt;&lt;br /&gt;All this sort of ignores your point, which is an extremely valid one, that switching to Git represents more than just a small change in the commands that a developer types in to the console. Yes, it's a small difference to say "git commit -a" instead of saying "svn commit". Yes, it's a small personal change for me to commit locally and then push. The far bigger issues are the community workflow and even the communtity culture changes that will occur because of Git. These things don't need to change, we could lock Git down and use it exactly the way we use SVN now, but I don't think any of the Git proponents in the debate want it that way.&lt;br /&gt;&lt;br /&gt;I would much rather plan for the changes in work flow and even build enthusiasm for them primarily than sell the small changes in tooling and then be swept up in larger culture changes that nobody is prepared for. I think we should want these changes and embrace them, in which case Git is just the necessary vehicle, and not he end in itself.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-4152059276838835007?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/4152059276838835007/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/05/merits-of-distributed-workflow.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/4152059276838835007'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/4152059276838835007'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/05/merits-of-distributed-workflow.html' title='The Merits of a Distributed Workflow'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-1221685109461480583</id><published>2010-05-05T18:21:00.000-04:00</published><updated>2010-05-05T18:21:46.405-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='Git'/><category scheme='http://www.blogger.com/atom/ns#' term='Parrot'/><title type='text'>SVN or Git</title><content type='html'>As I mentioned in my last post, the discussion about whether we move to Git from SVN has been raging pretty hard in the past few days, and while the general opinion seems to be one in favor of a move, there certainly isn't concensus on the point just yet.&lt;br /&gt;&lt;br /&gt;I'm in favor of Git, although I certainly wouldn't list my bent as a "passion" as some opinions in the community have been described. In Parrot world I use SVN exclusively. I could use git-svn, or I could try to use dukeleto's Git mirror on Github, but I don't. As far as I am concerned those add an additional layer of complexity, an additional abstraction that layer that I can ignore most of the time until something goes wrong. It simply isn't worth it to me to try and force a Git interface onto Parrot's SVN repository.&lt;br /&gt;&lt;br /&gt;Let me tell a short story.&lt;br /&gt;&lt;br /&gt;When I was a graduate student I was given a "computer" for my "desk" in my "office" that I could use to do my normal school work and also to develop my thesis. This computer was a complete beast of a performance machine. At least, it had been several years earlier when it was first purchased. By the time I got it, circa 2006, it's 12Gb hard drive and screaming 128Mb RAM weren't quite as impressive comparatively.&lt;br /&gt;&lt;br /&gt;I was paranoid about data integrity, considering I was working on the most important academic project of my whole life on the least qualified machine on campus. So, I took some steps to help shore up my defenses. First thing, I scrounged together several other small harddrives from other dusty computer carcasses. I removed the CDROM drive and jammed four HDDs into my case. Two of them were just hanging inside because there wasn't enough space to physically mount them all. I also had a shared network drive supplied by the university, a flash drive, and a personal laptop.&lt;br /&gt;&lt;br /&gt;I set up SVN on the computer, and backed up my work as follows: On the second drive was where my work was (the first drive was barely large enough to hold Windows XP and the other software I needed). I had a script that I used to simultaneously svn co my work to the third drive and xcopy my working directory to the fourth. I had an hourly task that would then backup my svn repo from the third drive and an xcopy of the directory on the fourth drive to the network shared drive. At the end of each day, I would xcopy my working directory to my flash drive, go home, and store a copy on my personal laptop. About once a week I would email myself a copy of my thesis paper, in hopes that if all else failed, at least Google could safeguard the all-important final deliverable.&lt;br /&gt;&lt;br /&gt;This may all sound excessive, but by the end of the year I had lost two hard drives (one of which actually caught on fire), my flash drive got crushed, and the network drive has experienced several outages.&lt;br /&gt;&lt;br /&gt;Story aside, the idea that there is only one copy of the entire history of the Parrot repository (or even two copies, assuming the server has a tape backup plan) is hardly reassuring to me. This is why I really like the Git idea that everybody has a complete copy of the entire history of the repository.&lt;br /&gt;&lt;br /&gt;Let's look at things from a different angle: Git is a distributed version control system. While it certainly supports it, there is no true need for a single, central, "master" repository to work from. We could, as a community, radically change our workflow if we had Git. If we were going to use it exactly the same way as we currently use SVN, there really isn't enough of a reason to switch.&lt;br /&gt;&lt;br /&gt;What we have now is a single master trunk, which is where most of the action takes place. People make branches, do work, merge them to trunk. Then, on the second Tuesday of every month, somebody copies trunk to a tag, calls it a release, and we continue on with our normal development.&lt;br /&gt;&lt;br /&gt;So let's imagine a different workflow. I'll posit one example, but this certainly isn't the only choice. With Git, everybody has their own local branch where they do work and make commits. Hell, "everybody" here can be a much larger group than the committers we have now: Anybody can make a fork and do work on their own local branches. We could have something like a master integration branch where trusted committers could pull changes from the entire ecosystem. The integration branch would be more like a testbed and less like a master reference copy.&lt;br /&gt;&lt;br /&gt;From the integration branch, a monthly release manager would cherry pick the good stable features of the integration branch into a release master branch. At the end of the cycle, the release manager's master branch would get tagged as a release, and would become the new baseline integration branch. Remaining changes from the previous integration branch could be pulled in if they were worthy or if they gained more maturity. At that point, the old integration branch could disappear, and the cycle starts again.&lt;br /&gt;&lt;br /&gt;This idea is more complex than our current system, but it does have a few nice features. First, we can strive for higher-quality releases by keeping closer track of what ends up in the release branch. Second, we can try to get more people involved by allowing everybody to create a development branch to do "real" work, not just our designated committers. Third, the release manager has more control over releases, including being able to shift the focus on the release and be able to drive it in a way that is not possible now.&lt;br /&gt;&lt;br /&gt;I like Git, I like the idea of it and I like the things that it makes possible. I don't dislike SVN, but it doesn't have any compelling features that make me want to stay with it. SVN is good but not great, and there's one usage pattern for SVN that works but is limiting. SVN isn't hurting Parrot, but it's not lifting us up to the next level either.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-1221685109461480583?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/1221685109461480583/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/05/svn-or-git.html#comment-form' title='3 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/1221685109461480583'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/1221685109461480583'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/05/svn-or-git.html' title='SVN or Git'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>3</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-5959340511496840590</id><published>2010-05-04T19:27:00.001-04:00</published><updated>2010-05-04T19:28:25.239-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='Parrot'/><title type='text'>Been away for a while</title><content type='html'>I've been quite absent from all things Parrot in recent weeks. Work and other engagements have been keeping me occupied, and the advent of spring has motivated my family and myself to spend more time outdoors. My son, who has started teething, has the same threshold for deciding to put things into his mouth as the Perl 6 language designers have for putting features into their language: "Doesn't matter what it is, jam it in there!". But I jest.&lt;br /&gt;&lt;br /&gt;A lot has happened in Parrot-land since I fell off the edge of the world. Parrot 2.3 was released to much fanfare, as always. Allison fixed TT #389, though her strategy was different from what chromatic and I thought was necessary. So long as it works and there are no more bugs, I'm happy to see it closed by any means. TT #389 is one of our older bugs, stemming all the way back to the first Parrot Developer Summit in 2008. It has certainly been a large pain to Rakudo developers in particular.&lt;br /&gt;&lt;br /&gt;Bacek and chromatic also put together an implementation of immutable strings, and following the 2.3 release merged it into trunk. Immutable strings have some performance advantages, but also as was quickly found, some performance drawbacks. Specifically, string append operations can be a little bit more expensive because a new buffer needs to be allocated instead of resizing one of the argument buffers in place. A strategy to get around that, which many of our developers have been pursuing is to reduce the number of append operations.&lt;br /&gt;&lt;br /&gt;The issue of version control software has been bubbling up to the surface again, and a long thread appeared on the mailing list yesterday after some lengthy chats on IRC. If I can continue clawing my way out of my little isolation hole and make another post this week, I would like to discuss that topic a little bit more.&lt;br /&gt;&lt;br /&gt;GSoC students also got selected this week. I'll have plenty to post about that in the coming days and week.&lt;br /&gt;&lt;br /&gt;I'm the release manager for 2.4 coming out on May 18th.&amp;nbsp; Hopefully it's going to be a good one!&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-5959340511496840590?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/5959340511496840590/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/05/ive-been-quite-absent-from-all-things.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/5959340511496840590'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/5959340511496840590'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/05/ive-been-quite-absent-from-all-things.html' title='Been away for a while'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-2798526707898274616</id><published>2010-04-11T15:21:00.000-04:00</published><updated>2010-04-11T15:21:23.869-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='Parrot'/><category scheme='http://www.blogger.com/atom/ns#' term='Community'/><title type='text'>Online Parrot Developer Summit</title><content type='html'>For anybody who hasn't heard the news, the &lt;a href="http://wknight8111.blogspot.com/2010/04/parrot-virtual-developer-meeting.html"&gt;Parrot online developer summit&lt;/a&gt; is starting in about 10 minutes from the time I am writing this blog post. By the time this post circulates through various aggregators and feed readers, it will be start time.&lt;br /&gt;&lt;br /&gt;Hope to see lots of people there today! Definitely make sure to stop by the IRC chatroom, even if you only want to lurk and not participate.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-2798526707898274616?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/2798526707898274616/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/04/online-parrot-developer-summit.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/2798526707898274616'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/2798526707898274616'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/04/online-parrot-developer-summit.html' title='Online Parrot Developer Summit'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-6013349566584762420</id><published>2010-04-11T08:00:00.002-04:00</published><updated>2010-04-11T08:00:00.725-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='MediaWiki'/><category scheme='http://www.blogger.com/atom/ns#' term='PeerReview'/><title type='text'>MediaWiki PeerReview Extension</title><content type='html'>Yesterday I put the code public for the new &lt;a href="http://github.com/Whiteknight/mediawiki-peerreview"&gt;PeerReview extension&lt;/a&gt; I've &lt;a href="http://wknight8111.blogspot.com/2009/06/wittie-wiki-project.html"&gt;been developing&lt;/a&gt;. Actually, I've only been completing and putting the finishing touches on the extension after a friend and colleague, Jason, wrote up the initial versions. &lt;br /&gt;&lt;br /&gt;The request was simple. We want a way for students in a class using MediaWiki to be able to take ownership of a page and be graded based on it's contents. There is a certain disconnect here with the whole open-content philosophy, but this is only for the narrowly-defined classroom setting where we have a central authority (teacher) and a group of editors with particular assignments. This kind of extension would obviously not be used in an open wiki like &lt;a href="http://en.wikibooks.org/"&gt;Wikibooks&lt;/a&gt;, but it could definitely find a happy home in a private wiki used for a class or closed group.&lt;br /&gt;&lt;br /&gt;The PeerReview extension has several components. The first, and most obvious, is a review form that appears at the top of reviewable pages. When a user logs in and goes to a reviewable page, the review form appears and the user can enter a review. The review consists of a single grade (letter or number, as appears in the DB) and a short comment.&lt;br /&gt;&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://1.bp.blogspot.com/_ztFc4AiNk_k/S7_Z432sqII/AAAAAAAAAKA/Uf9jtU0kBeA/s1600/PeerReviewForm.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"&gt;&lt;img border="0" src="http://1.bp.blogspot.com/_ztFc4AiNk_k/S7_Z432sqII/AAAAAAAAAKA/Uf9jtU0kBeA/s320/PeerReviewForm.png" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;&lt;br /&gt;The next component is the idea of a page "owner". A new special page and some database extensions allow a person with sufficient authority (a teacher or administrator) to assign ownership of a page to a student or a group of students. In this way reviews left on a page can be associated with the students who have ownership of that page.&lt;br /&gt;&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://1.bp.blogspot.com/_ztFc4AiNk_k/S7_Z-k65M_I/AAAAAAAAAKI/Yqn_1Fk4S4U/s1600/PageOwner.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"&gt;&lt;img border="0" src="http://1.bp.blogspot.com/_ztFc4AiNk_k/S7_Z-k65M_I/AAAAAAAAAKI/Yqn_1Fk4S4U/s320/PageOwner.png" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;&lt;br /&gt;The third, and currently final component of the extension is a simple dashboard-like interface where a user can view lists of reviews that they have given and received.&lt;br /&gt;&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://3.bp.blogspot.com/_ztFc4AiNk_k/S7_aC8WifnI/AAAAAAAAAKQ/kKvGJ-7acgE/s1600/MyReviews.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"&gt;&lt;img border="0" src="http://3.bp.blogspot.com/_ztFc4AiNk_k/S7_aC8WifnI/AAAAAAAAAKQ/kKvGJ-7acgE/s320/MyReviews.png" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;&lt;br /&gt;The PeerReview extension is functionally, in the sense that everything that is visible is operational and tested. However, it is still a little bit slim on features right now, and there is a large amount of new development needed to really bring this to the next level of utility and usability. Specifically:&lt;br /&gt;&lt;ol&gt;&lt;li&gt;A mechanism to associate students together into class groups, with one or more teachers&lt;/li&gt;&lt;li&gt;An interface for teachers to monitor the reviews given to/from students in their class, including a utility to aggregate reviews into average scores for each student&lt;/li&gt;&lt;li&gt;An improved API with more flexibility and more configurability&lt;/li&gt;&lt;li&gt;Improved aesthetics throughout&lt;/li&gt;&lt;/ol&gt;I am continuing development on this as I am able, but I would definitely like to solicit feedback and maybe even find people interested in contributing to the project.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-6013349566584762420?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/6013349566584762420/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/04/mediawiki-peerreview-extension.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/6013349566584762420'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/6013349566584762420'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/04/mediawiki-peerreview-extension.html' title='MediaWiki PeerReview Extension'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><media:thumbnail xmlns:media='http://search.yahoo.com/mrss/' url='http://1.bp.blogspot.com/_ztFc4AiNk_k/S7_Z432sqII/AAAAAAAAAKA/Uf9jtU0kBeA/s72-c/PeerReviewForm.png' height='72' width='72'/><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-4777704446371116539</id><published>2010-04-10T08:00:00.002-04:00</published><updated>2010-04-10T08:00:07.188-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='Parrot'/><category scheme='http://www.blogger.com/atom/ns#' term='C#'/><title type='text'>Parrot in C#</title><content type='html'>Here's a little bit of proof-of-concept code that is going to interest a few people:&lt;br /&gt;&lt;pre&gt;using System;&lt;br /&gt;using System.Text;&lt;br /&gt;using System.Runtime.InteropServices;&lt;br /&gt;&lt;br /&gt;public static class Program {&lt;br /&gt;    [DllImport("parrot")]&lt;br /&gt;    private static extern System.IntPtr Parrot_new(System.IntPtr parent);&lt;br /&gt;&lt;br /&gt;    [DllImport("parrot")]&lt;br /&gt;    private static extern void Parrot_destroy(System.IntPtr interp);&lt;br /&gt;&lt;br /&gt;    [DllImport("parrot", CharSet=CharSet.Ansi)]&lt;br /&gt;    private static extern System.IntPtr Parrot_new_string(System.IntPtr interp,&lt;br /&gt;        string text, int length, System.IntPtr whatever, int whatever2);&lt;br /&gt;&lt;br /&gt;    [DllImport("parrot", CharSet=CharSet.Ansi)]&lt;br /&gt;    private static extern System.IntPtr Parrot_compile_string(System.IntPtr interp,&lt;br /&gt;        System.IntPtr compiler_name, string code, StringBuilder errmsg);&lt;br /&gt;&lt;br /&gt;    [DllImport("parrot", CharSet=CharSet.Ansi)]&lt;br /&gt;    private static extern void Parrot_ext_call(System.IntPtr interp,&lt;br /&gt;        System.IntPtr sub, string sig);&lt;br /&gt;&lt;br /&gt;    public static void Main(string[] args) {&lt;br /&gt;        Console.WriteLine("Hello World from C#");&lt;br /&gt;&lt;br /&gt;        System.IntPtr interp = Parrot_new(System.IntPtr.Zero);&lt;br /&gt;        System.IntPtr pir = Parrot_new_string(interp, "PIR", 3, System.IntPtr.Zero, 0);&lt;br /&gt;        System.IntPtr sub = Parrot_compile_string(interp, pir, @"&lt;br /&gt;            .sub 'foo'&lt;br /&gt;                say 'Hello from PIR'&lt;br /&gt;            .end",&lt;br /&gt;            new StringBuilder(256)&lt;br /&gt;        );&lt;br /&gt;&lt;br /&gt;        Parrot_ext_call(interp, sub, "-&amp;gt;");&lt;br /&gt;        Parrot_destroy(interp);&lt;br /&gt;&lt;br /&gt;        Console.WriteLine("Hello again from C#");&lt;br /&gt;    }&lt;br /&gt;}&lt;br /&gt;&lt;/pre&gt;I've been kicking around the idea of starting a project to do full bindings between Parrot and .NET. I'm not sure that I'm ready to take on a new project, especially not one of that magnitude, but it's fun to think about.&lt;br /&gt;&lt;br /&gt;I've got some more code examples for this that I'm putting together. Some of them can be pretty messy, especially when we talk about marshalling data between Parrot's various structures and C# code. However, I think it should be very possible to wrap all that messiness inside a series of nice, clean abstractions. It would be especially nice if we could get some scripts to parse through the embedding and extending function lists and generate the C# extern declarations automatically. With those, we can write some nice wrapper classes around the raw functions.&lt;br /&gt;&lt;br /&gt;I'm sure I'll have more to write about this in the days ahead.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-4777704446371116539?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/4777704446371116539/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/04/parrot-in-c.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/4777704446371116539'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/4777704446371116539'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/04/parrot-in-c.html' title='Parrot in C#'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-4672191820424375920</id><published>2010-04-09T19:16:00.001-04:00</published><updated>2010-04-09T19:50:43.854-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='Parrot'/><category scheme='http://www.blogger.com/atom/ns#' term='GSOC'/><title type='text'>GSoC: The Contenders</title><content type='html'>The GSoC applications deadline came and went today at 19:00 UTC, and we have received quite an impressive crop of proposals from prospective students. I don't know how many slots Google will give to the Perl Foundation, and of those I don't know how many will be devoted to Parrot. However, even if only a few of the proposals we received get final approval it will be a great year for Parrot. Some students did submit multiple applications, and at least one student that I am aware of also submitted applications to other organizations, so even under the best circumstances we won't be getting all of the projects below approved and started.&lt;br /&gt;&lt;br /&gt;Anyway, I would like to briefly describe the proposals we have received. I can't go into too much detail because we're entering a period of feedback and revisions, but I can give the gist of what has been proposed.&lt;br /&gt;&lt;br /&gt;&lt;span style="font-size: large;"&gt;GMP Bindings for Parrot - Robert Kuo&lt;/span&gt;&lt;br /&gt;&lt;br /&gt;Robert, better known on IRC by the handle "bubaflub" has been a Parrot contributor for a while now. He has submitted two proposals to Parrot, both of which look quite interesting. His first proposal, proper bindings for the venerable GMP library, is related closely to some of the rantings and ravings I've made previously on this very blog. &lt;br /&gt;&lt;br /&gt;There are two premises behind this project:&lt;br /&gt;&lt;ol&gt;&lt;li&gt;The BigInt and BigNum libraries don't really belong in the Parrot core distribution&lt;/li&gt;&lt;li&gt;The GMP library has a large amount of power and utility that these two PMC types don't even begin to expose&lt;/li&gt;&lt;/ol&gt;Another big benefit here is that GMP bindings can be quite intensive for a new runtime, so if they are done properly in Parrot they could be made immediately available to all languages running on Parrot for free. &lt;br /&gt;&lt;br /&gt;&lt;span style="font-size: large;"&gt;A PAST Optimization Framework for Parrot - Tyler Curtis&lt;/span&gt;&lt;br /&gt;&lt;br /&gt;Tyler, "tcurtis" on IRC, is a Parrot newcomer. He originally showed some interest in my Immutable Strings project idea, but has since been steered in the direction of adding an optimization framework for PAST. Personally I'm happy he made the switch.&lt;br /&gt;&lt;br /&gt;This is another project that I've discussed at length on my blog, and I'm very happy somebody has taken up the challenge of it. A robust and extendable optimization framework would allow other people to quickly and easily write new optimizations. The framework is much more important in this regard than any individual optimization is. Of course, the proposal is also sort of open-ended: Once the framework is in place, Tyler can devote his energy to adding new optimization stages to it from an endless list of possibilities. For this reason, this is one of those projects where we won't be asking "when will he finish?" , but instead we will ask "how much will he complete before time is up?".&lt;br /&gt;&lt;br /&gt;The real power here is adding the optimization framework to PAST, which many (though not all!) of our compiler projects use. In this way, optimizations added once can be immediately used by several compilers. Write once, make faster everywhere. It's quite a compelling project for the performance-weary among us, and it looks like chromatic is quite keen to mentor this one if it gets accepted.&lt;br /&gt;&lt;br /&gt;&lt;span style="font-size: large;"&gt;Hybrid Threads for Parrot - Nat Tuck&lt;/span&gt;&lt;br /&gt;&lt;br /&gt;Threads are another huge issue in Parrot. Our current threading system, while technically "existing" is so messy, bug-ridden, and&amp;nbsp; lousy as to be essentially unusable by our various projects. I've said on a number of occasions that Parrot can't really call itself "Production Ready" as our 2.0 release tagline was supposed to be until it has the scalability and performance improvements that a robust threading system provides.&lt;br /&gt;&lt;br /&gt;Nat Tuck, "Chandon" on IRC, was another person originally interested in Immutable Strings, but has since submitted his application to revamp the threading system in Parrot. It's hard for me to say any other project has the same potential for reward, but it's also hard to say that any other project matches this one in terms of complexity. It's ambitious, to say the least.&lt;br /&gt;&lt;br /&gt;Threads are an extremely esoteric area of infrastructure development, similar to GC. When they work, they just work and nobody even thinks about them. Your software just "scales" the way people need it to. Plus, anybody can write threading code, but it can be extremely difficult to do it correctly. That's why Parrot getting it's part right is of utmost importance.&lt;br /&gt;&lt;br /&gt;Nat's general idea is to use a small number of OS threads to host a large number of discrete tasks and virtual "green" threads transparently. He's also looking into fixing some of the bugs in cloning interpreters that stymie the current threading system and would make implementation of the new system more difficult.&lt;br /&gt;&lt;br /&gt;&lt;span style="font-size: large;"&gt;Improvements to the NCI System and LLVM Stack Frame Builder - John Harrison&lt;/span&gt;&lt;br /&gt;&lt;br /&gt;John Harrison, also known as IRC regular "ash_", has submitted an interesting and extremely pertinent project proposal for an improved NCI system. The title mentions LLVM specifically, but he's also looking at variants using libFFI. He wants to build a pluggable system with multiple available dynamic frame-builders, and Ahead-Of-Time compliation of frames to save runtime effort.&lt;br /&gt;&lt;br /&gt;This project isn't as large or ambitious as some of the others, but in return comes higher probability that the system will be polished and thoroughly tested for robustness by the end of the summer. Massive improvements to the NCI system would also provide major benefits to projects writing library wrappers, like some of the other GSoC projects are doing.&lt;br /&gt;&lt;br /&gt;Chance of success for this project is high, and there are clear benefits to Parrot in having this feature.&lt;br /&gt;&lt;span style="font-size: large;"&gt;&lt;br /&gt;&lt;/span&gt;&lt;br /&gt;&lt;span style="font-size: large;"&gt;Implementing an Instrumentation Tool for Parrot VM - Muhd Khairul Syamil Hashim&lt;/span&gt;&lt;br /&gt;&lt;br /&gt;This project came completely out of left field and is the only one that I never suggested on my blog ahead of time. It's also the only project proposal that came to us with working proof-of-concept code, and the results of that small example already have some of the Parrot developers imagining of all sorts of fancy long-term uses.&lt;br /&gt;&lt;br /&gt;Muhd, "Khairul" on IRC, is planning a system of hooks and callbacks in Parrot that would allow one interpreter to attach to another interpreter at runtime and harvest information about the system as it executes. By adding the appropriate hooks and writing the correct handling routines, we could end up with all sorts of great tools, such as profilers, analyzers, debuggers, etc. And the best part is that we gain the ability to implement all these fun tools in PIR or even another HLL, not in C like some current tools use.&lt;br /&gt;&lt;br /&gt;You don't have to think about it for long to understand the benefits of this project: Identifying code hotspots and bottlenecks, analyzing PMC usage patterns, tracking and mapping control flow, and the list goes on. We may even get to a point when we can replace Parrot's current debugging, tracing, and profiling runcores with this single tool and a variety of different PIR programs. Quite fun to think about.&lt;br /&gt;&lt;br /&gt;&lt;span style="font-size: large;"&gt;NFG and Single-Representation for the Parrot VM - Daniel Arbelo Arrocha&lt;/span&gt;&lt;br /&gt;&lt;br /&gt;Daniel, everbody's favorite "darbelo" from last year's GSoC fame, has proposed two projects this year. The first is an attempt to implement NFG, a new normalization form for Unicode text strings. The benefits for NFG are a perfect one-to-one mapping between a character and it's underlying representation. This in turn can lead to some significant simplifications in the code of Parrot, and some proposed speedups as well.&lt;br /&gt;&lt;br /&gt;NFG is one of the few points in the Parrot design spec that has never even been prototyped in working code, that I am aware of. Everything else has at least been given prototype treatment, but NFG remains nowhere to be seen in the actual codebase.&lt;br /&gt;&lt;br /&gt;Daniel's proposal is to implement NFG in a testbed so that it's performance can be compared to the current string system. If it is a benefit (and we all hope it is!) we can merge it into trunk and start a new era of strings in Parrot. If not, we can continue development on it until it becomes better, or even scrap it entirely. &lt;br /&gt;&lt;br /&gt;&lt;span style="font-size: large;"&gt;Parrot on RTEMS - Daniel Arbelo Arrocha&lt;/span&gt;&lt;br /&gt;&lt;span style="font-size: large;"&gt;Parrot on RTEMS - Robert Kuo&lt;/span&gt;&lt;br /&gt;&lt;br /&gt;We had two submissions from prospective students with the exact same name. Both darbelo and bubflub have been inspired by dukeleto's work in the area and have proposed to work on the RTEMS port of Parrot. A major portion of this work will be improvements to the cross-compilation capability of our current configuration and build system, improvements which will have many benefits outside RTEMS. In addition, these two are proposing to help improve the operation of Parrot on RTEMS.&lt;br /&gt;&lt;br /&gt;As far as I am aware, this submissions were both duplicated to the RTEMS people as well, so even if one of these proposals get accepted it's possible that the student will actually be working with an RTEMS mentor instead of a Parrot one.&lt;br /&gt;&lt;br /&gt;In both cases, I think the alternate proposals for both students look a little more desirable (darbelo's NFG proposal, and bubaflub's GMP proposal), but I've never really used RTEMS so I'm probably biased. I wouldn't definitely not do any crying if one of these "Parrot on RTEMS" projects got accepted, however.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-4672191820424375920?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/4672191820424375920/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/04/gsoc-contenders.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/4672191820424375920'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/4672191820424375920'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/04/gsoc-contenders.html' title='GSoC: The Contenders'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-5207455239066629728</id><published>2010-04-09T12:27:00.000-04:00</published><updated>2010-04-09T12:27:48.494-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='Parrot'/><category scheme='http://www.blogger.com/atom/ns#' term='GSOC'/><title type='text'>GSoC Deadline Approaching</title><content type='html'>&lt;pre&gt;darbelo: less than 9450 seconds to 'SOCKHOP' deadline. Everyone panic!&lt;br /&gt;moritz panics&amp;nbsp;&lt;/pre&gt;&lt;pre&gt;&amp;nbsp;&lt;/pre&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://3.bp.blogspot.com/_ztFc4AiNk_k/S79VdPYFgaI/AAAAAAAAAJ4/vlc2MNsi6cs/s1600/panic.gif" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"&gt;&lt;img border="0" src="http://3.bp.blogspot.com/_ztFc4AiNk_k/S79VdPYFgaI/AAAAAAAAAJ4/vlc2MNsi6cs/s320/panic.gif" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;pre&gt;&amp;nbsp;&lt;/pre&gt;&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-5207455239066629728?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/5207455239066629728/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/04/gsoc-deadline-approaching.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/5207455239066629728'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/5207455239066629728'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/04/gsoc-deadline-approaching.html' title='GSoC Deadline Approaching'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><media:thumbnail xmlns:media='http://search.yahoo.com/mrss/' url='http://3.bp.blogspot.com/_ztFc4AiNk_k/S79VdPYFgaI/AAAAAAAAAJ4/vlc2MNsi6cs/s72-c/panic.gif' height='72' width='72'/><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-3945100156038028564</id><published>2010-04-08T20:00:00.001-04:00</published><updated>2010-04-08T20:00:01.602-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='PIR'/><category scheme='http://www.blogger.com/atom/ns#' term='Parrot'/><title type='text'>Writing Ops in PIR</title><content type='html'>I put together an interesting little code snippet a few days ago that, when finished, would allow writing ops in PIR. I am probably not going to complete work on this idea myself (at least, not any time soon), so I want to put the partially-written code up here for other people to look at and maybe run with.&lt;br /&gt;&lt;br /&gt;&lt;pre&gt;BEGIN_OPS_PREAMBLE&lt;br /&gt;&lt;br /&gt;opcode_t *&lt;br /&gt;Parrot_pir_op_lib_wrapper_func(opcode_t *cur_opcode, PARROT_INTERP)&lt;br /&gt;{&lt;br /&gt;    // TODO: Set dest to the address of the next instruction, after all arguments.&lt;br /&gt;    //       need to parse the number of args from the op signature&lt;br /&gt;    opcode_t * dest = cur_opcode + 3;&lt;br /&gt;    INTVAL opnum = (INTVAL)(*cur_opcode);&lt;br /&gt;    char * const opname = interp-&amp;gt;op_info_table[opnum]-&amp;gt;name;&lt;br /&gt;    STRING * name = string_make(interp, opname, 8, "ascii",&lt;br /&gt;        PObj_constant_FLAG|PObj_external_FLAG);&lt;br /&gt;    PMC * const root_ns = interp-&amp;gt;root_namespace;&lt;br /&gt;    PMC * const ns = Parrot_get_namespace_keyed(interp, root_ns, CONST_STRING(interp, "pir_op_lib"));&lt;br /&gt;    PMC * sub = VTABLE_get_pmc_keyed_str(interp, ns, name);&lt;br /&gt;&lt;br /&gt;    // TODO: Need to convert the op signature into a PCC signature here&lt;br /&gt;    PMC * call_sig = Parrot_pcc_build_sig_object_from_op(interp, PMCNULL, signature,&lt;br /&gt;        cur_opcode);&lt;br /&gt;    Parrot_pcc_set_signature(interp, CURRENT_CONTEXT(interp), call_sig);&lt;br /&gt;    interp-&amp;gt;current_cont   = NEED_CONTINUATION;&lt;br /&gt;&lt;br /&gt;    dest = VTABLE_invoke(interp, sub, dest);&lt;br /&gt;    goto ADDRESS(dest);&lt;br /&gt;}&lt;br /&gt;&lt;br /&gt;END_OPS_PREAMBLE&lt;br /&gt;&lt;br /&gt;op register_op_sub(IN STR, IN STR, IN PMC) {&lt;br /&gt;    PMC * const root_ns = interp-&amp;gt;root_namespace;&lt;br /&gt;    PMC * const sub = $3;&lt;br /&gt;    PMC * const key = Parrot_pmc_new(interp, enum_class_ResizableStringArray);&lt;br /&gt;    STRING * const opname = $1;&lt;br /&gt;    STRING * const sig = $2;&lt;br /&gt;    PMC * const ns = Parrot_get_namespace_keyed(interp, root_ns, CONST_STRING(interp, "pir_op_lib"));&lt;br /&gt;    INTVAL argcount = 3; // TODO: Count number of args in signature&lt;br /&gt;    STRING * const separator = CONST_STRING(interp, "_");&lt;br /&gt;    STRING * longname = Parrot_str_append(interp, opname, separator);&lt;br /&gt;    longname = Parrot_str_append(interp, longname, sig);&lt;br /&gt;&lt;br /&gt;    VTABLE_set_string_keyed_int(interp, key, 0, );&lt;br /&gt;&lt;br /&gt;    VTABLE_set_pmc_keyed_str(interp, ns, $2, sub);&lt;br /&gt;    op_info_t * new_op = (op_info_t *)malloc(sizeof(op_info_t));&lt;br /&gt;    new_op-&amp;gt;name = Parrot_str_to_cstring(interp, opname);&lt;br /&gt;    new_op-&amp;gt;full_name = Parrot_str_to_cstring(interp, longname);&lt;br /&gt;    new_op-&amp;gt;jump = 0;&lt;br /&gt;    new_op-&amp;gt;opcount = argcount + 1;&lt;br /&gt;&lt;br /&gt;    // TODO: Initialize -&amp;gt;type based on what's in signature&lt;br /&gt;    new_op-&amp;gt;type = (arg_type_t *)malloc(sizeof(arg_type_t) * PARROT_MAX_ARGS);&lt;br /&gt;&lt;br /&gt;    // TODO: Initialize based on the signature&lt;br /&gt;    new_op-&amp;gt;dirs = (arg_dir_t *)malloc(sizeof(arg_dir_t) * PARROT_MAX_ARGS);&lt;br /&gt;&lt;br /&gt;    // TODO: what is this?&lt;br /&gt;    new_op-&amp;gt;labels = (char *)malloc(sizeof(char) * PARROT_MAX_ARGS);&lt;br /&gt;&lt;br /&gt;    // TODO: Insert the new op into the optable.&lt;br /&gt;}&lt;/pre&gt;&lt;br /&gt;This code, intended to become a dynop library one day, introduces a single new op: register_op_sub. register_op_sub takes three arguments: the string name of the new op, the string signature of the new op, and the Sub PMC for the new op. An entry is prepared for the interpreter's op_lib table (I haven't found a good way to insert that record yet), and the sub is stored in the "pir_op_lib" namespace. The body of the new op is the function Parrot_pir_op_lib_wrapper. This function body looks up the name of the current op in the op_lib table, then looks up the Sub PMC in the namespace, and will attempt to invoke that sub.&lt;br /&gt;&lt;br /&gt;I haven't worked out all the mechanics of everything, obviously. But once a few more questions get answered and a modest amount of code gets written, this could turn into a very useful and very cool extension library for Parrot.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-3945100156038028564?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/3945100156038028564/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/04/writing-ops-in-pir.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/3945100156038028564'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/3945100156038028564'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/04/writing-ops-in-pir.html' title='Writing Ops in PIR'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-7104590079464839268</id><published>2010-04-07T20:45:00.000-04:00</published><updated>2010-04-07T20:45:23.443-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='Parrot'/><title type='text'>Parrot Virtual Developer Meeting</title><content type='html'>Allison sent a &lt;a href="http://lists.parrot.org/pipermail/parrot-dev/2010-April/004047.html"&gt;message to the list today&lt;/a&gt; to schedule a virtual developer meeting for all Parrot developers. The meeting will be this Sunday, 11 April, at 7:30 UTC.&lt;br /&gt;&lt;br /&gt;All Parrot devs should try to attend. All prospective Parrot devs and well-wishers (GSoC students?) should either try to watch or read the logs afterwards. It will definitely be extremely interesting stuff.&lt;br /&gt;&lt;br /&gt;I hope to see lots of people there, and I hope we get some good planning work accomplished!&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-7104590079464839268?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/7104590079464839268/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/04/parrot-virtual-developer-meeting.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/7104590079464839268'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/7104590079464839268'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/04/parrot-virtual-developer-meeting.html' title='Parrot Virtual Developer Meeting'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-2743449227830395063</id><published>2010-04-07T20:42:00.000-04:00</published><updated>2010-04-07T20:42:15.427-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='Parrot'/><category scheme='http://www.blogger.com/atom/ns#' term='GSOC'/><title type='text'>GSoC Proposal Deadline: Friday</title><content type='html'>&lt;div style="text-align: center;"&gt;&lt;span style="font-size: x-large;"&gt;ATTENTION PROSPECTIVE GSOC STUDENTS!!&lt;/span&gt;&lt;/div&gt;&lt;br /&gt;The deadline for proposals is this Friday at &lt;a href="http://www.timeanddate.com/counters/customcounter.html?month=04&amp;amp;day=09&amp;amp;year=2010&amp;amp;hour=19&amp;amp;min=0&amp;amp;sec=0&amp;amp;p0=0"&gt;19:00 UTC&lt;/a&gt;. Please submit your proposals soon, even if they are not complete: you will get &lt;a href="http://socghop.appspot.com/document/show/gsoc_program/google/gsoc2010/timeline"&gt;feedback and opportunity to revise&lt;/a&gt; and update the proposals. If you have multiple projects and cannot decide which to submit, submit all of them and we will weed them out later.&lt;br /&gt;&lt;br /&gt;Whatever you do, make sure to &lt;b&gt;submit your applications before the deadline&lt;/b&gt; or you will be disqualified!!!&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-2743449227830395063?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/2743449227830395063/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/04/gsoc-proposal-deadline-friday.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/2743449227830395063'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/2743449227830395063'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/04/gsoc-proposal-deadline-friday.html' title='GSoC Proposal Deadline: Friday'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-4991703266585193035</id><published>2010-04-06T21:47:00.001-04:00</published><updated>2010-04-06T21:50:46.512-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='C#'/><title type='text'>Problems with C#</title><content type='html'>I've been using C# at my day job for some time now. When I first started there the place was a little bit disorganized and different projects were using different languages. I did some work in VisualBasic.NET, Perl, C++, and even a little Tcl. Luckily as the place grew we put together some standards, and now all our PC-side and server-side development is done in C#.&lt;br /&gt;&lt;br /&gt;Yes, I am deeply involved in Parrot, the dynamic language VM. Yes, C# is hardly a dynamic language (even with some of the new "dynamic" features that have been added in newer revisions). It seems little bit weird that I would be so enamoured with C#, but I am. In fact, I would venture to say that for Windows systems it is my development language of choice. I've heard decent things about Mono on Linux but never used it myself. On Linux, I think my development language of choice may still be C (or NQP!).&lt;br /&gt;&lt;br /&gt;However, despite my general happiness with the language there are some problems I have with it. Here are my nits with C#:&lt;br /&gt;&lt;br /&gt;&lt;span style="font-size: x-large;"&gt;Constructors&lt;/span&gt;&lt;br /&gt;&lt;br /&gt;I feel like constructors aren't given a fair shake, and I have a few issues with their use. For instance, their use with generics is positively anemic:&lt;br /&gt;&lt;pre&gt;public void Foo&amp;lt;T&gt;() where T : new() {&lt;br /&gt;   T bar = new T(new Baz());&lt;br /&gt;}&lt;br /&gt;&lt;/pre&gt;The constaint "Where T : new()" only mandates that the type T has &lt;i&gt;any&lt;/i&gt; constructor, but in my Foo method I specifically require it to have a constructor that takes a Baz. Generics are supposed to provide typesafety for cases like this, but it's extremely easy to compile a program with this kind of construct and not have any compile-time indication that the types are incompatible.&lt;br /&gt;&lt;br /&gt;And while we're at it, interfaces should be able to specify a list of required constructors. I'm constantly baffled by this omission, I run into problems from it on a regular basis, and I can't imagine why it wouldn't be possible.&lt;br /&gt;&lt;br /&gt;&lt;span style="font-size: x-large;"&gt;Static Methods in Interfaces&lt;/span&gt;&lt;br /&gt;&lt;br /&gt;Speaking of interfaces, for the life of me I can't understand why interfaces cannot be used to specify static class methods and properties. I've seen dozens of half-hearted "explanations" for why we don't have this ability, and none of them compel me at all.&lt;br /&gt;&lt;br /&gt;Let me give an example from code I was working on this very day. I'm putting together an MDI application which can host a heterogenous collection of child forms. Different form types have different properties: Some are multiplicious. Some are "singletons", where only one child window of that type can open at a time. Some windows should start maximized, and some minimized. Some should open at program startup. Some of this information I need to have from the child form before I instantiate it, so it would be really nice to have a static method on the type that I can call to get this info. Basically, I want to do this:&lt;br /&gt;&lt;pre&gt;private method ShowMDIChild&amp;lt;TForm&gt;() where TForm : Form {&lt;br /&gt;  if (IsSingleton&amp;lt;TForm&gt;() &amp;amp;&amp;amp; this.SingletonCache.ContainsKey(typeof(TForm)))&lt;br /&gt;    return;&lt;br /&gt;  TForm form = new TForm(...);&lt;br /&gt;  form.Show()&lt;br /&gt;}&lt;br /&gt;&lt;/pre&gt;The "..." above really is a variable list of arguments, since different types of child forms may want different argument lists to be created. Sure, I could have a large common list of arguments and force all my forms to implement the same constructor with some of the arguments treated as unused shims, but thats a huge hassle (and I can't specify that constructor in a common interface).&lt;br /&gt;&lt;br /&gt;One thing I can do, which is particularly ugly in my mind, is this:&lt;br /&gt;&lt;pre&gt;public interface ISingletonMDIChild {}&lt;br /&gt;&lt;br /&gt;...&lt;br /&gt;&lt;br /&gt;private method ShowMDIChild&amp;lt;TForm&gt;() {&lt;br /&gt;  if (typeof(TForm).GetInterface(typeof(ISingletonMDIChild).FullName) == null) {&lt;br /&gt;    this.ShowMDIChildSingleton&amp;lt;TForm&gt;();&lt;br /&gt;  ...&lt;br /&gt;}&lt;br /&gt;&lt;/pre&gt;And I think we can all agree that this is a hideous mess. Plus, it would require me to write up all sorts of stub interfaces and treat them like boolean class flags. I could use attributes, but the code to reflect over the class and iterate over attributes is even uglier (and &lt;i&gt;much slower&lt;/i&gt;). We also can't always inherit from an abstract parent class because of C#'s restriction to single inheritance. What I want, in a nutshell, is a way to gather a standard set of metadata about classes of a related type in a clean way. I can't think of a better way to do this than static interface methods. I haven't heard any compelling arguments about why we can't do this, but I have heard a lot of half-hearted justifications for it.&lt;br /&gt;&lt;br /&gt;&lt;span style="font-size: x-large;"&gt;Multiple Inheritance.&lt;/span&gt;&lt;br /&gt;&lt;br /&gt;Yes, it's messy. Sure the implementation would be a little bit messy and the coder would probably have to specify the resolution order manually. I don't care. I find so many cases in my day-to-day coding when judicious use of multiple inheritance would be a perfect solution to a problem, and the alternatives are hackish, ugly, and very difficult to maintain.&lt;br /&gt;&lt;br /&gt;&lt;span style="font-size: x-large;"&gt;Type Switches&lt;/span&gt;&lt;br /&gt;&lt;br /&gt;Consider the case of a tree control which contains a number of objects of different types. Each TreeNode in the tree contains a reference to that object in the TreeNode.Tag property. When I right-click on different types of nodes I want to see different context menus pop up, depending on the type of the object at that node. That context menu should call methods on the parent form when clicked.&lt;br /&gt;&lt;br /&gt;One thing I could do is this:&lt;br /&gt;&lt;pre&gt;ContextMenu menu = currNode.Tag.GetContextMenu();&lt;br /&gt;&lt;/pre&gt;But this solution requires the object to build the menu programmatically. To call methods on the parent form I need to do something ugly like pass a variable list of delegates to the object as callbacks, or pass a reference to the Form and make all my methods public, or something like that. Now consider the case where some of the nodes in the list don't have context menus, and some of them call a default version, and we quickly end up with very messy code. What I would really like to do is something like this:&lt;br /&gt;&lt;pre&gt;switch (currNode.Tag is) {&lt;br /&gt;  case ClassFoo:&lt;br /&gt;  ...&lt;br /&gt;  break;&lt;br /&gt;case ClassBar:&lt;br /&gt;  ...&lt;br /&gt;  break;&lt;br /&gt;}&lt;br /&gt;&lt;/t&gt;&lt;/pre&gt;I guess I could do:&lt;br /&gt;&lt;pre&gt;switch (currNode.Tag.GetType().FullName) {&lt;br /&gt;  case "MyNamespace.ClassFoo":&lt;br /&gt;  ...&lt;br /&gt;&lt;/pre&gt;But that's horrible and unmaintainable. Plus, it doesn't respect inheritance. I could do this:&lt;br /&gt;&lt;pre&gt;if (currNode.Tag is ClassFoo) {&lt;br /&gt;  ...&lt;br /&gt;} else if (currNode.Tag is ClassBar) {&lt;br /&gt;  ...&lt;br /&gt;} &lt;br /&gt;...&lt;br /&gt;&lt;/pre&gt;But that's horrible too. On the bright side this one does respect inheritance, but then again it is harder to maintain.&lt;br /&gt;&lt;br /&gt;An ability to do a switch statement, which lends itself naturally to multiple alternatives, with object type would be a huge benefit.&lt;br /&gt;&lt;br /&gt;&lt;span style="font-size: x-large;"&gt;Switch Case Fallthrough&lt;/span&gt;&lt;br /&gt;&lt;br /&gt;And on the subject of the venerable switch statement, it kills me a little bit inside every time I try to implicitly fall through from one case to another and I get a compiler error about it. One forum post I saw actually suggested this gem to workaround the restriction:&lt;br /&gt;&lt;pre&gt;switch (whatever) {&lt;br /&gt;  case 1:&lt;br /&gt;    ...&lt;br /&gt;  goto _2;&lt;br /&gt;  case 2:&lt;br /&gt;_2:&lt;br /&gt;    ...&lt;br /&gt;}&lt;br /&gt;&lt;/pre&gt;Gag me with a stick, and deliver me from evil.&lt;br /&gt;&lt;br /&gt;This is a pretty small list of wishes and, except for the multiple inheritance thing, don't represent huge changes to the language. If anything I think they will make a few tasks much easier. You can complain about my contrived examples if you want.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-4991703266585193035?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/4991703266585193035/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/04/problems-with-c.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/4991703266585193035'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/4991703266585193035'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/04/problems-with-c.html' title='Problems with C#'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-3297426597200542081</id><published>2010-03-29T21:31:00.000-04:00</published><updated>2010-03-29T21:31:04.361-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='Parrot'/><category scheme='http://www.blogger.com/atom/ns#' term='Kakapo'/><title type='text'>Austin's Tickets.</title><content type='html'>Austin, of Kakapo and Close fame, has been creating new tickets avidly as he develops his various projects. I asked him to put together a prioritized list of his various tickets so I could focus my attentions fixing bugs that would benefit him the most. Without much delay, he complied. Here's a truncated list:&lt;br /&gt;&lt;pre&gt;&amp;nbsp;&lt;/pre&gt;&lt;pre&gt;Priorities&lt;br /&gt;1 = blocking something;&lt;br /&gt;2 = not blocking, but I strongly feel this is needed;&lt;br /&gt;3 = nice to have;&lt;br /&gt;4 = I'm not sure of the status.&lt;br /&gt;&lt;br /&gt;ticket priority summary&lt;br /&gt;&lt;strike&gt;1497 1 Expose 'meth_cache' attribute of class.pmc&lt;/strike&gt;1492 1 Get_class namespace) confuses NSes that have same final name.&lt;br /&gt;&lt;strike&gt;1473 1 Handle PMC type is broken&lt;/strike&gt;&lt;br /&gt;1520 1 P6metaclass cannot derive from already-`register`ed PMC type&lt;br /&gt;1429 2 Add -L support to parrot-nqp&lt;br /&gt;1515 2 Duplicate named args cause fatal error in subs&lt;br /&gt;&lt;strike&gt;1133 2 MMD matching does not match 'scalar' with primitive types&lt;/strike&gt;&lt;br /&gt;1448 2 Modify `throw` opcode to pass through pre-configured resume continuation&lt;br /&gt;1514 2 NQP-rx doesn't check storage mode for contextuals&lt;br /&gt;1510 2 NQP-rx doesn't create PMC for string // expressions&lt;br /&gt;1464 2 NQP-rx doesn't handle bare "return" from nested block correctly.&lt;br /&gt;809 2 Opcode 'isa' does not accept RSA PMC for class&lt;br /&gt;1431 2 Register assigned to .local string variable is mis-remembered&lt;br /&gt;1430 3 Change trace offset numbers to hex format&lt;br /&gt;799 3 Configure should explicitly check for symbolic link capability on Linux&lt;br /&gt;1138 3 Create a 'vivify' opcode&lt;br /&gt;1455 3 Exception 'birthtime' attribute not accessible&lt;br /&gt;1446 3 Exception PMC 'clone' does not work&lt;br /&gt;1088 3 Multidispatch 'concatenate_str' does not handle LHS undef&lt;br /&gt;1109 3 MultiSub PMC does not support get_namespace method&lt;br /&gt;1512 3 NQP-rx generates bad access code for non-twigiled attribute&lt;br /&gt;1308 3 NQPrx does not vivify globals in all cases&lt;br /&gt;1107 3 Opcode get_class has failed assertion with garbage input&lt;br /&gt;1481 3 P6metaclass.register does not catch second registration of PMC Class&lt;br /&gt;1437 3 pbc_dump: don't skip varargs words in -d mode&lt;br /&gt;1438 3 pbc_dump: Indicate sub/method boundaries in -d mode&lt;br /&gt;1065 3 PGE should emit inline PIR with Grammar namespace&lt;br /&gt;741 3 PIR line numbers in backtraces are substantially wrong.&lt;br /&gt;816 3 PMC 'Undef' does not support logical_not&lt;br /&gt;1273 3 Provide a parrot 'backtrace' with assertion failures.&lt;br /&gt;1442 3 Sub PMC should support getattribute&lt;br /&gt;1513 3 Test::Builder formats TODO tests wrong/poorly&lt;br /&gt;1439 3 pbc_dump: Mark branch targets in -d mode&lt;/pre&gt;&lt;br /&gt;&lt;br /&gt;I've crossed out the tickets I've already closed, and there are about a half-dozen more that I suspect will be closed within the next week. I'm trying to find the domain experts for some that I don't feel comfortable/competent to close myself.&lt;br /&gt;&lt;br /&gt;I would like to see all of these get resolved not just for the benefit of Kakapo, but for the good of Parrot in general.&amp;nbsp;&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-3297426597200542081?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/3297426597200542081/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/03/austins-tickets.html#comment-form' title='1 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/3297426597200542081'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/3297426597200542081'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/03/austins-tickets.html' title='Austin&apos;s Tickets.'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>1</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-4815264227436035222</id><published>2010-03-23T08:41:00.004-04:00</published><updated>2010-03-23T09:10:04.903-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='Parrot'/><category scheme='http://www.blogger.com/atom/ns#' term='GSOCIdeas'/><title type='text'>Potential GSoC students</title><content type='html'>I've gotten several emails from prospective GSoC students in response to some of the &lt;a href="http://wknight8111.blogspot.com/search/label/GSOCIdeas"&gt;blog posts I've written about potential project ideas&lt;/a&gt;. It looks like we're going to have interested applicants for several of the projects, including a few &lt;a href="http://lists.parrot.org/pipermail/parrot-dev/2010-March/004008.html"&gt;new ones&lt;/a&gt; that I hadn't even considered. These steps are especially important if you are &lt;a href="http://trac.parrot.org/parrot/wiki/NewParrotDeveloperGuide"&gt;new to the Parrot project&lt;/a&gt; and are applying for a GSoC position for the first time with us.&lt;br /&gt;&lt;br /&gt;In each case, the advice I give the students is always basically the same:&lt;br /&gt;&lt;br /&gt;&lt;ol&gt;&lt;li&gt;If you aren't a super C coder, or only have limited experience, pick up a &lt;a href="http://www.amazon.com/Programming-Language-2nd-Brian-Kernighan/dp/0131103628/ref=sr_1_1?ie=UTF8&amp;amp;s=books&amp;amp;qid=1269349318&amp;amp;sr=8-1"&gt;copy of K&amp;amp;R&lt;/a&gt;. &lt;span style="font-weight: bold;"&gt;Read it cover-to-cover&lt;/span&gt;. "real world" code, like that found in Parrot, can differ significantly from the simplified, mickey-mouse code you experience in most introductory programming classes or the exercises in the back of a "Learn C in 24 hours" book. K&amp;amp;R isn't as in-depth as some tutorial books, but it's complete, concise, and serves as a great reference. Likewise, if your project is centered on Perl code but you aren't a great Perl coder, pick up a copy of the &lt;a href="http://www.amazon.com/Programming-Perl-3rd-Larry-Wall/dp/0596000278/ref=sr_1_1?ie=UTF8&amp;amp;s=books&amp;amp;qid=1269349361&amp;amp;sr=1-1"&gt;Camel book&lt;/a&gt;. Nothing beats having a good reference available as you work. If you don't know all the answers, you should know where to &lt;span style="font-style: italic;"&gt;find&lt;/span&gt; all the answers.&lt;br /&gt;&lt;/li&gt;&lt;li&gt;Get a &lt;a href="http://www.parrot.org/download"&gt;copy of the Parrot source code&lt;/a&gt;, &lt;span style="font-weight: bold;"&gt;build it and run the tests&lt;/span&gt;. Start &lt;span style="font-weight: bold;"&gt;reading the code&lt;/span&gt;. You'll definitely want to skim over at least most of the &lt;a href="http://trac.parrot.org/parrot/browser/trunk/src"&gt;src directory&lt;/a&gt;, and you will want to pay particular focus to the subsystems involved in your proposed project. If you're doing a GC project, read the code in src/gc. Strings? src/string. As you read code, make changes; &lt;span style="font-weight: bold;"&gt;especially&lt;/span&gt; changes to code formatting, readability, and documentation. &lt;a href="http://docs.parrot.org/parrot/latest/html/docs/submissions.pod.html"&gt;Submit these changes as patches&lt;/a&gt; and be prepared for feedback. If you see problems but aren't able to fix them yourself, submit bug reports. It's hard to accept a GSoC project if we aren't familiar with the quality (or quantity) of your work.&lt;/li&gt;&lt;li&gt;&lt;span style="font-weight: bold;"&gt;Get involved&lt;/span&gt;. Join the &lt;a href="http://lists.parrot.org/mailman/listinfo/parrot-dev"&gt;parrot-dev mailing list&lt;/a&gt;. This list isn't super-high traffic, so don't worry about getting your inbox flooded. Send a welcome message to the list to &lt;span style="font-weight: bold;"&gt;introduce yourself&lt;/span&gt;. If you're really interested to see what's going on at a deeper level, join the &lt;a href="http://lists.parrot.org/mailman/listinfo/parrot-tickets"&gt;parrot-tickets&lt;/a&gt; list as well This list represents a live feed of events happening in our issue tracker. This will give you an idea of the kinds of problems users have and how we go about fixing them. One of the best things you can do is to join the &lt;a href="irc://irc.parrot.org/#parrot"&gt;#parrot chatroom on IRC&lt;/a&gt;. Introduce yourself to people there, follow the conversations, and get involved. Ask questions. It's hard to accept a GSoC project if we aren't familiar with you as a person, and if you aren't an active community member.&lt;/li&gt;&lt;/ol&gt;So here are my three recommendations in a nutshell: &lt;span style="font-weight: bold;"&gt;Beef up your skills&lt;/span&gt;, &lt;span style="font-weight: bold;"&gt;Get familiar with the project&lt;/span&gt;, and &lt;span style="font-weight: bold;"&gt;get involved in the community&lt;/span&gt;. Do these things, along with submitting a good proposal, and your chances of getting accepted go way up. Good luck!&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-4815264227436035222?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/4815264227436035222/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/03/potential-gsoc-students.html#comment-form' title='1 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/4815264227436035222'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/4815264227436035222'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/03/potential-gsoc-students.html' title='Potential GSoC students'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>1</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-4975098894091137584</id><published>2010-03-18T15:21:00.004-04:00</published><updated>2010-03-18T15:25:25.831-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='Parrot'/><title type='text'>Parrot is In GSoC!!</title><content type='html'>I just heard on IRC that The Perl Foundation and (through collaborative agreement) The Parrot Foundation have been accepted as mentoring organizations in this year's Google Summer of Code.&lt;br /&gt;&lt;br /&gt;I've posted a &lt;a href="http://wknight8111.blogspot.com/search/label/GSOCIdeas"&gt;few project ideas on my blog&lt;/a&gt;. I'll probably post more in the coming days. There are other ideas floating around as well, and you are always welcome to propose your own.&lt;br /&gt;&lt;br /&gt;&lt;span style="font-weight: bold;"&gt;Interested students should definitely get in touch with myself or other Parrot community members AS SOON AS POSSIBLE!!!!&lt;/span&gt;&lt;br /&gt;&lt;br /&gt;Seriously, the more applicants we have, the more cool projects we can end up with and the better the summer will be. If you or somebody you know is an able coder, especially if you have experience using Parrot or hacking on it, please apply.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-4975098894091137584?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/4975098894091137584/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/03/parrot-is-in-gsoc.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/4975098894091137584'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/4975098894091137584'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/03/parrot-is-in-gsoc.html' title='Parrot is In GSoC!!'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-3506752892079238090</id><published>2010-03-18T09:30:00.000-04:00</published><updated>2010-03-18T09:44:42.167-04:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='Parrot'/><category scheme='http://www.blogger.com/atom/ns#' term='Lorito'/><category scheme='http://www.blogger.com/atom/ns#' term='Performance'/><title type='text'>Lean and Mean Parrot.</title><content type='html'>I got a &lt;a href="http://wknight8111.blogspot.com/2010/02/pdd23-exceptions-critique.html?showComment=1268708164814#c7580278710367913112"&gt;comment from old-time Parrot contributor Melvin&lt;/a&gt; the other day in response to my &lt;a href="http://wknight8111.blogspot.com/2010/02/pdd23-exceptions-critique.html"&gt;criticisms of the PDD23 Exceptions system&lt;/a&gt;. He had a few things to say that really echo points that I've been trying to make in the past few months. I would like to reprint his comment here, in full (with some minor copyedits), and address it directly.&lt;br /&gt;&lt;blockquote&gt;&lt;br /&gt;If IMCC needs to look into the op lib during lexical analysis, there is  probably a design flaw. The opcode should probably only be checked  during the parse / semantic check phase. If you want to tweak that much  more performance out of IMCC you'd probably be better autogenerating the  short op-list into imcc.l before calling lex so the token list is built  directly into the scanner. As to your die vs. exit, I agree. This is  where there is too much Perl influence in the VM. Things like 'die'  semantics are better implemented in high level. Give the VM a simple  opcode for termination, let all other things (printing messages, etc.)  be implemented on top of the core op. I was arguing this in 2001/2002.  If I had it my way, the core ops would be lean, and all the PMC grunge  would have been implemented on top of those lean ops, not in the wacky  .pmc/VTABLE mechanism that exists now. 99% of the Parrot lib would be  written in a single, consistent HLL, and the JIT would be manageable.  Right now, even as an original Parrot developer, I cannot wade in.  Parrot requires a mix of Perl, PIR, C and custom build macros. PMCs are  not Objects and Objects are not PMCs. There are still multiple cores for  the fun of it. etc. etc. If Parrot is to ever really be done,  seriously, someone needs to take a scalpel to about 50% of the "sacred"  parts of the codebase and cut the amount of busy work and maintenance  that has to be done, and remove some of the "multiple options" for  accomplishing the same thing. Examples of projects that succeeded much  faster than us are Mono. They built a canonical compiler (C#) and then  proceeded in implementing the platform using the C# compiler. I was  arguing that this would have been Cola at the time, and was my intent  when I wrote PIR and the simple Cola compiler. But 8 years later, there  is no JIT and Parrot is looking at using projects that started well  after itself. I cringe when I look at all the hand-written .pir files in  the distro. This was not the intent of PIR. Anyway, good luck. Don't  take my comments the wrong way, hopefully they'll spark some debate. I'll  check in in 2012. :) &lt;/blockquote&gt;&lt;br /&gt;I don't know how actively Melvin has been following IRC discussion, the mailing list, or this blog. I assume that he hasn't been following too closely, and we really can't expect a person to be following discussion for a project he's not really involved in actively. One of the big points we've been pushing for in recent months is &lt;a href="http://wknight8111.blogspot.com/search/label/Lorito"&gt;Lorito&lt;/a&gt;, the lean set of low-level ops that we are going to use to rewrite large swaths of the codebase. In fact, seeing Melvin discuss this very idea is encouraging and validating: If other people come to the same conclusions that we come to internally, that means they are at least common ideas and maybe even good ones.&lt;br /&gt;&lt;br /&gt;Lorito gives us the opportunity to move the abstraction layer down, and reduce context switches that occur by moving from one side of the line to the other. You don't have to look far in the PCC system, for instance, to see the machinations we have to go through to manage method calls from C. There are complicated structures of function pointers to access parameters from variadic argument lists. There are all sorts of complicated serialization paths to marshal arguments into the CallContext PMC, make the call, and then marshal the arguments back out of it again.&lt;br /&gt;&lt;br /&gt;If we didn't have to worry about making calls from C, or if we at least didn't have to worry about making them regularly, we could avoid all this nonsense and delete several hundred lines of code, and the code we had left would be &lt;span style="font-style: italic;"&gt;faster&lt;/span&gt;.&lt;br /&gt;&lt;br /&gt;Another fact that comes up regularly is that the VTABLE interface is lousy. We've optimized for VTABLE access even though it's an extremely limited interface. There are simultaneously too many functions in the list, too few to support certain PMC types, and not nearly enough flexibility to support the kinds of interoperation mechanisms that we need. Ask this: How many mathematics-related VTABLEs exist? Now ask how many of them are even remotely useful outside the handful of numerical PMC types? Why does every single PMC type need to carry around pointers to all these functions that never get used? Why don't VTABLEs support proper MMD? Why don't all VTABLEs do any kind of MMD, even if it's wrong? Why do we have so much trouble inheriting VTABLEs between core PMC types and user-defined objects?&lt;br /&gt;&lt;br /&gt;If we re-write the whole PMC system in Lorito, we can improve VTABLE dispatch semantics, maybe unify VTABLEs and METHODs, and expand the list of VTABLEs to hold any operations that we need. If we make the distinction that a VTABLE is like a method but only has a fixed number of positional arguments and can only be called from Lorito code, we could use a separate streamlined dispatch path that would be far faster than VTABLE overrides in PIR are now, only marginally slower than C-level VTABLES are, with the flexibility to plug into Parrot's MMD system for proper dispatch, and significantly more powerful and flexible than what we currently have. Plus, using hashes and named lookups, types could define a Sub of any arbitrary name to be a VTABLE, look up by any name we want to use with any arity we need for that operation, and we would only need one "VTABLE not found" handler to fall back to if a suitable one hasn't been found.&lt;br /&gt;&lt;br /&gt;Runcores that don't build on every platform and are not well-tested by default should disappear. Sure, there is some benefit to having them as an academic exercise. Sure, it's cool to say we have an obscure and complicated dispatch mechanism with fun properties. But the reality is this: &lt;span style="font-style: italic;"&gt;We're not using them&lt;/span&gt;.   When we create a fakecutable binary for HLL projects like Rakudo, it generally hard-codes in the name of the runcore to use. So &lt;span style="font-weight: bold;"&gt;if&lt;/span&gt; our primary users can't select their own runcore, and &lt;span style="font-weight: bold;"&gt;if&lt;/span&gt; the primary way to use Parrot should be through HLLs (because I'll be damned if we want people in general to be writing PIR directly forever), then having multiple runcores is a huge waste.&lt;br /&gt;&lt;br /&gt;Slow core? It does do bounds checking, but those bounds checks have never been seen to fail in my tenure as a Parrot coder. Drop it.&lt;br /&gt;&lt;br /&gt;Computed-Goto and Predereferenced-Computed-Goto cores? Not all compilers even support these, and they require much more than their fair share of support code to operate at all.&lt;br /&gt;&lt;br /&gt;Switch core? It's available everywhere and it's pretty speedy, but performance decreases pretty dramatically when used with dynops.&lt;br /&gt;&lt;br /&gt;There has been some effort to make runcores pluggable using libraries, so a migration effort to move some of the uncommon ones out of core would be beneficial. I don't think we need them in core, but if other people would like to maintain them elsewhere, I won't complain about it at all.&lt;br /&gt;&lt;br /&gt;Let's get one thing straight: including unused or rarely-used code in your program is not free. There are costs associated with having that code in memory. There are costs associated with the branches and decisions that choose to avoid it. There are costs associated in loading and dynamically linking exported symbols. There are costs associated with compiling all this code These things may not be super-expensive, but they are not free. And keeping code that has no practical benefit for developers or end users without a good reason to have it is not good practice.&lt;br /&gt;&lt;br /&gt;Unnecessary opcodes need to disappear as well. We have well over 1300 opcodes now, probably closer to 1400 at this point. All of these add overhead in terms of memory footprint and lookup effort. It's a huge waste, and I suspect that if we did some actual profiling work we would find only about 200 of them were used with any regularity and about half would be almost completely unused except for one or two calls in obscure tests in the Parrot repo. If PGE and NQP don't use particular ops, and if there is no compelling need to keep them for particular HLLs, or if those HLLs can reasonably turn them into dynops, they should be removed.&lt;br /&gt;&lt;br /&gt;Adding more ops is not a case of "it's a small added convenience with no downside". There are very real downsides.&lt;br /&gt;&lt;br /&gt;Having too many core PMCs is a problem too. Parrot currently has 86 core PMC types by my count. I think we can drop this number to 60 without having to make any cuts that are too painful. If we wanted to be aggressive I think we could go even lower, but we do hit a point where we need to start weighing proportionally-smaller performance gains against the lost utility of removing PMC types that we're dependent on. In any case, I think we can definitely stand to move the following PMC types into dynpmc libraries or even delete them entirely: AddrRegistry, BigInt, BigNum, Boolean, Capture, CPointer, File, 9 of the array types, Key (to be replaced with an array type), Opcode, OpLib, OrderedHash, Pointer, Scalar, and Timer. This would bring us down to 63 PMC types, and several of these that I've listed are already either deprecated or are listed as experimental.&lt;br /&gt;&lt;br /&gt;We as a community have definitely been moving in this direction. Allison especially has been helping to spread a "less is more" philosophy to the design of Parrot and I do appreciate that effort on her part. Making a switch over to Lorito, and then using Lorito to implement Ops, PMCs, and a few other core systems would be a major benefit. chromatic has been a major pusher for the Lorito effort as well, and having somebody of his stature behind the effort lends major credence to it. The effort to slim down the number of ops and PMCs doesn't have as much momentum but there is real evidence that we could be moving in that direction in the near future. I welcome all these changes and hope to see them come to fruition.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-3506752892079238090?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/3506752892079238090/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/03/lean-and-mean-parrot.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/3506752892079238090'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/3506752892079238090'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/03/lean-and-mean-parrot.html' title='Lean and Mean Parrot.'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-2543701930048900987</id><published>2010-03-10T08:00:00.003-05:00</published><updated>2010-03-10T16:00:48.915-05:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='Parrot'/><category scheme='http://www.blogger.com/atom/ns#' term='GSOCIdeas'/><title type='text'>GSoC Idea: GMP Bindings</title><content type='html'>This conversation &lt;a href="http://irclog.perlgeek.de/parrot/2010-03-09#i_2081531"&gt;happened yesterday on IRC&lt;/a&gt;, with some off-topic things edited out:&lt;br /&gt;&lt;span style="font-family:courier;"&gt;&lt;br /&gt;&lt;span style="font-weight: bold;"&gt;darbelo&lt;/span&gt;: That reminds me. I hate our bignums and want them to die...&lt;br /&gt;&lt;span style="font-weight: bold;"&gt;whiteknight&lt;/span&gt;: darbelo: I agree with the bignums thing 100%. I want bignums out of the repo and moved to their own project&lt;br /&gt;&lt;span style="font-weight: bold;"&gt;whiteknight&lt;/span&gt;: There's no sense keeping them when I suspect a majority of users can't use them because they don't have GMP installed&lt;br /&gt;&lt;span style="font-weight: bold;"&gt;darbelo&lt;/span&gt;: Actually, I wouldn't mind them being in the core if they weren't dependant on a lib I don't have.&lt;br /&gt;&lt;span style="font-weight: bold;"&gt;darbelo&lt;/span&gt;: I actually had started to write a stand-alone BigInteger PMC after last year's SoC.&lt;br /&gt;&lt;span style="font-weight: bold;"&gt;whiteknight&lt;/span&gt;: that would make an awesome project too.&lt;br /&gt;&lt;span style="font-weight: bold;"&gt;whiteknight&lt;/span&gt;: I think we should have lots of projects like that, and for developers to be able to pick which solution they want&lt;br /&gt;&lt;span style="font-weight: bold;"&gt;whiteknight&lt;/span&gt;: as we are now, it's easier to force BigInt to pretend to do what we need instead of just using the best solution, which might be DecNumber or something else&lt;br /&gt;&lt;span style="font-weight: bold;"&gt;darbelo&lt;/span&gt;: Maybe, but GMP is much, much more than just bignums. It's a pretty big library.&lt;br /&gt;&lt;span style="font-weight: bold;"&gt;darbelo&lt;/span&gt;: Our PMCs don't even start to scratch the surface of what GMP can do.&lt;br /&gt;&lt;span style="font-weight: bold;"&gt;whiteknight&lt;/span&gt;: darbelo: so moving those PMCs out to a separate library and adding wrappers for other functionality might be nice&lt;br /&gt;&lt;span style="font-weight: bold;"&gt;darbelo&lt;/span&gt;: I would consider a GMP binding much more valuble to parrot than our current use of the lib, yes.&lt;br /&gt;&lt;span style="font-weight: bold;"&gt;bubaflub&lt;/span&gt;: last year i worked a bit with GMP library and suggested to dukeleto we work on a GMP binding for parrot&lt;br /&gt;&lt;span style="font-weight: bold;"&gt;bubaflub&lt;/span&gt;: last year with GSOC and perl 5&lt;br /&gt;&lt;span style="font-weight: bold;"&gt;darbelo&lt;/span&gt;: bubaflub: That would be nice to have.&lt;br /&gt;&lt;span style="font-weight: bold;"&gt;bubaflub&lt;/span&gt;: though we used an existing perl 5 binding (Math::GMPz)&lt;br /&gt;&lt;span style="font-weight: bold;"&gt;whiteknight&lt;/span&gt;: yes, that would be a wonderful project&lt;br /&gt;&lt;span style="font-weight: bold;"&gt;bubaflub&lt;/span&gt;: but we could nab the test suite and what not&lt;br /&gt;&lt;span style="font-weight: bold;"&gt;whiteknight&lt;/span&gt;: exactly. We have the two PMC types, and we could write wrappers for the rest of the library and get all sorts of additional power&lt;br /&gt;&lt;span style="font-weight: bold;"&gt;bubaflub&lt;/span&gt;: i think access to the GMP library in general would be nice; the stuff i worked on last year was setting some foundational stuff for cryptography libraries&lt;br /&gt;&lt;/span&gt;&lt;br /&gt;Parrot has two PMC types that wrap GMP: BigInt nd BigNum. I think, and apparently a few people agree, that these two types have no business being in the core Parrot repository and should be moved to another project. The immediate benefit to this would be that the bindings for GMP could be improved and expanded independently, instead of only providing what little functionality Parrot actually makes direct use of.&lt;br /&gt;&lt;br /&gt;A good GSoC project for this year would be to move (or fork) the current BigInt and BigNum PMC types to a new project and use them as the cornerstone for writing a more comprehensive interface for the GMP library.This could include other PMC types, NCI function wrappers, PMC methods, ops, and other  things to allow access to the power of the GMP library. Adding custom Integer-like and Float-like PMCs that autopromote to their Big- counterparts would be nice too.&lt;br /&gt;&lt;br /&gt;For more info about this project, you could probably get in touch with myself, darbelo, or bubaflub.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-2543701930048900987?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/2543701930048900987/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/03/gsoc-idea-gmp-bindings.html#comment-form' title='3 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/2543701930048900987'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/2543701930048900987'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/03/gsoc-idea-gmp-bindings.html' title='GSoC Idea: GMP Bindings'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>3</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-7535702514262922837</id><published>2010-03-09T12:01:00.000-05:00</published><updated>2010-03-09T12:01:00.475-05:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='PCC'/><category scheme='http://www.blogger.com/atom/ns#' term='Hackathon'/><category scheme='http://www.blogger.com/atom/ns#' term='Parrot'/><title type='text'>Weekend Hackathon</title><content type='html'>This weekend we were supposed to have a &lt;a href="http://irclog.perlgeek.de/parrotsketch/2010-03-02#i_2056733"&gt;hackathon to get the new PCC branch&lt;/a&gt; up and running. The purpose of this branch is, as &lt;a href="http://wknight8111.blogspot.com/2010/02/argument-passing-refactors.html"&gt;I have discussed before&lt;/a&gt;, to rearrange the call sequence so return values are processed after the function invocation instead of before. In the grand scheme of things, especially in comparison to the &lt;a href="http://wknight8111.blogspot.com/2009/10/pcc-hackathon-day.html"&gt;previous PCC refactors&lt;/a&gt;, this ends up being a minor change characterized mostly by massive code deletions instead of needing to write huge new functions or rewrite tons of existing functions. A few bugs stymied completion of the branch, but I have high hopes that the remaining bugs will get worked out soon. This branch was worked on primarily by allison, though I lent an eye as time permitted and chromatic lent some major debugging support as well.&lt;br /&gt;&lt;br /&gt;Very few people ended up working on the PCC branch, even though that was the "official" target of the hackathon. A large amount of effort instead went to work on other &lt;a href="http://trac.parrot.org/parrot/wiki/BranchDescriptions"&gt;branches&lt;/a&gt;. I'm certainly not complaining about the division of effort. In fact, I want to celebrate it. I'm extremely happy to see other worthwhile projects getting extra manhours devoted to them. It's very good to get people working on Parrot in any capacity, and as I mentioned above the PCC work was not a huge project that would have required a dozen developers focusing on it anyway.&lt;br /&gt;&lt;br /&gt;cotto and bacek focused their considerable talents on the ops_pct branch, which aims to replace the Perl5-based ops parser with a bootstrapped version written with PCT. Im not sure about the exact status of that branch, but there was a huge flurry of commits and I have to believe things are progressing rapidly.&lt;br /&gt;&lt;br /&gt;plobsing started a new branch to tackle &lt;a href="http://trac.parrot.org/parrot/ticket/1015"&gt;ticket #1015&lt;/a&gt;. Using some of the new mechanisms he's developed to find and prevent cycles in the freeze/thaw code, he decided to try and fix the problems with cycles in deep clones as well. I don't know the current status, but last I saw his work was going well.&lt;br /&gt;&lt;br /&gt;Coke has started a new branch to continue the makefile cleanups, this time focusing on the recursive makefile for the dynops. He seems to be running into some bugs this morning, but hopefully nothing that cannot be quickly overcome.&lt;br /&gt;&lt;br /&gt;Overall I would label the hackathon a great success. A lot of people came out to IRC to follow progress and work on various projects, and it is all much-appreciated.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-7535702514262922837?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/7535702514262922837/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/03/weekend-hackathon.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/7535702514262922837'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/7535702514262922837'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/03/weekend-hackathon.html' title='Weekend Hackathon'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-7338945764947998396</id><published>2010-03-09T09:04:00.002-05:00</published><updated>2010-03-09T09:10:08.169-05:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='Parrot'/><category scheme='http://www.blogger.com/atom/ns#' term='GSOCIdeas'/><title type='text'>GSoC: Parrot in Summer 2010</title><content type='html'>Jonathan Leto sent out a &lt;a href="http://lists.parrot.org/pipermail/parrot-dev/2010-March/003934.html"&gt;great email&lt;/a&gt; to the list today about Parrot's involvement in GSoC this year. Parrot will be combining together with the Perl foundation again and entering as a single organization. I very much like this arrangement, under the blind assumption that we do better together in terms of student allotment than we do apart. I have no reason to doubt that.&lt;br /&gt;&lt;br /&gt;&lt;span style="font-weight: bold;"&gt;Mentors:&lt;/span&gt; If you want to sign up to be a potential mentor, you can do it on the &lt;a href="http://www.perlfoundation.org/perl5/index.cgi?gsoc_mentors"&gt;Perl foundation wiki&lt;/a&gt;.&lt;br /&gt;&lt;br /&gt;&lt;span style="font-weight: bold;"&gt;Project Ideas&lt;/span&gt;: If you have any project ideas (&lt;a href="http://wknight8111.blogspot.com/search/label/GSOCIdeas"&gt;I know I do!&lt;/a&gt;), list them on the &lt;a href="http://www.perlfoundation.org/perl5/index.cgi?gsoc_2010_projects"&gt;Perl foundation wiki&lt;/a&gt;. If you tell me about the ideas as well, I'll feature them in a blog post and hopefully drum up some interest among prospective students.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-7338945764947998396?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/7338945764947998396/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/03/gsoc-parrot-in-summer-2010.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/7338945764947998396'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/7338945764947998396'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/03/gsoc-parrot-in-summer-2010.html' title='GSoC: Parrot in Summer 2010'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-6388241081430665097</id><published>2010-03-02T08:00:00.000-05:00</published><updated>2010-03-02T08:00:11.611-05:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='PMC'/><category scheme='http://www.blogger.com/atom/ns#' term='Parrot'/><category scheme='http://www.blogger.com/atom/ns#' term='Objects'/><title type='text'>Difference between PMCs and Objects</title><content type='html'>There has been lots of talk and activity lately that has to deal with Parrot Objects.  My rant about exceptions in Parrot has incited Tene to begin a flurry of development on that system, and &lt;a href="http://gitorious.org/kakapo/kakapo"&gt;Austin's Kakapo project&lt;/a&gt; has been regularly pushing the boundaries of what kinds of operations are and should be possible (and finding lots of bugs along the way!). Other people have been bringing up the topic as well, and lots of people are asking lots of questions about the implementation. I'm going to use this post to explain a bit about how &lt;a href="http://trac.parrot.org/parrot/browser/trunk/docs/pdds/pdd15_objects.pod"&gt;Objects&lt;/a&gt; and &lt;a href="http://trac.parrot.org/parrot/browser/trunk/docs/pdds/pdd17_pmc.pod"&gt;PMCs&lt;/a&gt; work in Parrot, and maybe later I'll devote a post or two to ideas for fixing this system.&lt;br /&gt;&lt;br /&gt;PMCs are basically &lt;a href="http://en.wikipedia.org/wiki/Object_%28computer_science%29"&gt;objects&lt;/a&gt;, though extremely simple, flexible, and low-level. PMCs are interacted with, primarily, through the &lt;a href="http://en.wikipedia.org/wiki/Virtual_function_table"&gt;VTABLE&lt;/a&gt; interface. VTABLEs in Parrot are long lists of C function pointers that implement various behaviors. Calling the in-place addition VTABLE, add_i, is done like this in C:&lt;br /&gt;&lt;pre&gt;&lt;br /&gt;VTABLE_add_i(interp, pmc, 5);&lt;br /&gt;&lt;/pre&gt;&lt;br /&gt;...Which translates to this:&lt;br /&gt;&lt;pre&gt;&lt;br /&gt;pmc-&gt;vtable-&gt;add_i(interp, pmc, 5);&lt;br /&gt;&lt;/pre&gt;&lt;br /&gt;By pointing to a per-type VTABLE structure, PMCs with the same type can access a common list of function behaviors without overlapping or needing to do expensive switch/cases over a list of direct function calls. Likewise, determining the type of a PMC means finding the type of the VTABLE it points to:&lt;br /&gt;&lt;pre&gt;&lt;br /&gt;pmc-&gt;vtable-&gt;base_type;  // type number&lt;br /&gt;pmc-&gt;vtable-&gt;whoami;     // type name (Parrot STRING)&lt;br /&gt;pmc-&gt;vtable-&gt;class;          // Class or PMCProxy PMC for the type&lt;br /&gt;&lt;/pre&gt;&lt;br /&gt;Also, if we have the type number, we can look up the particular VTABLE in an array:&lt;br /&gt;&lt;pre&gt;&lt;br /&gt;VTABLE * tbl = interp-&gt;vtable[index];&lt;br /&gt;&lt;/pre&gt;&lt;br /&gt;In a sense, that's all there is to a PMC. All interactions with a PMC happen through this interface of about 185 function pointers. A PMC, by itself, doesn't have things that we would normally associate with "objects" in higher-level systems: &lt;a href="http://en.wikipedia.org/wiki/Attribute_%28computing%29"&gt;Attributes&lt;/a&gt; and &lt;a href="http://en.wikipedia.org/wiki/Method_%28computer_science%29"&gt;Methods&lt;/a&gt;. Sure, PMCs do have a way to associate a C structure, and therefore maintain a list of what we call "attributes", but those aren't directly accessible from PIR without adding some kind of lookup routine to find them and maybe wrap them into one of the Parrot register types (INTVAL, FLOATVAL, STRING, PMC). PMCs also appear to have methods, but this really isn't the case when you look at it closely.&lt;br /&gt;&lt;br /&gt;As I describe in a previous post, the long way to invoke a method on a PMC is like this:&lt;br /&gt;&lt;pre&gt;&lt;br /&gt;$P0 = new ['Foo']&lt;br /&gt;$P1 = find_method $P0, "bar"&lt;br /&gt;callmethodcc $P0, $P1&lt;br /&gt;&lt;/pre&gt;&lt;br /&gt;The find_method opcode is a thin wrapper around the VTABLE_find_method interface function. If I translate this to an extremely condensed and wildly inaccurate pseudo-C listing, we get:&lt;br /&gt;&lt;pre&gt;&lt;br /&gt;PMC * p0 = Parrot_pmc_new(interp, type_Foo);&lt;br /&gt;PMC * p1 = VTABLE_find_method(interp, p0, "bar");&lt;br /&gt;setup_method_call(interp, p0);&lt;br /&gt;VTABLE_invoke(interp, p1);&lt;br /&gt;&lt;/pre&gt;&lt;br /&gt;This is obviously an extremely inaccurate listing, but should do well to illustrate my point. The method is actually a separate PMC type. It can be either a Sub (a .sub written in PIR) or an NCI (a wrapper type around a C function call). To make the call we set up the argument list (the invocant, $P0, is treated sort of like an argument but is kept distinct) and then invoke the method.&lt;br /&gt;&lt;br /&gt;Before they are invoked, methods are stored inside either a Class or PMCProxy PMC associated with that type. When we call VTABLE_find_method(interp, p0, "bar"), we go through this machination:&lt;br /&gt;&lt;pre&gt;&lt;br /&gt;PMC * class = pmc-&gt;vtable-&gt;class;&lt;br /&gt;PMC * methods = class-&gt;data-&gt;methods;&lt;br /&gt;PMC * method = VTABLE_get_pmc_keyed_str(interp, methods, "bar");&lt;br /&gt;&lt;/pre&gt;&lt;br /&gt;What we think of as an "object" and a "&lt;a href="http://en.wikipedia.org/wiki/Class_%28computer_science%29"&gt;class&lt;/a&gt;" is actually a small collection of interoperating PMCs. The PMC itself contains a long list of VTABLEs and a small amount of data stored in a C structure, which cannot be directly accessed from PIR code. The PMCProxy PMC (like Class, which I will describe later, but designed to work with PMC types written in C) contains a hash of methods and a variety of other data. Methods themselves are their own PMCs, complete with their own type data. To really blow your mind consider that, as a PMC, you can call a method on a method, or even a method on a method on a method.&lt;br /&gt;&lt;br /&gt;In short, a PMC is sort of like the building block that is used to create objects and a type system, though the PMCs themselves are not what we normally think of as "objects". The only way to interact with a PMC is through VTABLEs, not attributes or methods. Luckily, VTABLEs exist that allow us to query the object for related attributes and methods, though the PMC itself may not necessarily respond to these requests.&lt;br /&gt;&lt;br /&gt;Using PMCs, Parrot does provide a proper Object system through the use of two special PMC types: Object and Class. Class, as can be guessed, is a "metaobject" that defines type information for objects of a single type. The Class uses a series of PMCs internally to manage things like method PMCs and attributes. The Object PMC is the basic building block of a class instance object. It provides a series of default vtables that allow it to interact with Class the way we expect (to find methods that are stored in the class reliably, for instance) and to provide a set of attributes that are available for access from PIR. PMCs are the almost formless building blocks, Object is a very specific PMC type that provides behaviors that we expect from an OO type system.&lt;br /&gt;&lt;br /&gt;Now that we've covered basic definitions, what are the big operational differences between the two systems? Here's a short list:&lt;br /&gt;&lt;br /&gt;&lt;ol&gt;&lt;li&gt;Object types are defined by Class PMCs. PMCs are defined by PMCProxy PMCs&lt;/li&gt;&lt;li&gt;Class PMCs are created whenever we do a "newclass" or "subclass" operation from PIR. PMCProxy PMCs are created lazily, only when we actually need to introspect a built-in PMC type.&lt;br /&gt;&lt;/li&gt;&lt;li&gt;Objects must be created from a Class, which means the Class PMC must exist before any Objects of that type can be created. PMCs can be created by themselves and &lt;a href="http://trac.parrot.org/parrot/ticket/1473"&gt;generally&lt;/a&gt; don't require instantiation from another PMC.&lt;/li&gt;&lt;li&gt;Objects have very regimented behavior: You can (&lt;a href="http://trac.parrot.org/parrot/ticket/1487"&gt;and should&lt;/a&gt;) expect certain things when you access a named attribute or named Method. In a PMC these behaviors may be overridden to do &lt;a href="http://trac.parrot.org/parrot/ticket/1094"&gt;different&lt;/a&gt; and unexpected things. Specifically, it can be very difficult to get access to named attributes on a PMC unless they are explicitly made visible from PIR (which can be a lot of work, and not a lot of PMC types &lt;a href="http://trac.parrot.org/parrot/ticket/1442"&gt;do it completely&lt;/a&gt;)&lt;/li&gt;&lt;li&gt;Inheritance between PMCs happens at the C level, so C-level attribute structures are merged together and made visible from C code. Inheritance between objects happens at the PIR level, method and attribute lists are combined and made visible as expected when accessed from PIR code. Inheritance from a PMC to an object is almost always broken, if you expect the attributes and methods from the PMC to magically become visible as attributes and methods on the Object. I've never seen inheritance from an Object to a PMC subclass, but I suspect it is broken even worse.&lt;/li&gt;&lt;li&gt;The VTABLEs in the Object PMC all provide an option to use a PIR-based override routine to implement the behavior. To do this, every VTABLE function in the Object PMC searches the associated Class for a similarly named VTABLE Sub PMC and, if one is found, calls that. PMC types almost never search for an override in the Proxy, and if you define one it will never be called (unless you specifically implement the logic to search for and execute it). On a related note the VTABLEs of an Object, because they are stored as PMCs in a Hash in the Class, can be modified at runtime. The VTABLEs of a PMC cannot be (well, I guess you &lt;span style="font-style: italic;"&gt;could&lt;/span&gt; change the pointer to call a different function if your C-foo is strong, but I would prepare for fire and brimstone. Also, I won't fix any "bugs" that arise from this misguided behavior). I estimate at least &lt;a href="http://trac.parrot.org/parrot/ticket/1082"&gt;10% of reported bugs&lt;/a&gt; or feature requests in Parrot come from the "this sucks worse than I would expect" behavior of subclassing Objects from PMCs. If you can get away with it, it is almost always better to delegate to a built-in type instead of inheriting from it directly. But, I can talk more about problems and workaround solutions like this in another post.&lt;br /&gt;&lt;/li&gt;&lt;/ol&gt;So there you have a guide to the differences between Objects and PMCs. PMCs are the low-level building blocks of an object system, and Objects are combinations of several PMCs and a large number of default VTABLEs to implement an expected set of OO behaviors. In a sense, Objects &lt;span style="font-style: italic;"&gt;are &lt;/span&gt;PMCs, but in another sense they really aren't.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-6388241081430665097?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/6388241081430665097/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/03/difference-between-pmcs-and-objects.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/6388241081430665097'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/6388241081430665097'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/03/difference-between-pmcs-and-objects.html' title='Difference between PMCs and Objects'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-1948112077070872344</id><published>2010-03-01T19:01:00.002-05:00</published><updated>2010-03-01T19:07:32.533-05:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='Parrot'/><category scheme='http://www.blogger.com/atom/ns#' term='ParrotProjects'/><title type='text'>NQ-NQP Blog</title><content type='html'>A few days ago &lt;a href="http://wknight8111.blogspot.com/2010/02/parrotprojects-february-2010-edition.html"&gt;I mentioned&lt;/a&gt; an interesting new project called NQ-NQP, an implementation of the NQP language with a flex/bison frontend and an LLVM code generating backend. I've heard tonight that he's &lt;a href="http://greaterthaninfinity.com/"&gt;started blogging about it&lt;/a&gt;. Anybody who is interested in NQP or LLVM stuff might do well to give it a read.&lt;div class="blogger-post-footer"&gt;&lt;img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/4146794174400139442-1948112077070872344?l=wknight8111.blogspot.com' alt='' /&gt;&lt;/div&gt;</content><link rel='replies' type='application/atom+xml' href='http://wknight8111.blogspot.com/feeds/1948112077070872344/comments/default' title='Post Comments'/><link rel='replies' type='text/html' href='http://wknight8111.blogspot.com/2010/03/nq-nqp-blog.html#comment-form' title='0 Comments'/><link rel='edit' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/1948112077070872344'/><link rel='self' type='application/atom+xml' href='http://www.blogger.com/feeds/4146794174400139442/posts/default/1948112077070872344'/><link rel='alternate' type='text/html' href='http://wknight8111.blogspot.com/2010/03/nq-nqp-blog.html' title='NQ-NQP Blog'/><author><name>Whiteknight</name><uri>http://www.blogger.com/profile/16207472474429254890</uri><email>noreply@blogger.com</email><gd:image rel='http://schemas.google.com/g/2005#thumbnail' width='24' height='32' src='http://upload.wikimedia.org/wikibooks/en/0/07/User_Whiteknight.jpg'/></author><thr:total>0</thr:total></entry><entry><id>tag:blogger.com,1999:blog-4146794174400139442.post-5775631169531500324</id><published>2010-02-28T16:50:00.000-05:00</published><updated>2010-02-28T16:50:28.579-05:00</updated><category scheme='http://www.blogger.com/atom/ns#' term='Parrot'/><category scheme='http://www.blogger.com/atom/ns#' term='Objects'/><title type='text'>Proposal to Change find_method</title><content type='html'>Austin Hastings, as part of his &lt;a href="http://gitorious.org/kakapo/kakapo"&gt;Kakapo project&lt;/a&gt; (which I now have a &lt;a href="http://gitorious.org/%7Ewhiteknight"&gt;commit bit to&lt;/a&gt;!) has started creating a &lt;a href="http://gitorious.org/kakapo/kakapo/blobs/master/src/Mimidae/Maker.nqp"&gt;mock object framework&lt;/a&gt;. We were talking about how to implement expected method calls, so I took a look at the find_method VTABLE of the &lt;a href="http://trac.parrot.org/parrot/browser/trunk/src/pmc/object.pmc"&gt;Object PMC&lt;/a&gt; for some inspiration. What I saw was absolutely horrible, so I promptly created a branch to fix it. However, the more I looked and edited, the bigger I found the problems to be. I'll talk more about Kakapo in another post.&lt;br /&gt;&lt;br /&gt;When I do code like this:&lt;br /&gt;&lt;pre&gt;&lt;br /&gt;$P0 = new ['Foo']&lt;br /&gt;$P0.'Bar'()&lt;br /&gt;&lt;/pre&gt;&lt;br /&gt;What is really happening is something similar to this:&lt;br /&gt;&lt;pre&gt;&lt;br /&gt;$P0 = new ['Foo']&lt;br /&gt;$P1 = find_method $P0, 'Bar'&lt;br /&gt;callmethodcc $P0, $P1&lt;br /&gt;&lt;/pre&gt;&lt;br /&gt;Internally, the find_method opcode calls the VTABLE_find_method function on the given object. The object itself is expected then to walk the method resolution order (MRO) of it's inheritance hierachy to find a suitable method and return it. Along the way, the Object PMC needs to completely violate the encapsulation of the Class PMC to gather information about the MRO and then to search the list of methods in the Class for an entry with the given name. In short version, the C code from Object.find_method looks like this:&lt;br /&gt;&lt;pre&gt;&lt;br /&gt;int num_classes = VTABLE_elements(interp, class-&gt;all_parents);&lt;br /&gt;int i;&lt;br /&gt;for (i = 0; i &lt; num_classes; i++) {&lt;br /&gt; cur_class = VTABLE_get_pmc_keyed_int(interp,class-&gt;all_parents, i);&lt;br /&gt; if (VTABLE_exists_keyed_str(interp, class-&gt;methods, name))&lt;br /&gt;    return VTABLE_get_pmc_keyed_str(interp, class-&gt;methods, name);&lt;br /&gt;}&lt;br /&gt;&lt;/pre&gt;&lt;br /&gt;So Object reads the attributes of it's Class PMC directly, and manually traverses the MRO looking for the proper method. This causes a few problems. First, as a mostly stylistic point, this completely breaks encapsulation. We can't make a change to the MRO or the method storage and lookup mechanism in Class without likewise changing the behavior in Object.&lt;br /&gt;&lt;br /&gt;Second point, since Object needs to know how to traverse the MRO and lookup methods, and requires intimate internal knowledge of the classes in the MRO, we are extremely limited in the types of objects that can be in the inheritance hierarchy. That is, we can't define our own metaobject types, we &lt;span style="font-style: italic;"&gt;must&lt;/span&gt; use Class or PMCProxy, or a subclass thereof (and a careful reading of the code suggests that even subclasses will not work). This seems to be a remarkable limitation when you consider some of the diverse high-level languages that Parrot aims to support.&lt;br /&gt;&lt;br /&gt;One thing I tried to do was create a find_method VTABLE in the Class PMC, and then delegate traversal of the MRO to Class instead of Object. This helped improve encapsulation greatly, but created another problem: Now I couldn't call methods on Class itself. Here's example code that broke:&lt;br /&gt;&lt;pre&gt;&lt;br /&gt;$P0 = getclass 'Foo'&lt;br /&gt;$P0.'add_vtable_override'("bar")&lt;br /&gt;&lt;/pre&gt;&lt;br /&gt;What we want to do is call a method on the class object itself, but what we end up doing is finding a method on &lt;span style="font-style: italic;"&gt;objects of that type&lt;/span&gt;, and then trying to call that method on the class object. Problems.&lt;br /&gt;&lt;br /&gt;Let's recap some issues:&lt;br /&gt;&lt;ol&gt;&lt;li&gt;Find_method searches for a method to use on a given invocant&lt;/li&gt;&lt;li&gt;The Class type has methods that need to be accessible through find_method&lt;/li&gt;&lt;li&gt;Object has to break encapsulation and monkey around in Class's internals, which means we can &lt;span style="font-style: italic;"&gt;only use Class&lt;/span&gt; objects, and objects strictly isomorphic to Class (like PMCProxy) in an MRO&lt;/li&gt;&lt;li&gt;We cannot delegate the method lookup operation to the Class object, where it arguably belongs.&lt;/li&gt;&lt;/ol&gt;With these things in mind, I had an idea that &lt;a href="http://lists.parrot.org/pipermail/parrot-dev/2010-February/003891.html"&gt;I sent to the list&lt;/a&gt; which aims to fix all this: Create a new VTABLE function that searches for a method in a metaobject, instead of searching for the method on the invocant (like find_method does now). In terms of PIR, I'm thinking of enabling this kind of sequence:&lt;br /&gt;&lt;pre&gt;&lt;br /&gt;$P0 = new ['Foo']&lt;br /&gt;$P1 = getclass 'Foo'&lt;br /&gt;$P2 = find_class_method 'Bar'&lt;br /&gt;callmethodcc $P0, $P2&lt;br /&gt;&lt;/pre&gt;&lt;br /&gt;I don't want to remove find_method or change it in any way. Bu
