Evolving Bits

On Monday this week Google announced the students accepted into the 2008 Google Summer of Code, and therefore also those for our participation, dubbed the Phyloinformatics Summer of Code. We received 34 applications in total, which is about half as many as in 2007. However, unlike last year, only 1 of those was what we (and presumably Google) consider “spam,” and two have been withdrawn (one of which unfortunately was a really strong application, but the student needed a summer job and the deadline extension made it no longer an option to wait for the Summer of Code acceptance).

Of the remaining 31, about 80% were between reasonable to really strong, which presents a remarkable difference to last year, when the fraction of those was more around 25%. So in contrast to the total number of applications, we had more quality applications this year than we had last year.

By itself that’s a great development, though it already foreshadowed that we would have to make some tough decisions. On top of that, because our “popularity” (as measured by fraction of total applications received by all organizations) is only about half of last year’s, our current allocation of slots by Google is only 5, about half of what it was last year. Given that we offered 14 project ideas ourselves already (11 of which received strong applications), inevitably students ended up competing not only with those who applied for the same idea, but also with those who applied for other ideas. For many other participating organizations that may be the norm, but this level of competition was a bit new for us.

So why is the quality of applications so much better this year? While of course I can’t be sure, the fact that we spelled out our expectations more unequivocally on the ideas page probably helped. We asked students to bounce their ideas and draft applications off our mentor mailing list before they submit, and we worked with all students who wanted to be worked with to improve their applications.

Another modification we made was to establish an IRC channel (#phylosoc on Freenode) and specifically encourage students and mentors to take advantage of it even before applying. Notably, or should I say sadly, my feeling is that in the end the students used it much more than the mentors did (with one notable exception – thanks Brian for all your help!).

That said, it was interesting to see that there really needs to be both – some of our students were very communicative and comfortable over email and some were over IRC, but almost none on both. One of the students who made it to acceptance probably did so to a significant extent due to how he presented himself in IRC, but the other 4 communicated mostly or exclusively over email – so it’s an option more than a requirement, and an advantage primarily for those who communicate more effectively in direct back-and-forth exchanges than by carefully composing what they want to say. 

Back to the higher quality level and low fraction of spam applications. It turns out that it also seems to be a general trend as many other organizations report the same, so it’s not even clear how much our own working with the students is to receive credit for this. On the other hand, many organizations report applying increasing rigor to the application process, in fact requiring actual code contributions in the form of patches as a prerequisite for applying seems to enjoy increasing popularity.

So maybe what we’re seeing is that the bar has been raised continuously for new students, and that word of this and of the competitiveness of Summer of Code has spread widely meanwhile. 

Initially the high quality of so many applications looked like a Good Thing™. However, since the last week I’m no longer so sure I like the result. With this level of competition it’s difficult to see how less experienced students would have a chance to get in. Along this line, we didn’t receive this year what I like to call an empowerment-type application, where we could envision someone going all the way from a user to a fully capable contributor, so a repeat of Erica’s story from last year is unlikely. 

I’m also wondering whether some of this is self-inflicted. We’ve had a few inquiries from biology students with very little or no programming experience who wanted to get into writing code, but we really only had one project idea (phyloXML support in BioPerl) that could be adjusted well enough to a beginners level and still be meaningful. None of those inquiries yielded an application. Were there too few options to pick from that seemed attractive? Did the expectations for a strong application seem too daunting? Aside from that, perhaps not surprisingly that project idea also turned out to be one of the most popular ones, and so I’m not sure a truly inexperienced student would have been able to successfully compete.

Other than ensuring that we sail an entirely smooth course over the summer to hopefully be able to snag more slots next year, I’m not sure what to do about that, though. Suggestions or ideas anyone?

This entry was posted on April 25, 2008 at 12:27 am and is filed under Open Source, Phyloinformatics, Summer of Code with tags GSoC, Open Source, phylosoc. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.