Wednesday, December 4, 2013

GSoc Lessons: Part Deux: The Arms Race

This post title might be a little excessive, but I'll blame The Sum of All Fears that I was watching last night. This is the second part of a set of posts about ideas I heard at the Google Summer of Code 2013 Mentor Summit (you can read the first part about the application process).

This will explore an interesting anecdote I heard about the interaction between applicants from another organization that, on reflection, seemed to resonate somewhat with what I had seen in my corner of the Mozilla community.

The organization these students were applying to required patches to be fixed for a student's application to be accepted (as discussed in my previous post). For a particular project there existed multiple highly motivated and skilled students, but only one slot. Thus, a "patch race" of sorts occurred where the students competed by continually providing more patches that were increasingly complex. (Note that this wasn't a in response to a challenge from community members, it was a spontaneous situation.) Once a single student started to submit extra patches the other students felt they must also submit more patches to be considered equal/superior (hence my allusion to an "arms race"). Interestingly, they would also sometimes work on the same bug in a sort of race to see who could fix it first.

There's a couple things I took away from this:
  1. Great, the project just had a lot of things fixed!
  2. The students were investing escalating amounts of time during the application phase.
  3. The students were not working in an open manner.
I won't really expand much more about the first point, it's always good to fix things.

Although submitting patches might showcase a student's skill, it also relates to how much time the student is willing and able to put into the application period. This, in particular, matters since different areas of the world end their school year at different times. A student that has already finished his semester during the application period may have a lot of free time to attempt to get a GSoC slot (but will most likely not have as much time during the actual summer!) This something that mentors should keep in mind while reviewing applications.

A downside of increasing amounts of time invested is that the rejection is that much harder for both the mentor (especially if the student is now part of the community!), as well as for the student who has now vested a large amount of time in the project.

The realization that actually upset me, however, is that these students were not working in an open manner! Instead of collaborating, they were competing! To me, this would set off a very poor tone for the rest of GSoC. In fact, one of the biggest challenges I've had with GSoC students is getting them to work in the open (i.e. "show me the code", anyone in #instantbird is probably tired of hearing me say that).

At this point you might think this is a hypothetical case I made up! Upon letting it sink in and reflecting on it...I realized I had actually seen similar situations during the application periods I've been involved with. This year, we found a bug in Instantbird's IRC code (CTCP quoting and dequoting); after referencing some specifications, I was pretty quickly able to figure out the vague areas where people should look for a fix. A couple of GSoC students in the room started looking into it and exhibited a greatly reduced form of the behavior I discussed above. The students were sharing information, but were not comfortable sharing code. Unfortunately, this led to some very vague questions which I was unable to answer (or answered incorrectly) and led to me coining my catchphrase from above.

I by no means think this reflects poorly on our students! I think this is some what natural and expected for most students unfamiliar with open development. (Extrapolating from my experiences in school...) Students generally work individually (or in small groups) on projects and are directly competing for grades (at least if the course is graded on a curve). This would foster a sense of competition as opposed to cooperation! Luckily the students working with us understood (with very little prompting, I might add!) that we'd prefer they work together and help each other. We were able to successfully fix the dequoting bug (which then caused a bug in the quoting code to be visible...sigh...).

My short take away from all this: remember that students are not yet a community and they're competing with each other until they've been accepted. (And that they're used to competing, e.g. homework and exams, not collaborating!) I don't really know whether I feel the above situation is good or bad, but it's certainly an interesting effect from the way the GSoC process works.

Monday, December 2, 2013

GSoC Lessons: Part 1: Application Period

I briefly talked about my experiences at the Google Summer of Code 2013 Mentor Summit. I've been pretty remiss in sharing what was actually discussed there and for that I must apologize! This will hopefully be one of a few posts about what I learned and discussed at the Summit.

The first part I'd like to talk about is the application period: welcoming students, requirements for student applications, etc. Much of what I say on here is just ideas I've heard other organizations implement (with my personal opinion on them, please don't think this represents what Mozilla is suggesting students do, or even what I'm suggesting Mozilla should ask students to do!)

I had many separate conversations about what is required for an application to be accepted. It seems that Mozilla is actually on the side of one of the easier organizations to apply to. We don't (to my knowledge) require that students have contributed at all to the community beforehand. It is possible that some smaller communities inside of Mozilla require more than just an application, but there does not seem to be any rule across Mozilla. I said I wouldn't offer my opinion above...but I lied: I think Mozilla should make it clearer to applicants what is expected of them before the application.

There seem to be a variety of things different organizations "require" before accepting a student application, for example:
  • A patch / pull request
  • IRC / email involvement / idling
  • File a bug (I mean this in the "Mozilla" sense: an actual bug, a feature request, etc.)
  • Fix a bug / make a commit
I think all of these have pros and cons and making any a hard and fast rule would probably be a bad idea. Personally for Instantbird, we greatly encourage students to idle on IRC and get to know us; and to fix a minor bug or two or three. What I'm always looking for is: use, passion, and skill.

Asking for a patch / pull request (I include these together since they really just depend on how an organization accepts changes) can be a bit intimidating for a new user. I think this can be a pretty rough thing to ask for new contributors that might not want to share their work publicly with a large group of people (on a mailing list, public bug tracker, etc.) where they might be wrong. Even after being part of the community, I find that GSoC students are often very unwilling to publicly share code unless it's "perfect", but I digress. Anyway, if you're considering "requiring" this, I think it should be pretty clear that this changeset doesn't need to be perfect, it just needs to show that the student is able to read code, understand a bug report, provide a fix and test it.

I think it's perfect reasonable to ask students to idle on IRC and join mailing lists. They should definitely be trying to understand the community before attempting to join it. It isn't just a matter of if the community thinks the student would be a good fit, but also the student must ensure they can fit into the community.

Filing a bug is a great way for a student to show a few different things: they've used your software; they've used your software enough to find a bug in it (and there most likely is one!); they're able to express themselves in a clear and concise matter. If you're lucky they'll find something that actually annoys them and fix it themselves!

I have fix a bug listed last. You might ask how this differs from submitting a patch...and it does! Fixing a bug requires a patch to go through whatever review process your project uses, but builds upon just submitting a patch. My thoughts on this are pretty similar to just submitting a patch, but it depends on how large the bug is.

Something I found interesting is that almost everyone I talked to didn't treat their GSoC students any differently than they would treat a new contributor to their project. They still had to prove they were worthy of commit access, etc. Is there anything else you ask of your students before they apply to GSoC? I'd love to hear it!

Some other topics I'll hopefully find some time to write about include: community lessons, and handling a failing student. The community one will be very not-GSoC focused and could apply to just trying to incorporate new contributors...but I'll include it in this series.