Mozilla sets plan to dump Firefox Add-Ons In Favor Of Chrome Extensions
In the near future, major changes are expected to be incorporated in Firefox that would require extension developers to make big changes to their current extensions to ascertain that they are operational. To ascertain easy development of extensions ...
en meer »
Back in September, the TaskCluster Platform team held a workweek in Berlin to discuss upcoming feature development, focus on platform stability and monitoring and plan for the coming quarter’s work related to Release Engineering and supporting Firefox Release. These posts are documenting the many discussions we had there.
Jonas kicked off our workweek with a brief look back on the previous year of development.Prototype to Production
In the last year, TaskCluster went from an idea with a few tasks running to running all of FirefoxOS aka B2G continuous integration, which is about 40 tasks per minute in the current environment.
Architecture-wise, not a lot of major changes were made. We went from CloudAMQP to Pulse (in-house RabbitMQ). And shortly, Pulse itself be moving it’s backend to CloudAMQP! We introduced task statuses, and then simplified them.
On the implementation side, however, a lot changed. We added many features and addressed a ton of docker worker bugs. We killed Postgres and added Azure Table Storage. We rewrote the provisioner almost entirely, and moved to ES6. We learned a lot about babel-node.
We introduced the first alternative to the Docker worker, the Generic worker. We for the first time had Release Engineering create a worker, the Buildbot Bridge.
We have several new users of TaskCluster! Brian Anderson from Rust created a system for testing all Cargo packages for breakage against release versions. We’ve had a number of external contributors create builds for FirefoxOS devices. We’ve had a few Github-based projects jump on taskcluster-github.Features that go beyond BuildBot
One of the goals of creating TaskCluster was to not just get feature parity, but go beyond and support exciting, transformative features to make developer use of the CI system easier and fun.
Some of the features include:
- Interactive sessions
- Live logging (mentioned in our createArtifact() docs and visible in the task-inspector for a task)
- Public-first task statuses
- Easy Indexing
- Storage in S3 (see createArtifact() documentation)
- Public first, reference-style APIs
- Support for remote device lab workers
Release is a special use case that we need to support in order to take on Firefox production worload. The focus of development work in Q4 and beyond includes:
- Secrets handling to support Release and ops workflows. In Q4, we should see secrets.taskcluster.net go into production and UI for roles-based management.
- Scheduling support for coalescing, SETA and cache locality. In Q4, we’re focusing on an external data solution to support coalescing and SETA.
- Private data hosting. In Q4, we’ll be using a roles-based solution to support these.
Over time Mozilla has been trying to reduce the amount of time between developing a feature and getting it into a user’s hands. Some time ago we would do around one feature release of Firefox every year, more recently we’ve moved to doing one feature release every six weeks. But it still takes at least 12 weeks for a feature to get to users. In some cases we can speed that up by landing new things directly on the beta/aurora branches but the more we do this the harder it is for release managers to track the risk of shipping a given release.
The Go Faster project is investigating ways that we can speed up getting changes to users. System add-ons are one piece of this that will let us deliver updates to core Firefox features more often than the regular six week releases. Instead of being embedded in the rest of the code certain features will be developed as standalone system add-ons.
Building features as add-ons gives us more flexibility in how we deliver the features to users. System add-ons will ship in two different ways. First every Firefox release will include a default set of system add-ons. These are the latest versions of the features at the time the Firefox build was produced. Later during runtime Firefox will contact Mozilla’s update servers to ask for the current list of system add-ons. If there are new or updated versions listed Firefox will download and install them giving users access to the newest features without needing to update the entire application.
Building a feature as an add-on gives developers a lot of benefits too. Developers will be able to work on and test new features without doing custom Firefox builds. Users can even try out new features by just installing the add-ons. Once the feature is ready to ship it ships as an add-on with no code changes necessary for integration into Firefox. This is something we’ve attempted to do before with things like Test Pilot and pdf.js, but system add-ons make this process much smoother and reduces the differences between how the feature runs as an add-on and how it runs when shipped in the application.
The basic support for system add-ons is already included in current nightly builds and Firefox 44 should be the first release that we could use to deliver features like this if we choose. If you’re interested in the details you can read the client implementation plan or follow along the tracking bug for the client side of the feature.
Welcome to TaskCluster Platform’s 2015Q3 Retrospective! I’ve been managing this team this quarter and thought it would be nice to look back on what we’ve done. This report covers what we did for our quarterly goals. I’ve linked to “Publications” at the bottom of this page, and we have a TaskCluster Mozilla Wiki page that’s worth checking out.High level accomplishments
- Dramatically improved stability of TaskCluster Platform for Sheriffs by fixing TreeHerder ingestion logic and regexes, adding better logging and fixing bugs in our taskcluster-vcs and mozilla-taskcluster components
- Created and Deployed CI builds on three major platforms:
- Added Linux64 (CentOS), Mac OS X cross-compiled builds as Tier2 CI builds
- Completed and documented a prototype Windows 2012 builds in AWS and task configuration
- Deployed auth.taskcluster.net, enabling better security, better support for self-service authorization and easier contributions from outside our team
- Added region biasing based on cost and availability of spot instances to our AWS provisioner
- Managed the workload of two interns, and significantly mentored a third
- Onboarded Selena as a new manager
- Held a workweek to focus attention on bringing our environment into production support of Release Engineering
We laid out our Q3 goals in this etherpad. Our chosen themes this quarter were:
- Improve operational excellence — focus on sheriff concerns, data collection,
- Facilitate self-serve consumption — refactoring auth and supporting roles for scopes, and
- Exploit opportunities to differentiate from other platforms — support for interactive sessions, docker images as artifacts, github integration and more blogging/docs.
We had 139 Resolved FIXED bugs in TaskCluster product.
We also resolved 7 bugs in FirefoxOS, TreeHerder and RelEng products/components.
We received significant contributions from other teams: Morgan (mrrrgn) designed, created and deployed taskcluster-github; Ted deployed Mac OS X cross compiled builds; Dustin reworked the Linux TC builds to use CentOS, and resolved 11 bugs related to TaskCluster and Linux builds.
An additional 9 people contributed code to core TaskCluster, intree build scripts and and task definitions: aus, rwood, rail, mshal, gerard-majax, firstname.lastname@example.org, htsai, cmanchester, and echen.The Big Picture: TaskCluster integration into Platform Operations
Moving from B2G to Platform was a big shift. The team had already made a goal of enabling Firefox Release builds, but it wasn’t entirely clear how to accomplish that. We spent a lot of this quarter learning things from RelEng and prioritizing. The whole team spent the majority of our time supporting others use of TaskCluster through training and support, developing task configurations and resolving infrastructure problems. At the same time, we shipped docker-worker features, provisioner biasing and a new authorization system. One tricky infra issue that John and Jonas worked on early in the quarter was a strange AWS Provisioner failure that came down to an obscure missing dependency. We had a few git-related tree closures that Greg worked closely on and ultimately committed fixes to taskcluster-vcs to help resolve. Everyone spent a lot of time responding to bugs filed by the sheriffs and requests for help on IRC.
It’s hard to overstate how important the Sheriff relationship and TreeHerder work was. A couple teams had the impression that TaskCluster itself was unstable. Fixing this was a joint effort across TreeHerder, Sheriffs and TaskCluster teams.
When we finished, useful errors were finally being reported by tasks and starring became much more specific and actionable. We may have received a partial compliment on this from philor. The extent of artifact upload retries, for example, was made much clearer and we’ve prioritized fixing this in early Q4.
Both Greg and Jonas spent many weeks meeting with Ed and Cam, designing systems, fixing issues in TaskCluster components and contributing code back to TreeHerder. These meetings also led to Jonas and Cam collaborating more on API and data design, and this work is ongoing.
We had our own “intern” who was hired on as a contractor for the summer, Edgar Chen. He did some work with the docker-worker, implementing Interactive Sessions, and did analysis on our provisioner/worker efficiency. We made him give a short, sweet presentation on the interactive sessions. Edgar is now at CMU for his sophomore year and has referred at least one friend back to Mozilla to apply for an internship next summer.
Pete completed a Windows 2012 prototype build of Firefox that’s available from Try, with documentation and a completely automated process for creating AMIs. He hasn’t created a narrated video with dueling, British-English accented robot voices for this build yet.
We also invested a great deal of time in the RelEng interns. Jonas and Greg worked with Anhad on getting him productive with TaskCluster. When Anthony arrived, we also onboarded him. Jonas worked closely to get him working on a new project, hooks.taskcluster.net. To take these two bits of work from RelEng on, I pushed TaskCluster’s roadmap for generic-worker features back a quarter and Jonas pushed his stretch goal of getting the big graph scheduler into production to Q4.
We worked a great deal with other teams this quarter on taskcluster-github, supporting new Firefox and B2G builds, RRAs for the workers and generally telling Mozilla about TaskCluster.
Finally, we spent a significant amount of time interviewing, and then creating a more formal interview process that includes a coding challenge and structured-interview type questions. This is still in flux, but the first two portions are being used and refined currently. Jonas, Greg and Pete spent many hours interviewing candidates.Berlin Work Week
Toward the end of the quarter, we held a workweek in Berlin to focus our next round of work on critical RelEng and Release-specific features as well as production monitoring planning. Dustin surprised us with delightful laser cut acrylic versions of the TaskCluster logo for the team! All team members reported that they benefited from being in one room to discuss key designs, get immediate code review, and demonstrate work in progress.
We came out of this with 20+ detailed documents from our conversations, greater alignment on the priorities for Platform Operations and a plan for trainings and tutorials to give at Orlando. Dustin followed this up with a series of ‘TC Topics’ Vidyo sessions targeted mostly at RelEng.
Our Q4 roadmap is focused on key RelEng features to support Release.Publications
Our team published a few blog posts and videos this quarter:
- TaskCluster YouTube channel with two generic worker videos
- On Planet Taskcluster:
- On Air Mozilla
- Interactive Sessions (Edgar Chen)
- TaskCluster GitHub, Continuous integration for Mozillians by Mozillians (mrrrgn)
La llegada de la última versión de Windows causó mucho revuelo entre los usuarios al ver que su navegador predeterminado había sido cambiado. Por su parte, Mozilla reaccionó y su CEO Chris Beard le envió una carta a su similar de Microsoft Satya Nadella pidiendo que no retrocedieran en la elección y control de los usuarios.
Las versiones actuales del panda rojo deberían cambiar esto, pero si por alguna razón no lo hace y te gustaría recuperar Firefox u otro navegador como predeterminado, te recomiendo que sigas estos pasos:
- Haz clic en el botón de menú , después selecciona Opciones.
- En el panel General, haz clic en Convertir en predeterminado.
- La aplicación de Ajustes de Windows abrirá la pantalla de Selecciona programas predeterminados.
- Desplázate hacia abajo y haz clic en la entrada de Explorador web. En este caso, el icono mostrará Microsoft Edge o bien Selecciona tu navegador predeterminado.
- En la pantalla de Elegir una aplicación, haz clic en Firefox para establecerlo como el navegador predeterminado.
- Firefox ahora aparece como tu navegador predeterminado. Cierra la ventana para guardar tus cambios.
Y listo, ya tendrás Firefox de vuelta como tu navegador predeterminado y preferido.
Fuente: Mozilla Support
In the last week, we landed 69 PRs in the Servo repository!
Glenn wrote a short report on how webrender is coming along. Webrender is a new renderer for Servo which is specialized for web content. The initial results are quite promising!Notable additions
- Patrick and Corey reduced allocator churn in our DOMstring code and string joining code
- Josh restyled <select> so that <select multiple> works.
- Vladimir’s Windows work continues
- Corey implemented <font size>
- Martin simplified our stacking context creation code
Snazzy new form widgets:
I was invited as one of the speakers at the ApacheCon core conference in Budapest, Hungary on October 1-2, 2015.
I was once again spreading the news about HTTP/2, why it was made and how it works and of course: updated numbers on adoption right now.
The talk was unfortunately not filmed, but I’ve put my slides for this version of my talk online. Readers of this blog and those who’ve seen my presentations before will recognize large parts of it.
Following my talk was talks about mod_http2, the Apache module for HTTP/2 that will be coming in the upcoming 2.4.17 release of Apache Httpd, explained by its author Stefan Eissing. The name of the module was actually a bit of a surprise to me since it has been known as just mod_h2 for its entire life time up until now.
William A Rowe took us through the state of TLS for the main Apache servers and yeah, the state seem to be pretty good and they’re coming along really well. TLS and then HTTPS is important as that’s really a prerequisite for HTTP/2
I also got to listen to Mark Thomas explain the agonies of making Tomcat support HTTP/2, and then perhaps especially how ALPN and a good set of ciphers are hard to get in Java.
Jean-Frederic Clere then explained how to activate HTTP/2 on all the Apache servers (tomcat, httpd and traffic server) and a little about their HTTP/2 state, following with an explanation how they worked on tomcat to make that use OpenSSL for the TLS layer (including ALPN) to avoid the deadlock of decent TLS support in Java.
All in all, a great track and splendid talks with deep technical content. Exactly the way I like it. Thanks everyone. Apachecon certainly delivered for me! Twas fun.
Valgrind’s Memcheck tool works on Linux and MacOS, but not on Windows. Interestingly, there is something like it for Windows: “Dr Memory”. Similar in style to Memcheck, Dr Memory is an open source memory checking tool built on top of a JIT-based instrumentation framework called DynamoRIO. It provides essentially identical functionality: detection of invalid memory accesses, uninitialised value uses and memory leaks. Dr Memory claims to be considerably faster than Memcheck, so I was curious to see how it performed.
I recently tried Dr Memory 1.9.0-RC1 on Windows 7, running 32-bit Firefox builds, to see to what extent it can provide coverage for the Windows-specific parts of Gecko.
Installing and getting started isn’t difficult. There are command line flags to direct the output, control the level of instrumentation, specify files listing errors to hide, and so on. As you’d expect.
Despite considerable efforts with Dr Memory, I came away feeling it was a promising tool, but just a bit too hard to use. I encountered two kinds of problems.
Firstly, about half of my Firefox startups ended up spinning. Some of the time, Firefox would start (slowly, of course) and be usable after a couple of minutes. Other runs would spin for an hour or more and still not produce a usable browser. I never figured out why. This seems to be related to the instrumentation, because if I run Firefox uninstrumented on the DynamoRIO core, like Valgrind’s –tool=none, it works reliably.
A second problem was the considerable number of uninitialised memory read errors. I tried out both non-optimised (“/Zi /Od”) and optimised (“/Zi /O2 /Oy- /Ob0”) builds of Firefox.
For the non-optimised builds, Dr Memory reports no invalid accesses and a few uninitialised memory reads, which is what I’d expect. But it’s unusably slow, because the unoptimised build lacks reasonable register allocation, which easily doubles the number of memory accesses that have to be checked.
So my next step was to try an optimised build. This runs a great deal faster. There’s a down side, though: the number of uninitialised memory accesses goes way up. Most of these must be false positives, because they weren’t reported in the unoptimised runs.
I investigated further. It is likely that one source of false positives is Dr Memory’s incomplete description of the Windows system call interface. Valgrind’s description of the Linux syscall interface is itself complex, and it is said that the Windows interface makes the Linux interface look simple. Given that, I’m impressed that Dr Memory works as well as it does.
The other source of false positives appears to be bitfields. Dr Memory tracks the definedness state of each byte of memory using one bit for each byte. Consequently it has no way to accurately model partially initialised bytes, and so must unavoidably either report false positives, or miss real errors, depending on which of the two available shadow states partially initialised bytes are mapped to.
One way to detect probable false-positive bitfield errors in cross platform Gecko code is to check whether Memcheck reports errors at the same places. In many cases it doesn’t. I created a suppressions file, which tells Dr Memory to hide errors I identified as clearly false. A second line of defense is to add extra initialisation code for bitfields purely in order to keep Dr Memory happy. Neither of these are really what one wants to do, though.
The false positive problem seriously compromises Dr Memory’s usefulness on optimised Gecko code, compared to Memcheck. The effect is to create a lot more undefined value errors needing investigation. The situation is exacerbated because Dr Memory doesn’t have an equivalent to Memcheck’s origin-tracking feature, which makes it more difficult to analyse the errors and to determine where, if any, dummy initialisations should be placed.
Dr Memory does have a “light” mode, which restricts it to invalid-address and leak checking only. This increases usability at the expense of losing undefined value checking. If you’re looking for possible heap corruption on Windows, this would be worth a try.
As you may already know, last Friday – October 2nd – we held a new Testday event, for Firefox 42 Beta 3.
I must admit that this testday is by far one of the most successful event. Besides the fact that we had a large number of participants we also have an impressive number of verified bugs. Congratulations to all participants!
- no new issues were found while testing Tab visual sound indicator
- 3 potential issues were logged for Control Center – 1211064 1211073, 1211074 (Thanks PreethiDhinesh for finding it!)
- 26 bugs were verified: 1183044, 1179031, 486262, 1182769, 1047713, 1184170, 1184184, 1184243, 1180734, 1196313, 1183079, 1135812, 1185953, 1185960, 1101100, 1031661, 1149745, 1089240, 1196437, 1196947, 1209030, 1205100, 1180387, 1058251, 1181253, 1111555
- 7 bugs were triaged: 1209030, 1178781, 1210354, 1183156, 1182435, 1184916, 1185447
We’d like to take this opportunity to thank Bolaram Paul, Mohammed Adam , Ionce Stelian, Ruwan Ranganath, Jayesh KR, Arshad Abid, Moin Shaikh, Syed Muhammad Mahmudul Haque (Yamin), Nischaytv, Jyotsna Gupta, PreethiDhinesh, Kevin Le, Gunjan Tank and the people from our Bangladesh Community: Hossain Al Ikram, Khalid Syfullah Zaman, Ashickur Rahman, Md. Asiful Kabir, Rezaul Huque Nayeem, Kazi Nuzhat Tasnem, Nazir Ahmed Sabbir, Saheda Reza Antora, Md.Ehsanul Hassan, Mohammad Maruf Islam, Sayed Mohammad Amir, Meraj Kazi, Forhad Hossain, T.M. Sazzad Hossain and Towkir Ahmed for getting involved in this event and making Firefox as best as it could be.
Also a big thank you goes to all our active moderators.
Keep an eye on QMO for upcoming events!
Wow! I’ve been a way from this blog for too long. I also haven’t posted any new episodes for The Joy of Coding. I also haven’t been keeping up with my Things I’ve Learned posts.
Time to get back in the saddle. First thing’s first, here are 6 episodes of The Joy of Coding that have aired. Unfortunately, I haven’t put together summaries for any of them, but I’ve put their agendas near the videos so that might give some clues.
Here we go!Episode 23
Mozilla-Petition gegen Vorratsdatenspeicherung
„Angesichts der jüngsten Vorwürfe, die Bundesregierung habe Informationen über ihre Bürger mit den USA geteilt, ist es wichtiger denn je, die Datenspeicherung einzuschränken“ erläutert Mozilla. „Doch neue Gesetze zur Vorratsdatenspeicherung, wie sie ...
SHA-1 certificates are on their way out, and you should upgrade to a SHA-256 certificate as soon as possible... unless you have very old clients and must maintain SHA-1 compatibility for a while.
If youare in this situation, you need to either force your clients to upgrade (difficult) or implement some form of certificate selection logic: we call that "cert switching".
The most deterministic selection method is to serve SHA-256 certificates to clients that present a TLS1.2 CLIENT HELLO that explicitly announces their support for SHA256-RSA (0x0401) in the signature_algorithms extension.
Modern web browsers will send this extension. However, I am not aware of any open source load balancer that is currently able to inspect the content of the signature_algorithms extension. It may come in the future, but for now the easiest way to achieve cert switching is to use HAProxy SNI ACLs: if a client presents the SNI extension, direct it to a backend that presents a SHA-256 certificate. If it doesn't present the extension, assume that it's an old client that speaks SSLv3 or some broken version of TLS, and present it a SHA-1 cert.
This can be achieved in HAProxy by chaining frontend and backends:
The configuration above receives inbound traffic in the frontend called "https-in". That frontend is in TCP mode and inspects the CLIENT HELLO coming from the client for the value of the SNI extension. If that value exists and matches our target site, it sends the connection to the backend named "jve_https", which redirects to a frontend also named "jve_https" where the SHA256 certificate is configured and served to the client.
If the client fails to present a CLIENT HELLO with SNI, or presents a SNI that doesn't match our target site, it is redirected to the "https_jve_sha1" backend, then to its corresponding frontend where a SHA1 certificate is served. That frontend also supports an older ciphersuite to accommodate older clients.
Both frontends eventually redirect to a single backend named "jve" which sends traffic to the destination web servers.
This is a very simple configuration, and eventually it could be improved using better ACLs (HAproxy regularly adds news ones), but for a basic cert switching configuration, it gets the job done!
Como parte de la XVI Convención y Feria Internacional Informática Habana 2016 a realizarse en nuestra capital a partir del 14 y hasta el 18 de marzo de 2016, se desarrollará el VI Taller Internacional de Tecnologías de Software Libre y Código Abierto, un evento donde las personas podrán debatir en torno a diferentes temáticas como por ejemplo: la adopción de tecnologías de software libre y código abierto, el desarrollo y personalización de sistemas operativos, y aspectos económicos, legales y sociales.
Como parte del taller de software libre y código abierto en las temáticas se tratará lo siguiente:
- Adopción de tecnologías de software libre y código abiertoExperiencias en la conducción y ejecución de procesos de migración a aplicaciones de software libre y código abierto. Implantación de tecnologías libres en el sector público y privado. Modelos de madurez para la selección de tecnologías libres. Rol del software libre en el desarrollo sostenible.
- Construcción / personalización de sistemas operativos basados en fuentes abiertasDesarrollo de sistemas operativos libres a la medida para computadoras de escritorio, servidores y teléfonos móviles. Software libre embebido. Construcción de distribuciones GNU/Linux.
- Aspectos económicos, legales y socialesEstudios de viabilidad económica para la adopción de software libre. Modelos de negocios aplicables. Impacto social del software libre. Propiedad intelectual, derecho de autor y licenciamiento de tecnologías libres. Marco regulativo para el empleo de software libre.
- Tecnologías de software libreSoftware libre para la nube, seguridad en tecnologías libres. Estándares abiertos en el desarrollo de soluciones. OpenData. OpenCloud. Tecnologías de gestión de datos en código abierto. Software libre como servicio, aplicaciones empresariales de fuentes abiertas. Tecnologías móviles de código abierto.
Desde la página web del evento también se puede observar varios datos útiles para las personas que deseen participar entre los que destacan:
- Presentación de resúmenes y ponencias: 20 de octubre de 2015
- Notificación de aceptación: 20 de noviembre de 2015
- Envío del trabajo final para su publicación: 7 de diciembre de 2015
Pautas de Redacción
Los trabajos se presentarán a partir de las temáticas principales de los distintos eventos a realizarse en el marco de la Convención, teniendo en cuenta las siguientes especificaciones:
- Deben entregarse en ficheros compatibles con formato de documento abierto
- Límite de 10 hojas
- El tamaño de la hoja será tipo carta (8,5” x 11” ó 21,59 cm. x 27,94 cm.), con márgenes de 2 cm. por cada lado y escrito a dos columnas
- Deberá utilizarse tipografía Arial a 11 puntos para los encabezados y a 10 puntos para los textos, con un interlineado sencillo
- Se redactará en los idiomas del evento (español o inglés)
Estructura de los trabajos:
- Título (en inglés)
- Autor y coautores
- Afiliación y datos de contactos
- Resumen y Palabras Claves (en español y en inglés)
- Agradecimientos (opcional)
- Referencias bibliográficas
Usted deberá enviar su trabajo a través de la Plataforma para la Gestión de Ponencias, recuerde que debe adjuntar además una síntesis curricular del autor principal. Las presentaciones de los trabajos a exponer en el evento presencial deben ser entregados por los ponentes en la Oficina de Recepción de Medios Audiovisuales, en el Palacio de Convenciones, un día antes de la exposición en sala. Las conferencias, presentaciones y otros materiales de la Convención se publicarán en un disco compacto con su registro ISBN.
tl;dr: I want to liberate people; software is a (critical) tool to that end. There is a conference this weekend that understands that, but I worry it isn’t FSF’s.
This morning, social network chatter reminded me of FSF‘s 30th birthday celebration. These travel messages were from friends who I have a great deal of love and respect for, and represent a movement to which I essentially owe my adult life.
Despite that, I had lots of mixed feels about the event. I had a hard time capturing why, though.
While I was still processing these feelings, late tonight, Twitter reminded me of a new conference also going on this weekend, appropriately called Open Source and Feelings. (I badly wanted to submit a talk for it, but a prior commitment kept me from both it and FSF@30.)
I saw the OSFeels agenda for the first time tonight. It includes:
- Design and empathy (learning to build open software that empowers all users, not just the technically sophisticated)
- Inclusive development (multiple talks about this, including non-English, family, and people of color) (so that the whole planet can access, and participate in developing, open software)
- Documentation (so that users understand open software)
- Communications skills (so that people feel welcome and engaged to help develop open software)
This is an agenda focused on liberating human beings by developing software that serves their needs, and engaging them in the creation of that software. That is incredibly exciting. I’ve long thought (following Sen and Nussbaum’s capability approach) that it is not sufficient to free people; they must be empowered to actually enjoy the benefits of that freedom. This is a conference that seems to get that, and I can’t wait to go (and hopefully speak!) next year.
The Free Software Foundation event’s agenda:
- boot firmware
These are important topics. But there is clearly a difference in focus here — technology first, not people. No mention of community, or of design.
This difference in focus is where this morning’s conflicted feels came from. On the one hand, I support FSF, because they’ve done an incredible amount to make the world a better place. (OSFeels can take open development for granted precisely because FSF fought so many battles about source code.) But precisely because I support FSF, I’d challenge it, in the next 15 years, to become more clearly and forcefully dedicated to liberating people. In this world, FSF would talk about design, accessibility, and inclusion as much as licensing, and talk about community-building protocols as much as communication protocols. This is not impossible: LibrePlanet had at least some people-focused talks (e.g.), and inclusion and accessibility are a genuine concern of staff, even if they didn’t rise to today’s agenda. But it would still be a big change, because at the deepest level, it would require FSF to see source code as just one of many requirements for freedom, rather than “the point of free software“.
At the same time, OSFeels is clearly filled with people who see the world through a broad, thoughtful ethical lens. It is a sad sign, both for FSF and how it is perceived, that such a group uses the deliberately apolitical language of openness rather than the language of a (hopefully) aligned ethical movement — free software. I’ll look forward to the day (maybe FSF’s 45th (or 31st!) birthday) that both groups can speak and work together about their real shared concern: software that liberates people. I’d certainly have no conflicted feelings about signing up for a conference on that :)
At least once in your career as a programmer, and hopefully more than once and with deliberate regularity, it is important to leave the comfort of your usual place along the stack and travel up or down it. While you usually fix bugs and add features using a particular application, tool, or API and work on top of some platform, SDK, or operating system, this time you choose to climb down a wrung and work below. In doing so you are working on instead of with something.
There are a number of important outcomes of changing levels. First, what you previously took for granted, and simply used as such, suddenly comes into view as a thing unto itself. This layer that you've been standing on, the one that felt so secure...it turns out to have also have been built, just like the things you build from above! It sounds obvious, but in my experience, the effect it has on your approach from this point forward is drastically changed. Second, and very much related to the first, your likelihood to lash out when you encounter bugs or performance and implementation issues gets abated. You gain empathy and understanding.
If all you ever do is use an implementation, tool, or API, and never build or maintain one, it's easy to take them for granted, and speak about them in detached ways: Why would anyone do it this way? Why is this so slow? Why won't they fix this bug after 12 years! And then, Why are they so stupid as to have done this thing?
Now, the same works in the other direction, too. Even though there are more people operating at the higher level and therefore need to descend to do what I'm talking about, those underneath must also venture above ground. If all you've ever done is implement things, and never gone and used such implementations to build real things, you're just as guilty, and equally, if not more likely to snipe and complain about people above you, who clearly don't understand how things really work. It's tantalizingly easy to dismiss people who haven't worked at your level: it's absolutely true that most of them don't understand your work or point of view. The way around this problem is not to wait and hope that they will come to understand you, but to go yourself, and understand them.
Twitter, HN, reddit, etc. are full of people at both levels making generalizations, lobbing frustration and anger at one another, and assuming that their level is the only one that actually matters (or exists). Fixing this problem globally will never happen; but you can do something at the personal level.
None of us enjoys looking foolish or revealing our own ignorance. And one of the best ways to avoid both is to only work on what we know. What I'm suggesting is that you purposefully move up or down the stack and work on code, and with tools, people, and processes that you don't know. I'm suggesting that you become a Fool, at least in so far as you allow yourself to be humbled by this other world, with its new terminology, constraints, and problems. By doing this you will find that your ability to so easily dismiss the problems of the other level will be greatly reduced. Their problems will become your problems, and their concerns your concerns. You will know that you've correctly done what I'm suggesting when you start noticing yourself referring to "our bug" and "how we do this" instead of "their" and "they."
Becoming a Fool Stack Programmer is not about becoming an expert at every level of the stack. Rather, its goal is to erase the boundary between the levels such that you can reach up or down in order to offer help, even if that help is only to offer a kind word of encouragement when the problem is particularly hard: these are our problems, after all.
I'm grateful to this guy who first taught me this lesson, and encouraged me to always keep moving up the stack.
This week I migrated this blog from Wordpress to Jekyll, a popular static site generator. This post explains why and how I did this, maybe it will be useful to someone.Why?
Wordpress powers thousands of websites, is regularly updated, has a ton of features. Why did I abandon it?
I still think Wordpress is great for a lot of websites and blogs, but I felt it was overkill for my simple website. It had so many features I never used and this came at a price: it was hard to understand how everything worked, it was hard to make changes and it required regular security updates.
This is what I like most about Jekyll compared to Wordpress:
- Maintainance, security: I don't blog often, yet I still had to update Wordpress every few weeks or months. Even though the process is pretty straight-forward, it got cumbersome after a while.
- Setup: Setting up a local Wordpress instance with the same content and configuration was annoying. I never bothered so the little development I did was directly on the webserver. This didn't feel very good or safe. Now I just have to install Jekyll, clone my repository and generate plain HTML files. No database to setup. No webserver to install (Jekyll comes with a little webserver, see below).
- Transparency: With Wordpress, the blog posts were stored somewhere in a MySQL database. With Jekyll, I have Markdown files in a Git repository. This makes it trivial to backup, view diffs, etc.
- Customizability: After I started using Jekyll, customizing this blog (see below) was very straight-forward. It took me less than a few hours. With Wordpress I'm sure it'd have taken longer and I'd have introduced a few security bugs in the process.
- Performance: The website is just some static HTML files, so it's fast. Also, when writing a blog post, I like to preview it after writing a paragraph or so. With Wordpress it was always a bit tedious to wait for the website to save the blog post and reload the page. With Jekyll, I save the markdown file in my text editor and, in the background, jekyll serve immediately updates the site, so I can just refresh the page in the browser. Everything runs locally.
- Hosting: In the future I may move this blog to GitHub Pages or another free/cheaper host.
I went with Jekyll because it's widely used, so there's a lot of documentation and it'll likely still be around in a year or two. Octopress is also popular but under the hood it's just Jekyll with some plugins and changes, and it seems to be updated less frequently.How?
I decided to use the default template and customize it where needed. I made the following changes:
- Links to previous/next post at the end of each post, see post.html
- Pagination on the homepage, based on the docs. I also changed the home page to include the contents instead of just the post title.
- Archive page, a list of posts grouped by year, see archive.html
- Category pages. I wrote a small plugin to generate a page + feed for each category. This is based on the example in the plugin documentation. See _plugins/category-generator.rb and _layouts/category.html
- List of categories in the header of each post (with a link to the category page), see post.html
- Disqus comments and number of comments in the header of each post, based on the docs, see post.html. I was able to export the Wordpress comments to Disqus.
- In _config.yml I changed the post URL format ("permalink" option) to not include the category names. This way links to my posts still work.
- Some minor tweaks here and there.
I still want to change the code highlighting style, but that can wait for now.Conclusion
After using Jekyll for a few hours, I'm a big fan. It's simple, it's fun, it's powerful. If you're tired of Wordpress and Blogger, or just want to experiment with something else, I highly recommend giving it a try.
Firefox 42 will be released on November 3rd. Here’s the list of changes that went into this version that can affect add-on compatibility. There is more information available in Firefox 42 for Developers, so you should also give it a look.General
- Remove support for the prefixed mozRequestAnimationFrame.
- Convert newTab.xul to newTab.xhtml.
- Provide visual indicator as to which tab is causing sound. This required some changes to the tab XBL bindings, so it can break add-ons that change tab behavior or tab appearance.
- Badged button (like social buttons) causes havoc with overflow and panel menu. This was broken for a while, but it’s fixed now.
- Calling Map/Set/WeakMap() (without `new`) should throw.
- Split nsIContentPolicy::TYPE_SUBDOCUMENT into TYPE_FRAME and TYPE_IFRAME. This should only affect your add-on if it implements nsIContentPolicy.
- Use origin for nsIPermissionManager. Permissions in the Permission Manager used to be handled per-host, and now they will be handled per-origin. This means that https://mozilla.org and http://mozilla.org have different permission entries. This changes some methods in the Permission Manager so they accept URIs instead of strings.
- Create API for add-ons to add large header image to speed-dial home panels. This API is available for add-ons on Firefox for Android.
Please let me know in the comments if there’s anything missing or incorrect on these lists. If your add-on breaks on Firefox 42, I’d like to know.
The automatic compatibility validation and upgrade for add-ons on AMO will happen in the coming weeks, so keep an eye on your email if you have an add-on listed on our site with its compatibility set to Firefox 41.
Contribution takes many forms where each person has different reasons to contribute or help people contribute. One problem we saw a need to fix was when a new contributor came to Mozilla and picked up a “good first bug”, then upon completion was left not knowing what to do next and picking up other random bugs. The essential problem is that we had no clear path defined for someone to start making more substantial improvements to our projects. This can easily lead toward no clear mentorship as well as a lot of wasted time setting up and learning new things. In response to this, we decided to launch the Summer of Contribution program.
Back in May we announced two projects to pilot this new program: perfherder and developer experience. In the announcement we asked that interested hackers commit to dedicating 5-10 hours/week for 8 weeks to one of these projects. In return, we would act as a dedicated mentor and do our best to ensure success.
I want to outline how the program was structured, what worked well, and what we want to do differently next time.
The program worked well enough, with some improvising, here is what we started with:
- we created a set of bugs that would be good to get new contributors started and working for a few weeks
- anybody could express interest via email/irc, we envisioned taking 2-3 participants based on what we thought we could handle as mentors.
That was it, we improvised a little by doing:
- accepting more than 2-3 people to start (4-6)- we had a problem saying no
- folks got ramped up and just kept working (there was no official start date)
- blogging about who was involved and what they would be doing (intro to the perfherder team, intro to the dx team)
- setting up communication channels with contributors like etherpad, email, wunderlist, bugzilla, irc
- setting up regular meetings with contributors
- picking an end date
- summarizing the program (wlach‘s perfherder post, jmaher’s dx post)
What worked well
A lot worked very well, specifically advertising by blog post and newsgroup post and then setting the expectation of a longer contribution cycle rather than a couple weeks. Both :wlach and myself have had a good history of onboarding contributors, and feel that being patient, responding quickly, communicating effectively and regularly, and treating contributors as team members goes a long way. Onboarding is easier if you spend the time to create docs for setup (we have the ateam bootcamp). Without mentors being ready to onboard, there is no chance for making a program like this work.
Setting aside a pile of bugs to work on was successful. The first contribution is hard as there is so much time required for setup, so many tools and terms to get familiar with, and a lot of process to learn. After the first bug is completed, what comes next? Assuming it was enjoyable, one of two paths usually take place:
- Ask what is next to the person that reviewed your code or was nice to you on IRC
- Find another bug and ask to work on it
Both of these are OK models, but there is a trap where you could end up with a bug that is hard to fix, not well defined, outdated/irrelevant, or requires a lot of new learning/setup. This trap is something to avoid where we can build on the experience of the first bug and work on the same feature but on a bug that is a bit more challenging.
A few more thoughts on the predefined set of bugs to get started:
- These should not be easily discoverable as “good first bugs“, because we want people who are committed to this program to work on them, rather than people just looking for an easy way to get involved.
- They should all have a tracking bug, tag, or other method for easily seeing the entire pool of bugs
- All bugs should be important to have fixed, but they are not urgent- think about “we would like to fix this later this quarter or next quarter”. If we do not have some form of urgency around getting the bugs fixed, our eagerness to help out in mentoring and reviewing will be low. A lot of times while working on a feature there are followup bugs, those are good candidates!
- There should be an equal amount (5-10) of starter bugs, next bugs, and other bugs
- Keep in mind this is a starter list, imagine 2-3 contributors hacking on this for a month, they will be able to complete them all.
- This list can grow as the project continues
Another thing that worked is we tried to work in public channels (irc, bugzilla) as much as possible, instead of always private messaging or communicating by email. Also communicating to other team members and users of the tools that there are new team members for the next few months. This really helped the contributors see the value of the work they are doing while introducing them to a larger software team.
Blog posts were successful at communicating and helping keep things public while giving more exposure to the newer members on the team. One thing I like to do is ensure a contributor has a Mozillians profile as well as links to other discoverable things (bugzilla id, irc nick, github id, twitter, etc.) and some information about why they are participating. In addition to this, we also highlighted achievements in the fortnightly Engineering Productivity meeting and any other newsgroup postings we were doing.
Lastly I would like to point out a dedicated mentor was successful. As a contributor it is not always comfortable to ask questions, or deal with reviews from a lot of new people. Having someone to chat with every day you are hacking on the project is nice. Being a mentor doesn’t mean reviewing every line of code, but it does mean checking in on contributors regularly, ensuring bugs are not stuck waiting for needinfo/reviews, and helping set expectations of how work is to be done. In an ideal world after working on a project like this a contributor would continue on and try to work with a new mentor to grow their skills in working with others as well as different code bases.
What we can do differently next time?
A few small things are worth improving on for our next cycle, here is a few things we will plan on doing differently:
- Advertising 4-5 weeks prior and having a defined start/end date (e.g. November 20th – January 15th)
- Really limiting this to a specific number of contributors, ideally 2-3 per mentor.
- Setting acceptance criteria up front. This could be solving 2 easy bugs prior to the start date.
- Posting an announcement welcoming the new team members, posting another announcement at the halfway mark, and posting a completion announcement highlighting the great work.
- Setting up a weekly meeting schedule that includes status per person, great achievements, problems, and some kind of learning (guest speaker, Q&A, etc.). This meeting should be unique per project.
- Have a simple process for helping folks transition out of they have less time than they thought- this will happen, we need to account for it so the remaining contributors get the most out of the program.
In summary we found this to be a great experience and are looking to do another program in the near future. We named this Summer of Contribution for our first time around, but that is limiting to when it can take place and doesn’t respect the fact that the southern hemisphere is experiencing Winter during that time. With that :maja_zf suggested calling it Quarter of Contribution which we plan to announce our next iteration in the coming weeks!
Dennis is a Python command line utility (and library) for working with localization. It includes:
- a linter for finding problems in strings in .po files like invalid Python variable syntax which leads to exceptions
- a template linter for finding problems in strings in .pot files that make translator's lives difficult
- a statuser for seeing the high-level translation/error status of your .po files
- a translator for strings in your .po files to make development easier
It's been 10 months since the last release. In that time, I:
- Added a lot more tests and fixed bugs discovered with those tests.
- Added lint rule for bad format characters like %a (#68)
- Missing python-format variables is now an error (#57)
- Fix notype test to handle more cases (#63)
- Implement rule exclusion (#60)
- Rewrite --rule spec verification to work correctly (#61)
- Add --showfuzzy to status command (#64)
- Add untranslated word counts to status command (#55)
- Change Var to Format and use gettext names (#48)
- Handle the standalone } case (#56)
I thought I was close to 1.0, but now I'm less sure. I want to unify the .po and .pot linters and generalize them so that we can handle other l10n file formats. I also want to implement a proper plugin system so that it's easier to add new rules and it'd allow other people to create separate Python packages that implement rules, tokenizers and translaters. Plus I want to continue fleshing out the tests.
At the (glacial) pace I'm going at, that'll take a year or so.
If you're interested in dennis development, helping out or have things you wish it did, please let me know. Otherwise I'll just keep on keepin on at the current pace.Where to go for more
For more specifics on this release, see here: https://dennis.readthedocs.org/en/v0.7/changelog.html#version-0-7-0-october-2nd-2015
Documentation and quickstart here: https://dennis.readthedocs.org/en/v0.7/
Source code and issue tracker here: https://github.com/willkg/dennis
Source code and issue tracker for Denise (Dennis-as-a-service): https://github.com/willkg/denise
47 out of 80 Silicon Valley companies say their last round of funding depended solely on having dennis in their development pipeline and translating their business plan into Dubstep.
I've mentored a number of students in 2013, 2014 and 2015 for Debian and Ganglia and most of the companies I've worked with have run internships and graduate programs from time to time. GSoC 2015 has just finished and with all the excitement, many students are already asking what they can do to prepare and be selected for Outreachy or GSoC in 2016.
My own observation is that the more time the organization has to get to know the student, the more confident they can be selecting that student. Furthermore, the more time that the student has spent getting to know the free software community, the more easily they can complete GSoC.
Here I present a list of things that students can do to maximize their chance of selection and career opportunities at the same time. These tips are useful for people applying for GSoC itself and related programs such as GNOME's Outreachy or graduate placements in companies.Disclaimers
There is no guarantee that Google will run the program again in 2016 or any future year until the Google announcement.
There is no guarantee that any organization or mentor (including myself) will be involved until the official list of organizations is published by Google.
Do not follow the advice of web sites that invite you to send pizza or anything else of value to prospective mentors.
Following the steps in this page doesn't guarantee selection. That said, people who do follow these steps are much more likely to be considered and interviewed than somebody who hasn't done any of the things in this list.Understand what free software really is
You may hear terms like free software and open source software used interchangeably.
They don't mean exactly the same thing and many people use the term free software for the wrong things. Not all projects declaring themselves to be "free" or "open source" meet the definition of free software. Those that don't, usually as a result of deficiencies in their licenses, are fundamentally incompatible with the majority of software that does use genuinely free licenses.
Google Summer of Code is about both writing and publishing your code and it is also about community. It is fundamental that you know the basics of licensing and how to choose a free license that empowers the community to collaborate on your code well after GSoC has finished.
Please review the definition of free software early on and come back and review it from time to time. The The GNU Project / Free Software Foundation have excellent resources to help you understand what a free software license is and how it works to maximize community collaboration.Don't look for shortcuts
There is no shortcut to GSoC selection and there is no shortcut to GSoC completion.
The student stipend (USD $5,500 in 2014) is not paid to students unless they complete a minimum amount of valid code. This means that even if a student did find some shortcut to selection, it is unlikely they would be paid without completing meaningful work.
If you are the right candidate for GSoC, you will not need a shortcut anyway. Are you the sort of person who can't leave a coding problem until you really feel it is fixed, even if you keep going all night? Have you ever woken up in the night with a dream about writing code still in your head? Do you become irritated by tedious or repetitive tasks and often think of ways to write code to eliminate such tasks? Does your family get cross with you because you take your laptop to Christmas dinner or some other significant occasion and start coding? If some of these statements summarize the way you think or feel you are probably a natural fit for GSoC.An opportunity money can't buy
The GSoC stipend will not make you rich. It is intended to make sure you have enough money to survive through the summer and focus on your project. Professional developers make this much money in a week in leading business centers like New York, London and Singapore. When you get to that stage in 3-5 years, you will not even be thinking about exactly how much you made during internships.
GSoC gives you an edge over other internships because it involves publicly promoting your work. Many companies still try to hide the potential of their best recruits for fear they will be poached or that they will be able to demand higher salaries. Everything you complete in GSoC is intended to be published and you get full credit for it. Imagine a young musician getting the opportunity to perform on the main stage at a rock festival. This is how the free software community works. It is a meritocracy and there is nobody to hold you back.
Having a portfolio of free software that you have created or collaborated on and a wide network of professional contacts that you develop before, during and after GSoC will continue to pay you back for years to come. While other graduates are being screened through group interviews and testing days run by employers, people with a track record in a free software project often find they go straight to the final interview round.Register your domain name and make a permanent email address
Free software is all about community and collaboration. Register your own domain name as this will become a focal point for your work and for people to get to know you as you become part of the community.
This is sound advice for anybody working in IT, not just programmers. It gives the impression that you are confident and have a long term interest in a technology career.
Choosing the provider: as a minimum, you want a provider that offers DNS management, static web site hosting, email forwarding and XMPP services all linked to your domain. You do not need to choose the provider that is linked to your internet connection at home and that is often not the best choice anyway. The XMPP foundation maintains a list of providers known to support XMPP.
Create an email address within your domain name. The most basic domain hosting providers will let you forward the email address to a webmail or university email account of your choice. Configure your webmail to send replies using your personalized email address in the From header.
Update your ~/.gitconfig file to use your personalized email address in your Git commits.Create a web site and blog
Start writing a blog. Host it using your domain name.
Some people blog every day, other people just blog once every two or three months.
Create links from your web site to your other profiles, such as a Github profile page. This helps reinforce the pages/profiles that are genuinely related to you and avoid confusion with the pages of other developers.
Many mentors are keen to see their students writing a weekly report on a blog during GSoC so starting a blog now gives you a head start. Mentors look at blogs during the selection process to try and gain insight into which topics a student is most suitable for.Create a profile on Github
Github is one of the most widely used software development web sites. Github makes it quick and easy for you to publish your work and collaborate on the work of other people. Create an account today and get in the habbit of forking other projects, improving them, committing your changes and pushing the work back into your Github account.
Github will quickly build a profile of your commits and this allows mentors to see and understand your interests and your strengths.
In your Github profile, add a link to your web site/blog and make sure the email address you are using for Git commits (in the ~/.gitconfig file) is based on your personal domain.Start using PGP
Pretty Good Privacy (PGP) is the industry standard in protecting your identity online. All serious free software projects use PGP to sign tags in Git, to sign official emails and to sign official release files.
The most common way to start using PGP is with the GnuPG (GNU Privacy Guard) utility. It is installed by the package manager on most Linux systems.
When you create your own PGP key, use the email address involving your domain name. This is the most permanent and stable solution.
Print your key fingerprint using the gpg-key2ps command, it is in the signing-party package on most Linux systems. Keep copies of the fingerprint slips with you.
This is what my own PGP fingerprint slip looks like. You can also print the key fingerprint on a business card for a more professional look.
Using PGP, it is recommend that you sign any important messages you send but you do not have to encrypt the messages you send, especially if some of the people you send messages to (like family and friends) do not yet have the PGP software to decrypt them.
Once you have a PGP key, you will need to find other developers to sign it. For people I mentor personally in GSoC, I'm keen to see that you try and find another Debian Developer in your area to sign your key as early as possible.Free software events
Try and find all the free software events in your area in the months between now and the end of the next Google Summer of Code season. Aim to attend at least two of them before GSoC.
Look closely at the schedules and find out about the individual speakers, the companies and the free software projects that are participating. For events that span more than one day, find out about the dinners, pub nights and other social parts of the event.
Try and identify people who will attend the event who have been GSoC mentors or who intend to be. Contact them before the event, if you are keen to work on something in their domain they may be able to make time to discuss it with you in person.
Take your PGP fingerprint slips. Even if you don't participate in a formal key-signing party at the event, you will still find some developers to sign your PGP key individually. You must take a photo ID document (such as your passport) for the other developer to check the name on your fingerprint but you do not give them a copy of the ID document.
Events come in all shapes and sizes. FOSDEM is an example of one of the bigger events in Europe, linux.conf.au is a similarly large event in Australia. There are many, many more local events such as the Debian UK mini-DebConf in Cambridge, November 2015. Many events are either free or free for students but please check carefully if there is a requirement to register before attending.
On your blog, discuss which events you are attending and which sessions interest you. Write a blog during or after the event too, including photos.
Quantcast generously hosted the Ganglia community meeting in San Francisco, October 2013. We had a wild time in their offices with mini-scooters, burgers, beers and the Ganglia book. That's me on the pink mini-scooter and Bernard Li, one of the other Ganglia GSoC 2014 admins is on the right.Install Linux
GSoC is fundamentally about free software. Linux is to free software what a tree is to the forest. Using Linux every day on your personal computer dramatically increases your ability to interact with the free software community and increases the number of potential GSoC projects that you can participate in.
This is not to say that people using Mac OS or Windows are unwelcome. I have worked with some great developers who were not Linux users. Linux gives you an edge though and the best time to gain that edge is now, while you are a student and well before you apply for GSoC.
If you must run Windows for some applications used in your course, it will run just fine in a virtual machine using Virtual Box, a free software solution for desktop virtualization. Use Linux as the primary operating system.
Here are links to download ISO DVD (and CD) images for some of the main Linux distributions:
If you are nervous about getting started with Linux, install it on a spare PC or in a virtual machine before you install it on your main PC or laptop. Linux is much less demanding on the hardware than Windows so you can easily run it on a machine that is 5-10 years old. Having just 4GB of RAM and 20GB of hard disk is usually more than enough for a basic graphical desktop environment although having better hardware makes it faster.
Your experiences installing and running Linux, especially if it requires some special effort to make it work with some of your hardware, make interesting topics for your blog.Decide which technologies you know best
In a GSoC program, you will typically do most of your work in just one of these languages.
From the outset, decide which language you will focus on and do everything you can to improve your competence with that language. For example, if you have already used Java in most of your course, plan on using Java in GSoC and make sure you read Effective Java (2nd Edition) by Joshua Bloch.Decide which themes appeal to you
Find a topic that has long-term appeal for you. Maybe the topic relates to your course or maybe you already know what type of company you would like to work in.
Here is a list of some topics and some of the relevant software projects:
- System administration, servers and networking: consider projects involving monitoring, automation, packaging. Ganglia is a great community to get involved with and you will encounter the Ganglia software in many large companies and academic/research networks. Contributing to a Linux distribution like Debian or Fedora packaging is another great way to get into system administration.
- Desktop and user interface: consider projects involving window managers and desktop tools or adding to the user interface of just about any other software.
- Big data and data science: this can apply to just about any other theme. For example, data science techniques are frequently used now to improve system administration.
- Business and accounting: consider accounting, CRM and ERP software.
- Finance and trading: consider projects like R, market data software like OpenMAMA and connectivity software (Apache Camel)
- Real-time communication (RTC), VoIP, webcam and chat: look at the JSCommunicator or the Jitsi project
Before the GSoC application process begins, you should aim to learn as much as possible about the theme you prefer and also gain practical experience using the software relating to that theme. For example, if you are attracted to the business and accounting theme, install the PostBooks suite and get to know it. Maybe you know somebody who runs a small business: help them to upgrade to PostBooks and use it to prepare some reports.Make something
Make some small project, less than two week's work, to demonstrate your skills. It is important to make something that somebody will use for a practical purpose, this will help you gain experience communicating with other users through Github.
For an example, see the servlet Juliana Louback created for fixing phone numbers in December 2013. It has since been used as part of the Lumicall web site and Juliana was selected for a GSoC 2014 project with Debian.
There is no better way to demonstrate to a prospective mentor that you are ready for GSoC than by completing and publishing some small project like this yourself. If you don't have any immediate project ideas, many developers will also be able to give you tips on small projects like this that you can attempt, just come and ask us on one of the mailing lists.
Ideally, the project will be something that you would use anyway even if you do not end up participating in GSoC. Such projects are the most motivating and rewarding and usually end up becoming an example of your best work. To continue the example of somebody with a preference for business and accounting software, a small project you might create is a plugin or extension for PostBooks.Getting to know prospective mentors
Many web sites provide useful information about the developers who contribute to free software projects. Some of these developers may be willing to be a GSoC mentor.
For example, look through some of the following:
- Planet / Blog aggregation sites: these sites all have links to the blogs of many developers. They are useful sources of information about events and also finding out who works on what.
- Developer profile pages. Many projects publish a page about each developer and the packages, modules or other components he/she is responsible for. Look through these lists for areas of mutual interest.
- Developer github profiles. Github makes it easy to see what projects a developer has contributed to. To see many of my own projects, browse through the history at my own Github profile
Once you have identified projects that are interesting to you and developers who work on those projects, it is important to get yourself on the developer's shortlist.
Basically, the shortlist is a list of all students who the developer believes can complete the project. If I feel that a student is unlikely to complete a project or if I don't have enough information to judge a student's probability of success, that student will not be on my shortlist.
If I don't have any student on my shortlist, then a project will not go ahead at all. If there are multiple students on the shortlist, then I will be looking more closely at each of them to try and work out who is the best match.
One way to get a developer's attention is to look at bug reports they have created. Github makes it easy to see complaints or bug reports they have made about their own projects or other projects they depend on. Another way to do this is to search through their code for strings like FIXME and TODO. Projects with standalone bug trackers like the Debian bug tracker also provide an easy way to search for bug reports that a specific person has created or commented on.
Once you find some relevant bug reports, email the developer. Ask if anybody else is working on those issues. Try and start with an issue that is particularly easy and where the solution is interesting for you. This will help you learn to compile and test the program before you try to fix any more complicated bugs. It may even be something you can work on as part of your academic program.Find successful projects from the previous year
Contact organizations and ask them which GSoC projects were most successful. In many organizations, you can find the past students' project plans and their final reports published on the web. Read through the plans submitted by the students who were chosen. Then read through the final reports by the same students and see how they compare to the original plans.Start building your project proposal now
Don't wait for the application period to begin. Start writing a project proposal now.
When writing a proposal, it is important to include several things:
- Think big: what is the goal at the end of the project? Does your work help the greater good in some way, such as increasing the market share of Linux on the desktop?
- Details: what are specific challenges? What tools will you use?
- Time management: what will you do each week? Are there weeks where you will not work on GSoC due to vacation or other events? These things are permitted but they must be in your plan if you know them in advance. If an accident or death in the family cut a week out of your GSoC project, which work would you skip and would your project still be useful without that? Having two weeks of flexible time in your plan makes it more resilient against interruptions.
- Communication: are you on mailing lists, IRC and XMPP chat? Will you make a weekly report on your blog?
- Users: who will benefit from your work?
- Testing: who will test and validate your work throughout the project? Ideally, this should involve more than just the mentor.
If your project plan is good enough, could you put it on Kickstarter or another crowdfunding site? This is a good test of whether or not a project is going to be supported by a GSoC mentor.Learn about packaging and distributing software
Packaging is a vital part of the free software lifecycle. It is very easy to upload a project to Github but it takes more effort to have it become an official package in systems like Debian, Fedora and Ubuntu.
Packaging and the communities around Linux distributions help you reach out to users of your software and get valuable feedback and new contributors. This boosts the impact of your work.
To start with, you may want to help the maintainer of an existing package. Debian packaging teams are existing communities that work in a team and welcome new contributors. The Debian Mentors initiative is another great starting place. In the Fedora world, the place to start may be in one of the Special Interest Groups (SIGs).Think from the mentor's perspective
After the application deadline, mentors have just 2 or 3 weeks to choose the students. This is actually not a lot of time to be certain if a particular student is capable of completing a project. If the student has a published history of free software activity, the mentor feels a lot more confident about choosing the student.
Some mentors have more than one good student while other mentors receive no applications from capable students. In this situation, it is very common for mentors to send each other details of students who may be suitable. Once again, if a student has a good Github profile and a blog, it is much easier for mentors to try and match that student with another project.
Getting into the world of software engineering is much like joining any other profession or even joining a new hobby or sporting activity. If you run, you probably have various types of shoe and a running watch and you may even spend a couple of nights at the track each week. If you enjoy playing a musical instrument, you probably have a collection of sheet music, accessories for your instrument and you may even aspire to build a recording studio in your garage (or you probably know somebody else who already did that).
The things listed on this page will not just help you walk the walk and talk the talk of a software developer, they will put you on a track to being one of the leaders. If you look over the profiles of other software developers on the Internet, you will find they are doing most of the things on this page already. Even if you are not selected for GSoC at all or decide not to apply, working through the steps on this page will help you clarify your own ideas about your career and help you make new friends in the software engineering community.