mozilla

Mozilla Nederland LogoDe Nederlandse
Mozilla gemeenschap

Abonneren op feed Mozilla planet
Planet Mozilla - http://planet.mozilla.org/
Bijgewerkt: 10 uur 19 min geleden

Jordan Lund: This Week In Releng - Aug 4th, 2014

ma, 11/08/2014 - 03:09

Major Highlights:

  • Kim enabled c3.xlarge slaves for selected b2g tests - bug 1031083
  • Catlee added pvtbuilds to list of things that pass through proxxy
  • Coop implemented the ability to enable/disable a slave directly from slave health

Completed work (resolution is 'FIXED'):

In progress work (unresolved and not assigned to nobody):

Categorieën: Mozilla-nl planet

Mike Conley: DocShell in a Nutshell – Part 2: The Wonder Years (1999 – 2004)

ma, 11/08/2014 - 01:04

When I first announced that I was going to be looking at the roots of DocShell, and how it has changed over the years, I thought I was going to be leafing through a lot of old CVS commits in order to see what went on before the switch to Mercurial.

I thought it’d be so, and indeed it was so. And it was painful. Having worked with DVCS’s like Mercurial and Git so much over the past couple of years, my brain was just not prepared to deal with CVS.

My solution? Take the old CVS tree, and attempt to make a Git repo out of it. Git I can handle.

And so I spent a number of hours trying to use cvs2git to convert my rsync’d mirror of the Mozilla CVS repository into something that I could browse with GitX.

“But why isn’t the CVS history in the Mercurial tree?” I hear you ask. And that’s a good question. It might have to do with the fact that converting the CVS history over is bloody hard – or at least that was my experience. cvs2git has the unfortunate habit of analyzing the entire repository / history and spitting out any errors or corruptions it found at the end.1 This is fine for small repositories, but the Mozilla CVS repo (even back in 1999) was quite substantial, and had quite a bit of history.

So my experience became: run cvs2git, wait 25 minutes, glare at an error message about corruption, scour the web for solutions to the issue, make random stabs at a solution, and repeat.

Not the greatest situation. I did what most people in my position would do, and cast my frustration into the cold, unfeeling void that is Twitter.

But, lo and behold, somebody on the other side of the planet was listening. Unfocused informed me that whoever created the gecko-dev Github mirror somehow managed to type in the black-magic incantations that would import all of the old CVS history into the Git mirror. I simply had to clone gecko-dev, and I was golden.

Thanks Unfocused. :)

So I had my tree. I cracked open Gitx, put some tea on, and started pouring through the commits from the initial creation of the docshell folder (October 15, 1999) to the last change in that folder just before the switch over to 2005 (December 15, 2004)2.

The following are my notes as I peered through those commits.

Artist’s rendering of me reading some old commit messages. I’m not claiming to have magic powers.

“First landing”

That’s the message for the first commit when the docshell/ folder was first created by Travis Bogard.

Without even looking at the code, that’s a pretty strange commit just by the message alone. No bug number, no reviewer, no approval, nothing even approximating a vague description of what was going on.

Leafing through these early commits, I was surprised to find that quite common. In fact, I learned that it was about a year after this work started that code review suddenly became mandatory for commits.

So, for these first bits of code, nobody seems to be reviewing it – or at least, nobody is signing off on it in commit messages.

Like I mentioned, the date for this commit is October 15, 1999. If the timeline in this Wikipedia article about the history of the Mozilla Suite is reliable, that puts us somewhere between Milestone 10 and Milestone 11 of the first 1.0 Mozilla Suite release.3

That means that at the time that this docshell/ folder was created, the Mozilla source code had been publicly available for over a year4, but nothing had been released from it yet.

Travis Bogard

Before we continue, who is this intrepid Travis Bogard who is spearheading this embedding initiative and the DocShell / WebShell rewrite?

At the time, according to his LinkedIn page, he worked for America Online (which at this point in time owned Netscape.5) He’d been working for AOL since 1996, working his way up the ranks from lowly intern all the way to Principal Software Engineer.

Travis was the one who originally wrote the wiki page about how painful it was embedding the web engine, and how unwieldy nsWebShell was.6 It was Travis who led the charge to strip away all of the complexity and mess inside of WebShell, and create smaller, more specialized interfaces for the behind the scenes DocShell class, which would carry out most of the work that WebShell had been doing up until that point.7

So, for these first few months, it was Travis who would be doing most of the work on DocShell.

Parallel development

These first few months, Travis puts the pedal to the metal moving things out of WebShell and into DocShell. Remember – the idea was to have a thin, simple nsWebBrowser that embedders could touch, and a fat, complex DocShell that was privately used within Gecko that was accessible via many small, specialized interfaces.

Wholesale replacing or refactoring a major part of the engine is no easy task, however – and since WebShell was core to the very function of the browser (and the mail/news client, and a bunch of other things), there were two copies of WebShell made.

The original WebShell existed in webshell/ under the root directory. The second WebShell, the one that would eventually replace it, existed under docshell/base. The one under docshell/base is the one that Travis was stripping down, but nobody was using it until it was stable. They’d continue using the one under webshell/, until they were satisfied with their implementation by both manual and automated testing.

When they were satisfied, they’d branch off of the main development line, and start removing occurances of WebShell where they didn’t need to be, and replace them with nsWebBrowser or DocShell where appropriate. When they were done that, they’d merge into main line, and celebrate!

At least, that was the plan.

That plan is spelled out here in the Plan of Attack for the redesign. That plan sketches out a rough schedule as well, and targets November 30th, 1999 as the completion point of the project.

This parallel development means that any bugs that get discovered in WebShell during the redesign needs to get fixed in two places – both under webshell/ and docshell/base.

Breaking up is so hard to do

So what was actually changing in the code? In Travis’ first commit, he adds the following interfaces:

along with some build files. Something interesting here is this nsIHTMLDocShell – where it looked like at this point, the plan was to have different DocShell interfaces depending on the type of document being displayed. Later on, we see this idea dissipate.

If DocShell was a person, these are its baby pictures. At this point, nsIDocShell has just two methods: LoadDocument, LoadDocumentVia, and a single nsIDOMDocument attribute for the document.

And here’s the interface for WebShell, which Travis was basing these new interfaces off of. Note that there’s no LoadDocument, or LoadDocumentVia, or an attribute for an nsIDOMDocument. So it seems this wasn’t just a straight-forward breakup into smaller interfaces – this was a rewrite, with new interfaces to replace the functionality of the old one.8

This is consistent with the remarks in this wikipage where it was mentioned that the new DocShell interfaces should have an API for the caller to supply a document, instead of a URI – thus taking the responsibility of calling into the networking library away from DocShell and putting it on the caller.

nsIDocShellEdit seems to be a replacement for some of the functions of the old nsIClipboardCommands methods that WebShell relied upon. Specifically, this interface was concerned with cutting, copying and pasting selections within the document. There is also a method for searching. These methods are all just stubbed out, and don’t do much at this point.

nsIDocShellFile seems to be the interface used for printing and saving documents.

nsIGenericWindow (which I believe is the ancestor of nsIBaseWindow), seems to be an interface that some embedding window must implement in order for the new nsWebBrowser / DocShell to be embedded in it. I think. I’m not too clear on this. At the very least, I think it’s supposed to be a generic interface for windows supplied by the underlying operating system.

nsIGlobalHistory is an interface for, well, browsing history. This was before tabs, so we had just a single, linear global history to maintain, and I guess that’s what this interface was for.

nsIScrollable is an interface for manipulating the scroll position of a document.

So these magnificent seven new interfaces were the first steps in breaking up WebShell… what was next?

Enter the Container

nsIDocShellContainer was created so that the DocShells could be formed into a tree and enumerated, and so that child DocShells could be named. It was introduced in this commit.

about:face

In this commit, only five days after the first landing, Travis appears to reverse the decision to pass the responsibility of loading the document onto the caller of DocShell. LoadDocument and LoadDocumentVia are replaced by LoadURI and LoadURIVia. Steve Clark (aka “buster”) is also added to the authors list of the nsIDocShell interface. It’s not clear to me why this decision was reversed, but if I had to guess, I’d say it proved to be too much of a burden on the callers to load all of the documents themselves. Perhaps they punted on that goal, and decided to tackle it again later (though I will point out that today’s nsIDocShell still has LoadURI defined in it).

First implementor

The first implementation of nsIDocShell showed up on October 25, 1999. It was nsHTMLDocShell, and with the exception of nsIGlobalHistory, it implemented all of the other interfaces that I listed in Travis’ first landing.

The base implementation

On October 25th, the stubs of a DocShell base implementation showed up in the repository. The idea, I would imagine, is that for each of the document types that Gecko can display, we’d have a DocShell implementation, and each of these DocShell implementations would inherit from this DocShell base class, and only override the things that they need specialized for their particular document type.

Later on, when the idea of having specialized DocShell implementations evaporates, this base class will end up being nsDocShell.cpp.

That same day, most of the methods were removed from the nsHTMLDocShell implementation, and nsHTMLDocShell was made to inherit from nsDocShellBase.

“Does not compile yet”

The message for this commit on October 27th, 1999 is pretty interesting. It reads:

added a bunch of initial implementation. does not compile yet, but that’s ok because docshell isn’t part of the build yet.

So not only are none of these patches being reviewed (as far as I can tell), and are not mapped to any bugs in the bug tracker, but the patches themselves just straight-up do not build. They are not building on tinderbox.

This is in pretty stark contrast to today’s code conventions. While it’s true that we might land code that is not entered for most Nightly users, we usually hide such code behind an about:config pref so that developers can flip it on to test it. And I think it’s pretty rare (if it ever occurs) for us to land code in mozilla-central that’s not immediately put into the build system.

Perhaps the WebShell tests that were part of the Plan of Attack were being written in parallel and just haven’t landed, but I suspect that they haven’t been written at all at this point. I suspect that the team was trying to stand something up and make it partially work, and then write tests for WebShell and try to make them pass for both old WebShell and DocShell. Or maybe just the latter.

These days, I think that’s probably how we’d go about such a major re-architecture / redesign / refactor; we’d write tests for the old component, land code that builds but is only entered via an about:config pref, and then work on porting the tests over to the new component. Once the tests pass for both, flip the pref for our Nightly users and let people test the new stuff. Once it feels stable, take it up the trains. And once it ships and it doesn’t look like anything is critically wrong with the new component, begin the process of removing the old component / tests and getting rid of the about:config pref.

Note that I’m not at all bashing Travis or the other developers who were working on this stuff back then – I’m simply remarking on how far we’ve come in terms of development practices.

Remember AOL keywords?

Tangentially, I’m seeing some patches go by that have to do with hooking up some kind of “Keyword” support to WebShell.

Remember those keywords? This was the pre-Google era where there were only a few simplistic search engines around, and people were still trying to solve discoverability of things on the web. Keywords was, I believe, AOL’s attempt at a solution.

You can read up on AOL Keywords here. I just thought it was interesting to find some Keywords support being written in here.

One DocShell to rule them all

Now that we have decided that there is only one docshell for all content types, we needed to get rid of the base class/ content type implementation. This checkin takes and moves the nsDocShellBase to be nsDocShell. It now holds the nsIHTMLDocShell stuff. This will be going away. nsCDocShell was created to replace the previous nsCHTMLDocShell.

This commit lands on November 12th (almost a month from the first landing), and is the point where the DocShell-implementation-per-document-type plan breaks down. nsDocShellBase gets renamed to nsDocShell, and the nsIHTMLDocShell interface gets moved into nsIDocShell.idl, where a comment above it indicates that the interface will soon go away.

We have nsCDocShell.idl, but this interface will eventually disappear as well.

[PORKJOCKEY]

So, this commit message on November 13th caught my eye:

pork jockey paint fixes. bug=18140, r=kmcclusk,pavlov

What the hell is a “pork jockey”? A quick search around, and I see yet another reference to it in Bugzilla on bug 14928. It seems to be some kind of project… or code name…

I eventually found this ancient wiki page that documents some of the language used in bugs on Bugzilla, and it has an entry for “pork jockey”: a pork jockey bug is a “bug for work needed on infrastructure/architecture”.

I mentioned this in #developers, and dmose (who was hacking on Mozilla code at the time), explained:

16:52 (dmose) mconley: so, porkjockeys
16:52 (mconley) let’s hear it
16:52 (dmose) mconley: at some point long ago, there was some infrastrcture work that needed to happen
16:52 mconley flips on tape recorder
16:52 (dmose) and when people we’re talking about it, it seemed very hard to carry off
16:52 (dmose) somebody said that that would happen on the same day he saw pigs fly
16:53 (mconley) ah hah
16:53 (dmose) so ultimately the group of people in charge of trying to make that happen were…
16:53 (dmose) the porkjockeys
16:53 (dmose) which was the name of the mailing list too

Here’s the e-mail that Brendan Eich sent out to get the Porkjockey’s flying.

Development play-by-play

On November 17th, the nsIGenericWindow interface was removed because it was being implemented in widget/base/nsIBaseWindow.idl.

On November 27th, nsWebShell started to implement nsIBaseWindow, which helped pull a bunch of methods out of the WebShell implementations.

On November 29th, nsWebShell now implements nsIDocShell – so this seems to be the first point where the DocShell work gets brought into code paths that might be hit. This is particularly interesting, because he makes this change to both the WebShell implementation that he has under the docshell/ folder, as well as the WebShell implementation under the webshell/ folder. This means that some of the DocShell code is now actually being used.

On November 30th, Travis lands a patch to remove some old commented out code. The commit message mentions that the nsIDocShellEdit and nsIDocShellFile interfaces introduced in the first landing are now defunct. It doesn’t look like anything is diving in to replace these interfaces straight away, so it looks like he’s just not worrying about it just yet. The defunct interfaces are removed in this commit one day later.

On December 1st, WebShell (both the fork and the “live” version) is made to implement nsIDocShellContainer.

One day later, nsIDocShellTreeNode interface is added to replace nsIDocShellContainer. The interface is almost identical to nsIDocShellContainer, except that it allows the caller to access child DocShells at particular indices as opposed to just returning an enumerator.

December 3rd was a very big day! Highlights include:

Noticing something? A lot of these changes are getting dumped straight into the live version of WebShell (under the webshell/ directory). That’s not really what the Plan of Attack had spelled out, but that’s what appears to be happening. Perhaps all of this stuff was trivial enough that it didn’t warrant waiting for the WebShell fork to switch over.

On December 12th, nsIDocShellTreeOwner is introduced.

On December 15th, buster re-lands the nsIDocShellEdit and nsIDocShellFile interfaces that were removed on November 30th, but they’re called nsIContentViewerEdit and nsIContentViewerFile, respectively. Otherwise, they’re identical.

On December 17th, WebShell becomes a subclass of DocShell. This means that a bunch of things can get removed from WebShell, since they’re being taken care of by the parent DocShell class. This is a pretty significant move in the whole “replacing WebShell” strategy.

Similar work occurs on December 20th, where even more methods inside WebShell start to forward to the base DocShell class.

That’s the last bit of notable work during 1999. These next bits show up in the new year, and provides further proof that we didn’t all blow up during Y2K.

On Feburary 2nd, 2000, a new interface called nsIWebNavigation shows up. This interface is still used to this day to navigate a browser, and to get information about whether it can go “forwards” or “backwards”.

On February 8th, a patch lands that makes nsGlobalWindow deal entirely in DocShells instead of WebShells. nsIScriptGlobalObject also now deals entirely with DocShells. This is a pretty big move, and the patch is sizeable.

On February 11th, more methods are removed from WebShell, since the refactorings and rearchitecture have made them obsolete.

On February 14th, for Valentine’s day, Travis lands a patch to have DocShell implement the nsIWebNavigation interface. Later on, he lands a patch that relinquishes further control from WebShell, and puts the DocShell in control of providing the script environment and providing the nsIScriptGlobalObjectOwner interface. Not much later, he lands a patch that implements the Stop method from the nsIWebNavigation interface for DocShell. It’s not being used yet, but it won’t be long now. Valentine’s day was busy!

On February 24th, more stuff (like the old Stop implementation) gets gutted from WebShell. Some, if not all, of those methods get forwarded to the underlying DocShell, unsurprisingly.

Similar story on February 29th, where a bunch of the scroll methods are gutted from WebShell, and redirected to the underlying DocShell. This one actually has a bug and some reviewers!9 Travis also landed a patch that gets DocShell set up to be able to create its own content viewers for various documents.

March 10th saw Travis gut plenty of methods from WebShell and redirect to DocShell instead. These include Init, SetDocLoaderObserver, GetDocLoaderObserver, SetParent, GetParent, GetChildCount, AddChild, RemoveChild, ChildAt, GetName, SetName, FindChildWithName, SetChromeEventHandler, GetContentViewer, IsBusy, SetDocument, StopBeforeRequestingURL, StopAfterURLAvailable, GetMarginWidth, SetMarginWidth, GetMarginHeight, SetMarginHeight, SetZoom, GetZoom. A few follow-up patches did something similar. That must have been super satisfying.

March 11th, Travis removes the Back, Forward, CanBack and CanForward methods from WebShell. Consumers of those can use the nsIWebNavigation interface on the DocShell instead.

March 30th sees the nsIDocShellLoadInfo interface show up. This interface is for “specifying information used in a nsIDocShell::loadURI call”. I guess this is certainly better than adding a huge amount of arguments to ::loadURI.

During all of this, I’m seeing references to a “new session history” being worked on. I’m not really exploring session history (at this point), so I’m not highlighting those commits, but I do want to point out that a replacement for the old Netscape session history stuff was underway during all of this DocShell business, and the work intersected quite a bit.

On April 16th, Travis lands a commit that takes yet another big chunk out of WebShell in terms of loading documents and navigation. The new session history is now being used instead of the old.

The last 10% is the hardest part

We’re approaching what appears to be the end of the DocShell work. According to his LinkedIn profile, Travis left AOL in May 2000. His last commit to the repository before he left was on April 24th. Big props to Travis for all of the work he put in on this project – by the time he left, WebShell was quite a bit simpler than when he started. I somehow don’t think he reached the end state that he had envisioned when he’d written the original redesign document – the work doesn’t appear to be done. WebShell is still around (in fact, parts of it around were around until only recently!10 ). Still, it was a hell of chunk of work he put in.

And if I’m interpreting the rest of the commits after this correctly, there is a slow but steady drop off in large architectural changes, and a more concerted effort to stabilize DocShell, nsWebBrowser and nsWebShell. I suspect this is because everybody was buckling down trying to ship the first version of the Mozilla Suite (which finally occurred June 5th, 2002 – still more than 2 years down the road).

There are still some notable commits though. I’ll keep listing them off.

On June 22nd, a developer called “rpotts” lands a patch to remove the SetDocument method from DocShell, and to give the implementation / responsibility of setting the document on implementations of nsIContentViewer.

July 5th sees rpotts move the new session history methods from nsIWebNavigation to a new interface called nsIDocShellHistory. It’s starting to feel like the new session history is really heating up.

On July 18th, a developer named Judson Valeski lands a large patch with the commit message “webshell-docshell consolodation changes”. Paraphrasing from the bug, the point of this patch is to move WebShell into the DocShell lib to reduce the memory footprint. This also appears to be a lot of cleanup of the DocShell code. Declarations are moved into header files. The nsDocShellModule is greatly simplified with some macros. It looks like some dead code is removed as well.

On November 9th, a developer named “disttsc” moves the nsIContentViewer interface from the webshell folder to the docshell folder, and converts it from a manually created .h to an .idl. The commit message states that this work is necessary to fix bug 46200, which was filed to remove nsIBrowserInstance (according to that bug, nsIBrowserInstance must die).

That’s probably the last big, notable change to DocShell during the 2000′s.

2001: A DocShell Odyssey

On March 8th, a developer named “Dan M” moves the GetPersistence and SetPersistence methods from nsIWebBrowserChrome to nsIDocShellTreeOwner. He sounds like he didn’t really want to do it, or didn’t want to be pegged with the responsibility of the decision – the commit message states “embedding API review meeting made me do it.” This work was tracked in bug 69918.

On April 16th, Judson Valeski makes it so that the mimetypes that a DocShell can handle are not hardcoded into the implementation. Instead, handlers can be registered via the CategoryManager. This work was tracked in bug 40772.

On April 26th, a developer named Simon Fraser adds an implementation of nsISimpleEnumerator for DocShells. This implementation is called, unsurprisingly, nsDocShellEnumerator. This was for bug 76758. A method for retrieving an enumerator is added one day later in a patch that fixes a number of bugs, all related to the page find feature.

April 27th saw the first of the NSPR logging for DocShell get added to the code by a developer named Chris Waterson. Work for that was tracked in bug 76898.

On May 16th, for bug 79608, Brian Stell landed a getter and setter for the character set for a particular DocShell.

There’s a big gap here, where the majority of the landings are relatively minor bug fixes, cleanup, or only slightly related to DocShell’s purpose, and not worth mentioning.11

And beyond the infinite…

On January 8th, 2002, for bug 113970, Stephen Walker lands a patch that takes yet another big chunk out of WebShell, and added this to the nsIWebShell.h header:

/** * WARNING WARNING WARNING WARNING WARNING WARNING WARNING WARNING !!!! * * THIS INTERFACE IS DEPRECATED. DO NOT ADD STUFF OR CODE TO IT!!!! */

I’m actually surprised it too so long for something like this to get added to the nsIWebShell interface – though perhaps there was a shared understanding that nsIWebShell was shrinking, and such a notice wasn’t really needed.

On January 9th, 2003 (yes, a whole year later – I didn’t find much worth mentioning in the intervening time), I see the first reference to “deCOMtamination”, which is an effort to reduce the amount of XPCOM-style code being used. You can read up more on deCOMtamination here.

On January 13th, 2003, Nisheeth Ranjan lands a patch to use “machine learning” in order to order results in the urlbar autocomplete list. I guess this was the precursor to the frencency algorithm that the AwesomeBar uses today? Interestingly, this code was backed out again on February 21st, 2003 for reasons that aren’t immediately clear – but it looks like, according to this comment, the code was meant to be temporary in order to gather “weights” from participants in the Mozilla 1.3 beta, which could then be hard-coded into the product. The machine-learning bug got closed on June 25th, 2009 due to AwesomeBar and frecency.

On Februrary 11th, 2004, the onbeforeunload event is introduced.

On April 17, 2004, gerv lands the first of several patches to switch the licensing of the code over to the MPL/LGPL/GPL tri-license. That’d be MPL 1.1.

On July 7th, 2004, timeless lands a patch that makes it so that embedders can disable all plugins in a document.

On November 23rd, 2004, the parent and treeOwner attributes for nsIDocShellTreeItem are made scriptable. They are read-only for script.

On December 8th, 2004, bz lands a patch that makes DocShell inherit from DocLoader, and starts a move to eliminate nsIWebShell, nsIWebShellContainer, and nsIDocumentLoader.

And that’s probably the last notable patch related to DocShell in 2004.

Lord of the rings

Reading through all of these commits in the docshell/ and webshell/ folders is a bit like taking a core sample of a very mature tree, and reading its rings. I can see some really important events occurring as I flip through these commits – from the very birth of Mozilla, to the birth of XPCOM and XUL, to porkjockeys, and the start of the embedding efforts… all the way to the splitting out of Thunderbird, deCOMtamination and introduction of the MPL. I really got a sense of the history of the Mozilla project reading through these commits.

I feel like I’m getting a better and better sense of where DocShell came from, and why it does what it does. I hope if you’re reading this, you’re getting that same sense.

Stay tuned for the next bit, where I look at years 2005 to 2010.

  1. Messages like: ERROR: A CVS repository cannot contain both cvsroot/mozilla/browser/base/content/metadata.js,v and cvsroot/mozilla/browser/base/content/Attic/metadata.js,v, for example 

  2. Those commits have hashes 575e116287f99fbe26f54ca3f3dbda377002c5e7 and  60567bb185eb8eea80b9ab08d8695bb27ba74e15 if you want to follow along at home. 

  3. Mozilla Suite 1.0 eventually ships on June 5th, 2002 

  4. According to Mozilla’s timeline, the source code was posted publicly on March 31st, 1998 

  5. According to Wikipedia, Netscape was purchased by AOL on March 17, 1999 

  6. That wiki page was first posted on October 8th, 1999 – exactly a week before the first docshell work had started. 

  7. I should point out that there is no mention of this embedding work in the first roadmap that was published for the Mozilla project. It was only in the second roadmap, published almost a year after the DocShell work began, that embedding was mentioned, albeit briefly. 

  8. More clues as to what WebShell was originally intended for are also in that interface file:

    /** * The web shell is a container for implementations of nsIContentViewer. * It is a content-viewer-container and also knows how to delegate certain * behavior to an nsIWebShellContainer. * * Web shells can be arranged in a tree. * * Web shells are also nsIWebShellContainer's because they can contain * other web shells. */

    So that helps clear things up a bit. I think. 

  9. travis reviewed it, and the patch was approved by rickg. Not sure what approval meant back then… was that like a super review? 

  10. This is the last commit to the webshell folder before it got removed on September 30th, 2010. 

  11. Hopefully I didn’t miss anything interesting in there! 

Categorieën: Mozilla-nl planet

Peter Bengtsson: Gzip rules the world of optimization, often

zo, 10/08/2014 - 01:20

So I have a massive chunk of JSON that a Django view is sending to a piece of Angular that displays it nicely on the page. It's big. 674Kb actually. And it's likely going to be bigger in the near future. It's basically a list of dicts. It looks something like this:

>>> pprint(d['events'][0]) {u'archive_time': None, u'archive_url': u'/manage/events/archive/1113/', u'channels': [u'Main'], u'duplicate_url': u'/manage/events/duplicate/1113/', u'id': 1113, u'is_upcoming': True, u'location': u'Cyberspace - Pacific Time', u'modified': u'2014-08-06T22:04:11.727733+00:00', u'privacy': u'public', u'privacy_display': u'Public', u'slug': u'bugzilla-development-meeting-20141115', u'start_time': u'15 Nov 2014 02:00PM', u'start_time_iso': u'2014-11-15T14:00:00-08:00', u'status': u'scheduled', u'status_display': u'Scheduled', u'thumbnail': {u'height': 32, u'url': u'/media/cache/e7/1a/e71a58099a0b4cf1621ef3a9fe5ba121.png', u'width': 32}, u'title': u'Bugzilla Development Meeting'}

So I thought one hackish simplification would be to convert each of these dicts into an list with a known sort order. Something like this:

>>> event = d['events'][0] >>> pprint([event[k] for k in sorted(event)]) [None, u'/manage/events/archive/1113/', [u'Main'], u'/manage/events/duplicate/1113/', 1113, True, u'Cyberspace - Pacific Time', u'2014-08-06T22:04:11.727733+00:00', u'public', u'Public', u'bugzilla-development-meeting-20141115', u'15 Nov 2014 02:00PM', u'2014-11-15T14:00:00-08:00', u'scheduled', u'Scheduled', {u'height': 32, u'url': u'/media/cache/e7/1a/e71a58099a0b4cf1621ef3a9fe5ba121.png', u'width': 32}, u'Bugzilla Development Meeting']

So I converted my sample events.json file like that:

$ l -h events* -rw-r--r-- 1 peterbe wheel 674K Aug 8 14:08 events.json -rw-r--r-- 1 peterbe wheel 423K Aug 8 15:06 events.optimized.json

Excitingly the file is now 250Kb smaller because it no longer contains all those keys.

Now, I'd also send the order of the keys so I could do something like this in the AngularJS code:

.success(function(response) { events = [] response.events.forEach(function(event) { var new_event = {} response.keys.forEach(function(key, i) { new_event[k] = event[i] }) }) })

Yuck! Nested loops! It was just getting more and more complicated.
Also, if there are keys that are not present in every element, it means I'd have to replace them with None.

At this point I stopped and I could smell the hackish stink of sulfur of the hole I was digging myself into.
Then it occurred to me, gzip is really good at compressing repeated things which is something we have plenty of in a document store type data structure that a list of dicts is.

So I packed them manually to see what we could get:

$ apack events.json.gz events.json $ apack events.optimized.json.gz events.optimized.json

And without further ado...

$ l -h events* -rw-r--r-- 1 peterbe wheel 674K Aug 8 14:08 events.json -rw-r--r-- 1 peterbe wheel 90K Aug 8 14:20 events.json.gz -rw-r--r-- 1 peterbe wheel 423K Aug 8 15:06 events.optimized.json -rw-r--r-- 1 peterbe wheel 81K Aug 8 15:07 events.optimized.json.gz

Basically, all that complicated and slow hoopla for saving 10Kb. No thank you.

Thank you gzip for existing!

Categorieën: Mozilla-nl planet

Mozilla Release Management Team: Firefox 32 beta4 to beta5

za, 09/08/2014 - 16:59

  • 27 changesets
  • 61 files changed
  • 1186 insertions
  • 161 deletions

ExtensionOccurrences cpp19 h16 js3 ini3 html3 xml2 xhtml2 java2 in2 txt1 py1 jsm1 dep1 c1 build1 asm1

ModuleOccurrences content14 dom12 security9 mobile5 netwerk4 js4 toolkit3 docshell3 media2 browser2 memory1

List of changesets:

Brian HackettBug 1037666. r=billm, a=abillings - 784c7fb4c431 Byron Campen [:bwc]Bug 963524 - Avoid setting SO_REUSEADDR when binding a UDP socket to port 0, since the linux kernel might select an already open port. r=mcmanus, a=sledru - 92364eef664a Mike HommeyBug 1047791 - Use integers for audio when on Android, or when using ARM on other OSes, and disable webm encoding. r=padenot, a=NPOTB - 333e7f930c63 Boris ZbarskyBug 1045096 - Make sure initial about:blank in iframes have a nsDOMNavigationTiming object, so we don't end up with window.performance throwing when accessed on their window. r=smaug, a=sledru - 70277dbb9071 Bobby HolleyBug 1048330 - Null-check the XBL scope in more places. r=smaug, a=sledru - f5df74fab22f Brian NicholsonBug 987864 - Move API 13 styles to values-v13 folder. r=wesj, a=sledru - 68181edc64c1 Blair McBrideBug 1024073 - Whitelist plugin: Roblox Plugin launcher. r=gfritzsche, a=sledru - cda6534d9780 Blair McBrideBug 1029654 - Whitelist plugin: Box Edit. r=gfritzsche, a=sledru - 4d812a850eb1 Blair McBrideBug 1024965 - Whitelist plugin: Nexus Personal. r=gfritzsche, a=sledru - c2fc1e357ca0 Martin ThomsonBug 1048261 - Safe dispatch from DTLS connect to PeerConnectionImpl. r=jesup, r=bwc, a=abillings - adb28e85421f Kai EngertBug, 1048876, Update Mozilla 32 to use NSS 3.16.4, a=lmandel - 22589028e561 Cameron McCormackBug 1018524 - Make copies of more SVGLength objects when manipulating SVGLengthLists. r=longsonr, a=lmandel - 3315c53c4bb7 Cameron McCormackBug 1040575 - Make a copy of SVGSVGElement.currentTranslate if it is inserted into an SVGPointList. r=longsonr, a=lmandel - 4da65dc7d057 Valentin GosuBug 1035007 - Make sure hostname is inserted in the right place. r=mcmanus, a=sledru - e6cee3b7907e Ben KellyBug 995688 - Always show scrollbars in test to avoid fade effect evicting document from BF cache. r=smaug, a=test-only - 8f34703f5065 Richard NewmanBug 1049217 - Fix CharacterValidator for API levels below 12. r=margaret, a=sledru - fc7ce6481ea9 Randell JesupBug 1037000 - Include "default communications device" in the audio input enumeration for gUM. r=padenot, a=sledru - ee74d30a8968 Gregory SzorcBug 1045421 - Remove date time bomb from test_crash_manager.js, add logging. r=gfritzsche, a=test-only - ac0afa7b1b25 Jason OrendorffBug 1043690 - Part 1: Provide helper function for HTMLDocument and HTMLFormElement proxies to use from [[Set]]. r=efaust, a=lmandel - 8375886783f2 Boris ZbarskyBug 1043690 - Part 2: Change the codegen for DOM proxies to ignore named props when looking up property descriptors on [[Set]]. r=efaust, a=lmandel - 06542873b0dc Jan de MooijBug 1041148 - Move HoldJSObjects call to XMLHttpRequest constructor to simplify the code. r=bz, a=lmandel - 7b29fabbf26a Ryan VanderMeulenBug 1043690 - Remove extraneous const usage. r=jorendorff, a=bustage - c755d28a5266 Michael ComellaBug 1047264 - Dynamically retrieve Views for Display Mode on tablet. r=lucasr, a=sledru - e8fbf14de311 Richard NewmanBug 1047549 - Copy libraries out of the APK when they're missing. r=blassey, a=lmandel - 9d8f79b400bf Andrew McCreightBug 1038243 - Avoid static casting from nsIDOMAttr to Attr. r=smaug, a=lmandel - 551f71d3138f Michael WuBug 1044193 - Always cleanup on heap allocation path. r=glandium, a=lmandel - e51295fe2c54 Jonas JenwaldBug 1028735 - Revert one aspect of CIDFontType2 font parsing to the state prior to CMAP landing. r=yury, a=lmandel - 599c7756380c

Categorieën: Mozilla-nl planet

Mozilla Release Management Team: Firefox 32 beta4 to beta5

za, 09/08/2014 - 15:00

  • 27 changesets
  • 61 files changed
  • 1186 insertions
  • 161 deletions

ExtensionOccurrences cpp19 h16 js3 ini3 html3 xml2 xhtml2 java2 in2 txt1 py1 jsm1 dep1 c1 build1 asm1

ModuleOccurrences content14 dom12 security9 mobile5 netwerk4 js4 toolkit3 docshell3 media2 browser2 memory1

List of changesets:

Brian HackettBug 1037666. r=billm, a=abillings - 784c7fb4c431 Byron Campen [:bwc]Bug 963524 - Avoid setting SO_REUSEADDR when binding a UDP socket to port 0, since the linux kernel might select an already open port. r=mcmanus, a=sledru - 92364eef664a Mike HommeyBug 1047791 - Use integers for audio when on Android, or when using ARM on other OSes, and disable webm encoding. r=padenot, a=NPOTB - 333e7f930c63 Boris ZbarskyBug 1045096 - Make sure initial about:blank in iframes have a nsDOMNavigationTiming object, so we don't end up with window.performance throwing when accessed on their window. r=smaug, a=sledru - 70277dbb9071 Bobby HolleyBug 1048330 - Null-check the XBL scope in more places. r=smaug, a=sledru - f5df74fab22f Brian NicholsonBug 987864 - Move API 13 styles to values-v13 folder. r=wesj, a=sledru - 68181edc64c1 Blair McBrideBug 1024073 - Whitelist plugin: Roblox Plugin launcher. r=gfritzsche, a=sledru - cda6534d9780 Blair McBrideBug 1029654 - Whitelist plugin: Box Edit. r=gfritzsche, a=sledru - 4d812a850eb1 Blair McBrideBug 1024965 - Whitelist plugin: Nexus Personal. r=gfritzsche, a=sledru - c2fc1e357ca0 Martin ThomsonBug 1048261 - Safe dispatch from DTLS connect to PeerConnectionImpl. r=jesup, r=bwc, a=abillings - adb28e85421f Kai EngertBug, 1048876, Update Mozilla 32 to use NSS 3.16.4, a=lmandel - 22589028e561 Cameron McCormackBug 1018524 - Make copies of more SVGLength objects when manipulating SVGLengthLists. r=longsonr, a=lmandel - 3315c53c4bb7 Cameron McCormackBug 1040575 - Make a copy of SVGSVGElement.currentTranslate if it is inserted into an SVGPointList. r=longsonr, a=lmandel - 4da65dc7d057 Valentin GosuBug 1035007 - Make sure hostname is inserted in the right place. r=mcmanus, a=sledru - e6cee3b7907e Ben KellyBug 995688 - Always show scrollbars in test to avoid fade effect evicting document from BF cache. r=smaug, a=test-only - 8f34703f5065 Richard NewmanBug 1049217 - Fix CharacterValidator for API levels below 12. r=margaret, a=sledru - fc7ce6481ea9 Randell JesupBug 1037000 - Include "default communications device" in the audio input enumeration for gUM. r=padenot, a=sledru - ee74d30a8968 Gregory SzorcBug 1045421 - Remove date time bomb from test_crash_manager.js, add logging. r=gfritzsche, a=test-only - ac0afa7b1b25 Jason OrendorffBug 1043690 - Part 1: Provide helper function for HTMLDocument and HTMLFormElement proxies to use from [[Set]]. r=efaust, a=lmandel - 8375886783f2 Boris ZbarskyBug 1043690 - Part 2: Change the codegen for DOM proxies to ignore named props when looking up property descriptors on [[Set]]. r=efaust, a=lmandel - 06542873b0dc Jan de MooijBug 1041148 - Move HoldJSObjects call to XMLHttpRequest constructor to simplify the code. r=bz, a=lmandel - 7b29fabbf26a Ryan VanderMeulenBug 1043690 - Remove extraneous const usage. r=jorendorff, a=bustage - c755d28a5266 Michael ComellaBug 1047264 - Dynamically retrieve Views for Display Mode on tablet. r=lucasr, a=sledru - e8fbf14de311 Richard NewmanBug 1047549 - Copy libraries out of the APK when they're missing. r=blassey, a=lmandel - 9d8f79b400bf Andrew McCreightBug 1038243 - Avoid static casting from nsIDOMAttr to Attr. r=smaug, a=lmandel - 551f71d3138f Michael WuBug 1044193 - Always cleanup on heap allocation path. r=glandium, a=lmandel - e51295fe2c54 Jonas JenwaldBug 1028735 - Revert one aspect of CIDFontType2 font parsing to the state prior to CMAP landing. r=yury, a=lmandel - 599c7756380c

Categorieën: Mozilla-nl planet

Sean Bolton: Community Lessons from LEGO (and other thoughts on community)

vr, 08/08/2014 - 22:24

Communities form when you find a group of people that you feel you can be yourself around. In this environment, you don’t have to self-edit as much to fit in. You get the feeling that these people share a similar view of the world in some way. Communities form when the answer to, “are they like me?” is “yes.”

It’s not very surprising then that communities can form around brands as brands often represent a certain view of the world. Smart companies notice this and they engage these people in a win-win partnership to give their community value and to create value for the organization. That is no easy task. Here is a bit about how LEGO does it…

LEGO cares deeply about their community. They know that these people drive their brand, provide the best form of marketing and can help fuel new product development. And the community members get a powerful sense of fulfillment in meeting similar people and getting to be more of a creator than a consumer – they get to make LEGO their own. When working with community LEGO follows these principles (a non-exhaustive list):

  1.  Every engagement must be win-win
    • This matters because only in win-win partnerships do both sides have a good experience and can community continue to grow. It means that there is some saying no. But the saying no is so that more yes can happen later.
  2. LEGO does not interfere with community
    • This is powerful because it lets people all over the world organize their groups in a way that best makes sense for them. LEGO does not have to waste resources and can be more culturally sensitive by letting others manage their groups in the ways that best suits them.
  3. Members are respected and expected to respect each other
    • Every community should have a code of conduct to ensure that people respect each other. This is common practice so that members don’t get abused.
  4. Empower people to do what they want to do
    • You can’t force community to do something there is no passion behind doing – it’s just not how it works. The fact that LEGO is explicit about that is a big deal and it makes community respect them for it. That’s powerful.

LEGO has high quality standards and the way their community team works is no exception. They have a communication structure that empowers people to help and learn from each other so that not every person needs to communicate with LEGO directly. There are designated ‘ambassadors’ that communicate directly with LEGO – they help distill and clarify communication, taking a sort of leadership role in the community. This helps LEGO better focus resources and helps build a stronger sense of community among members (win-win).

There is a massive community surrounding LEGO, over 300K. To reference, we currently have about 8K at Mozilla. While our communities do different things, we are all fundamentally driven to be part of that group that makes us feel like we can be more of ourselves. It gives people a sense of belonging and fulfillment – something we all want in life. That is why community management matters. Without it, these large groups of people can’t exist and that sense of belonging can be easily lost.

[Post featured on Mozilla's Community blog.]


Categorieën: Mozilla-nl planet

Kim Moir: Mozilla pushes - July 2014

vr, 08/08/2014 - 20:16
Here's the July 2014 monthly analysis of the pushes to our Mozilla development trees. You can load the data as an HTML page or as a json file.
 
Trends
Like every month for the past while, we had a new record number of pushes. In reality, given that July is one day longer than June, the numbers are quite similar.

Highlights

  • 12,755 pushes
    • new record
  •  411 pushes/day (average)
  • Highest number of pushes/day: 625 pushes on July 3, 2014
  • Highest 23.51 pushes/hour (average)
    • new record

General remarks
Try keeps on having around 38% of all the pushes. Gaia-Try is in second place with around 31% of pushes.  The three integration repositories (fx-team, mozilla-inbound and b2g-inbound) account around 22% of all the pushes.

Records 
July 2014 was the month with most pushes (12,755 pushes)
June 2014 has the highest pushes/day average with 662 pushes/day
July 2014 has the highest average of "pushes-per-hour" with 23.51 pushes/hour
June 4th, 2014 had the highest number of pushes in one day with 662 
 
Categorieën: Mozilla-nl planet

Eric Shepherd: The Sheppy Report: August 8, 2014

vr, 08/08/2014 - 19:47

It’s been another good week of Making Things Happen. I’m pleased with the productivity this week: not just mine, but the entire writing community’s.

What I’m up to

As usual, what I did this week doesn’t entirely line up with what I’d expected, but it was still productive, which is the important thing, right?

Work continued on finishing up the doc project planning page migration work, and better integration with other pages on MDN and elsewhere. I also put together the first draft of the WebGL doc plan.

I’m working on trying to reshuffle my many personal appointments that happen regularly to less often interfere with meetings, but unfortunately there’s only so much I can do.

What I did this week
  • Replaced the documentation project planning page on wikimo with a link to the new page on MDN, with an explanation of why it moved.
  • Finished work on moving the Learning area doc project plan to MDN.
  • Migrated the “Writing chrome code” doc project plan to MDN, then emailed Will Bamberg to let him know his planning docs had moved.
  • Wrote first draft of the WebGL doc plan and emailed dev team to request feedback.
    • Got quick feedback with WebGL 2.0 ship date and added that information to the doc plan.
  • Added the dev-doc-needed keyword to man WebGL 2.0 related bugs.
  • Filed a bug about a problem with the link editor not suggesting zone pages that have moved out of the /docs/ hierarchy.
  • Added a link to the WebRTC doc project plan to the WebRTC documentation status page.
  • Posted to the dev-media list asking for suggestions on topics to cover in the WebRTC docs.
  • Updated the MDN page about KumaScript macros to link to the new article on troubleshooting them that Stephanie Hobson wrote.
  • Did a quick copy-edit pass on the troubleshooting article, and added some information about how to use search keywords to get to macro pages quickly (to read the built-in documentation most have).
  • Emailed out various meeting reminders.
  • Updated the team priority list spreadsheet with updated URLs and new information.
  • Wrote agenda for writers’ staff meeting and Web API docs meeting.
  • Wrote a nifty new MDN macro, ContentFromWikimo, which imports the content of a specified block (by ID) from a page on wikimo and inserts it into the MDN page.
  • Used the ContentFromWikimo macro to embed module owner information about WebRTC, WebGL, Web Workers, and XPCOM to their doc plans.
  • Filed a number of meta/tracking bugs for the various doc plans.
  • Created meta/tracking bugs for all current documentation project plans. See my standup for today for links; I’m not going to copy and paste them all here. :)
Meetings attended this week Monday
  • #mdndev bug/triage/planning meetings.
Tuesday
  • Messaging discussion for MDN feature planning.
Wednesday
  • 1:1 meeting with Ali.
Thursday
  • MDN writers’ staff meeting.
Friday
  • MDN development bug swat meeting.
  • Web APIs documentation meeting.

So, whew! Lots done! I’m particularly proud of the ContentFromWikimo macro work. It was also a lot of fun to do. I think it’ll be useful, too, at least sometimes.

I have a good feeling about next week. I think it’ll be even more productive!

Categorieën: Mozilla-nl planet

Mozilla Open Policy & Advocacy Blog: Thank you, Mr. President.

vr, 08/08/2014 - 18:52

Today, Mozilla joined with dozens of advocacy organizations and companies to urge President Obama to take action on net neutrality in response to his recent vocal support for fair and meaningful net neutrality rules. Expressing views echoed by millions of Americans, the groups urged the FCC to stand against fast lanes for those who can afford it and slow lanes for the rest of us. Below find the full text of the letter.

***

Mr. President:

Earlier this week, you made a strong statement in support of net neutrality by saying:

“One of the issues around net neutrality is whether you are creating different rates or charges for different content providers. That’s the big controversy here. So you have big, wealthy media companies who might be willing to pay more and also charge more for spectrum, more bandwidth on the Internet so they can stream movies faster.

I personally, the position of my administration, as well as a lot of the companies here, is that you don’t want to start getting a differentiation in how accessible the Internet is to different users. You want to leave it open so the next Google and the next Facebook can succeed.”

We want to thank you for making your support for net neutrality clear and we are counting on you to take action to ensure equality on the Internet. A level playing field has been vital for innovation, opportunity and freedom of expression, and we agree that the next big thing will not succeed without it. We need to avoid a future with Internet slow lanes for everybody except for a few large corporations who can pay for faster service.

Like you, we believe in preserving an open Internet, where Internet service providers treat data equally, regardless of who is creating it and who is receiving it. Your vision of net neutrality is fundamentally incompatible with FCC’s plan, which would explicitly allow for paid prioritization. The only way for the FCC to truly protect an open Internet is by using its clear Title II authority. Over the next few months, we need your continued and vocal support for fair and meaningful net neutrality rules. Our organizations will continue to pressure the FCC to put forth solidly based rules, and will continue to encourage you and other elected officials to join us in doing so.

Thank you again for standing up for the open Internet so that small businesses and people everywhere have a fair shot.

Signed,

ACLU, 18 Million Rising, Center for Media Justice, Center for Rural Strategies, ColorOfChange, Common Cause, Consumers Union, CREDO, Daily Kos, Demand Progress, Democracy for America, EFF, Engine, Enjambre Digital, Etsy, EveryLibrary, Fandor, Fight for the Future, Free Press, Future of Music Coalition, Greenpeace, Kickstarter, Louder, Media Action Grassroots Network, Media Alliance, Media Literacy Project, Media Mobilizing Project, MoveOn.org, Mozilla, Museums and the Web, National Alliance for Media Arts and Culture, National Hispanic Media Coalition, Open Technology Institute, OpenMedia International, Presente.org, Progressive Change Campaign Committee, Progressives United, Public Knowledge, Reddit, Rural Broadband Policy Group, SumOfUs, The Student Net Alliance, ThoughtWorks, United Church of Christ, OC Inc., Women’s Institute for Freedom of the Press, Women’s Media Center, Y Combinator

Categorieën: Mozilla-nl planet

Matt Brubeck: Let's build a browser engine!

vr, 08/08/2014 - 18:40

I’m building a toy HTML rendering engine, and I think you should too. This is the first in a series of articles describing my project and how you can make your own. But first, let me explain why.

You’re building a what?

Let’s talk terminology. A browser engine is the portion of a web browser that works “under the hood” to fetch a web page from the internet, and translate its contents into forms you can read, watch, hear, etc. Blink, Gecko, WebKit, and Trident are browser engines. In contrast, the the browser’s own UI—tabs, toolbar, menu and such—is called the chrome. Firefox and SeaMonkey are two browsers with different chrome but the same Gecko engine.

A browser engine includes many sub-components: an HTTP client, an HTML parser, a CSS parser, a JavaScript engine (itself composed of parsers, interpreters, and compilers), and much more. The many components involved in parsing web formats like HTML and CSS and translating them into what you see on-screen are sometimes called the layout engine or rendering engine.

Why a “toy” rendering engine?

A full-featured browser engine is hugely complex. Blink, Gecko, WebKit—these are millions of lines of code each. Even younger, simpler rendering engines like Servo and WeasyPrint are each tens of thousands of lines. Not the easiest thing for a newcomer to comprehend!

Speaking of hugely complex software: If you take a class on compilers or operating systems, at some point you will probably create or modify a “toy” compiler or kernel. This is a simple model designed for learning; it may never be run by anyone besides the person who wrote it. But making a toy system is a useful tool for learning how the real thing works. Even if you never build a real-world compiler or kernel, understanding how they work can help you make better use of them when writing your own programs.

So, if you want to become a browser developer, or just to understand what happens inside a browser engine, why not build a toy one? Like a toy compiler that implements a subset of a “real” programming language, a toy rendering engine could implement a small subset of HTML and CSS. It won’t replace the engine in your everyday browser, but should nonetheless illustrate the basic steps needed for rendering a simple HTML document.

Try this at home.

I hope I’ve convinced you to give it a try. This series will be easiest to follow if you already have some solid programming experience and know some high-level HTML and CSS concepts. However, if you’re just getting started with this stuff, or run into things you don’t understand, feel free to ask questions and I’ll try to make it clearer.

Before you start, a few remarks on some choices you can make:

On Programming Languages

You can build a toy layout engine in any programming language. Really! Go ahead and use a language you know and love. Or use this as an excuse to learn a new language if that sounds like fun.

If you want to start contributing to major browser engines like Gecko or WebKit, you might want to work in C++ because it’s the main language used in those engines, and using it will make it easier to compare your code to theirs. My own toy project, robinson, is written in Rust. I’m part of the Servo team at Mozilla, so I’ve become very fond of Rust programming. Plus, one of my goals with this project is to understand more of Servo’s implementation. (I’ve written a lot of browser chrome code, and a few small patches for Gecko, but before joining the Servo project I knew nothing about many areas of the browser engine.) Robinson sometimes uses simplified versions of Servo’s data structures and code. If you too want to start contributing to Servo, try some of the exercises in Rust!

On Libraries and Shortcuts

In a learning exercise like this, you have to decide whether it’s “cheating” to use someone else’s code instead of writing your own from scratch. My advice is to write your own code for the parts that you really want to understand, but don’t be shy about using libraries for everything else. Learning how to use a particular library can be a worthwhile exercise in itself.

I’m writing robinson not just for myself, but also to serve as example code for these articles and exercises. For this and other reasons, I want it to be as tiny and self-contained as possible. So far I’ve used no external code except for the Rust standard library. (This also side-steps the minor hassle of getting multiple dependencies to build with the same version of Rust while the language is still in development.) This rule isn’t set in stone, though. For example, I may decide later to use a graphics library rather than write my own low-level drawing code.

Another way to avoid writing code is to just leave things out. For example, robinson has no networking code yet; it can only read local files. In a toy program, it’s fine to just skip things if you feel like it. I’ll point out potential shortcuts like this as I go along, so you can bypass steps that don’t interest you and jump straight to the good stuff. You can always fill in the gaps later if you change your mind.

First Step: The DOM

Are you ready to write some code? We’ll start with something small: data structures for the DOM. Let’s look at robinson’s dom module.

The DOM is a tree of nodes. A node has zero or more children. (It also has various other attributes and methods, but we can ignore most of those for now.)

struct Node { // data common to all nodes: children: Vec<Node>, // data specific to each node type: node_type: NodeType, }

There are several node types, but for now we will ignore most of them and say that a node is either an Element or a Text node. In a language with inheritance these would be subtypes of Node. In Rust they can be an enum (Rust’s keyword for a “tagged union” or “sum type”):

enum NodeType { Text(String), Element(ElementData), }

An element includes a tag name and any number of attributes, which can be stored as a map from names to values. Robinson doesn’t support namespaces, so it just stores tag and attribute names as simple strings.

struct ElementData { tag_name: String, attributes: AttrMap, } type AttrMap = HashMap<String, String>;

Finally, some constructor functions to make it easy to create new nodes:

fn text(data: String) -> Node { Node { children: vec![], node_type: Text(data) } } fn elem(name: String, attrs: AttrMap, children: Vec<Node>) -> Node { Node { children: children, node_type: Element(ElementData { tag_name: name, attributes: attrs, }) } }

And that’s it! A full-blown DOM implementation would include a lot more data and dozens of methods, but this is all we need to get started. In the next article, we’ll add a parser that turns HTML source code into a tree of these DOM nodes.

Exercises

These are just a few suggested ways to follow along at home. Do the exercises that interest you and skip any that don’t.

  1. Start a new program in the language of your choice, and write code to represent a tree of DOM text nodes and elements.

  2. Install the latest version of Rust, then download and build robinson. Open up dom.rs and extend NodeType to include additional types like comment nodes.

  3. Write code to pretty-print a tree of DOM nodes.

References

Here’s a short list of “small” open source web rendering engines. Most of them are many times bigger than robinson, but still way smaller than Gecko or WebKit. WebWhirr, at 2000 lines of code, is the only other one I would call a “toy” engine.

You may find these useful for inspiration or reference. If you know of any other similar projects—or if you start your own—please let me know!

Categorieën: Mozilla-nl planet

Irving Reid: Telemetry Results for Add-on Compatibility Check

vr, 08/08/2014 - 17:30

Earlier this year (in Firefox 32), we landed a fix for bug 760356, to reduce how often we delay starting up the browser in order to check whether all your add-ons are compatible. We landed the related bug 1010449 in Firefox 31 to gather telemetry about the compatibility check, so that we could to before/after analysis.

Background

When you upgrade to a new version of Firefox, changes to the core browser can break add-ons. For this reason, every add-on comes with metadata that says which versions of Firefox it works with. There are a couple of straightforward cases, and quite a few tricky corners…

  • The add-on is compatible with the new Firefox, and everything works just fine.
  • The add-on is incompatible and must be disabled.
    • But maybe there’s an updated version of the add-on available, so we should upgrade it.
    • Or maybe the add-on was installed in a system directory by a third party (e.g. an antivirus toolbar) and Firefox can’t upgrade it.
  • The add-on says it’s compatible, but it’s not – this could break your browser!
    • The add-on author could discover this in advance and publish updated metadata to mark the add-on incompatible.
    • Mozilla could discover the incompatibility and publish a metadata override at addons.mozilla.org to protect our users.
  • The add-on says it’s not compatible, but it actually is.
    • Again, either the add-on author or Mozilla can publish a compatibility override.

We want to keep as many add-ons as possible enabled, because our users love (most of) their add-ons, while protecting users from incompatible add-ons that break Firefox. To do this, we implemented a very conservative check every time you update to a new version. On the first run with a new Firefox version, before we load any add-ons we ask addons.mozilla.org *and* each add-on’s update server whether there is a metadata update available, and whether there is a newer version of the add-on compatible with the new Firefox version. We then enable/disable based on that updated metadata, and offer the user the chance to upgrade those add-ons that have new versions available. Once this is done, we can load up the add-ons and finish starting up the browser.

This check involves multiple network requests, so it can be rather slow. Not surprisingly, our users would rather not have to wait for these checks, so in bug 760356 we implemented a less conservative approach:

  • Keep track of when we last did a background add-on update check, so we know how out of date our metadata is.
  • On the first run of a new Firefox version, only interrupt startup if the metadata is too out of date (two days, in the current implementation) *or* if some add-ons were disabled by this Firefox upgrade but are allowed to be upgraded by the user.
Did it work?

Yes! On the Aurora channel, we went from interrupting 92.7% of the time on the 30 -> 31 upgrade  (378091 out of
407710 first runs reported to telemetry) to 74.8% of the time (84930 out of 113488) on the 31 -> 32 upgrade, to only interrupting 16.4% (10158 out of 61946) so far on the 32 -> 33 upgrade.

The change took effect over two release cycles; the new implementation was in 32, so the change from “interrupt if there are *any* add-ons the user could possibly update” to “interrupt if there is a *newly disabled* add-on the user could update” is in effect for the 31 -> 32 upgrade. However, since we didn’t start tracking the metadata update time until 32, the “don’t interrupt if the metadata is fresh” change wasn’t effective until the 32 -> 33 upgrade. I wish I had thought of that at the time; I would have added the code to remember the update time into the telemetry patch that landed in 31.

Cool, what else did we learn?

On Aurora 33, the distribution of metadata age was:

Age (days) Sessions < 1 37207 1 9656 2 2538 3 997 4 535 5 319 6 – 10 565 11 – 15 163 16 – 20 94 21 – 25 69 26 – 30 82 31 – 35 50 36 – 40 48 41 – 45 53 46 – 50 6

so about 88% of profiles had fresh metadata when they upgraded. The tail is longer than I expected, though it’s not too thick. We could improve this by forcing a metadata ping (or a full add-on background update) when we download a new Firefox version, but we may need to be careful to do it in a way that doesn’t affect usage statistics on the AMO side.

What about add-on upgrades?

We also started gathering detailed information about how many add-ons are enabled or disabled during various parts of the upgrade process. The measures are all shown as histograms in the telemetry dashboard at http://telemetry.mozilla.org;

SIMPLE_MEASURES_ADDONMANAGER_XPIDB_STARTUP_DISABLED
The number of add-ons (both user-upgradeable and non-upgradeable) disabled during the upgrade because they are not compatible with the new version.
SIMPLE_MEASURES_ADDONMANAGER_APPUPDATE_DISABLED
The number of user-upgradeable add-ons disabled during the upgrade.
SIMPLE_MEASURES_ADDONMANAGER_APPUPDATE_METADATA_ENABLED
The number of add-ons that changed from disabled to enabled because of metadata updates during the compatibility check.
SIMPLE_MEASURES_ADDONMANAGER_APPUPDATE_METADATA_DISABLED
The number of add-ons that changed from enabled to disabled because of metadata updates during the compatibility check.
SIMPLE_MEASURES_ADDONMANAGER_APPUPDATE_UPGRADED
The number of add-ons upgraded to a new compatible version during the add-on compatibility check.
SIMPLE_MEASURES_ADDONMANAGER_APPUPDATE_UPGRADE_DECLINED
The number of add-ons that had upgrades available during the compatibility check, but the user chose not to upgrade.
SIMPLE_MEASURES_ADDONMANAGER_APPUPDATE_UPGRADE_FAILED
The number of add-ons that appeared to have upgrades available, but the attempt to install the upgrade failed.

For these values, we got good telemetry data from the Beta 32 upgrade. The counts represent the number of Firefox sessions that reported that number of affected add-ons (e.g. 3170 Telemetry session reports said that 2 add-ons were XPIDB_DISABLED by the upgrade):

Add-ons affected XPIDB DISABLED APPUPDATE DISABLED METADATA ENABLED METADATA DISABLED UPGRADED DECLINED FAILED 0 2.6M 2.6M  2.6M 2.6M  2.6M 2.6M  2.6M 1 36230 7360  59240 14780  824  121  98 2 3170 1570  2 703  5  1  0 3 648 35  0 43  1  0  0 4 1070 14  1 6  0  0  0 5 53 20  0 0  0  0  0 6 157 194  0 0  0  0  0 7+ 55 9  0 1  0  0  0

The things I find interesting here are:

  • The difference between XPIDB disabled and APPUPDATE disabled is (roughly) the add-ons installed in system directories by third party installers. This implies that 80%-ish of add-ons made incompatible by the upgrade are 3rd party installs.
  • upgraded + declined + failed is (roughly) the add-ons a user *could* update during the browser upgrade, which works out to fewer than one in 2000 browser upgrades having a useful add-on update available. I suspect this is because most add-on updates have already been performed by our regular background update. In any case, to me this implies that further work on updating add-ons during browser upgrade won’t improve our user experience much.
Categorieën: Mozilla-nl planet

Mozilla WebDev Community: Webdev Extravaganza – August 2014

vr, 08/08/2014 - 16:22

Once a month, web developers from across Mozilla gather to summon cyber phantoms and techno-ghouls in order to learn their secrets. It’s also a good opportunity for us to talk about what we’ve shipped, share what libraries we’re working on, meet newcomers, and just chill. It’s the Webdev Extravaganza! Despite the danger of being possessed, the meeting is open to the public; you should stop by!

You can check out the wiki page that we use to organize the meeting, check out the Air Mozilla recording, or amuse yourself with the wild ramblings that constitute the meeting notes. Or, even better, read on for a more PG-13 account of the meeting.

Shipping Celebration

The shipping celebration is for anything we finished and deployed in the past month, whether it be a brand new site, an upgrade to an existing one, or even a release of a library.

Peep 1.3 is out!

There’s a new release of ErikRose‘s peep out! Peep is essentially pip, which installs Python packages, but with the ability to check downloaded packages against cryptographic hashes to ensure you’re receiving the same code each time you install. The latest version now passes through most arguments for pip install, supports Python 3.4, and installs a secondary script tied to the active Python version.

Open-source Citizenship

Here we talk about libraries we’re maintaining and what, if anything, we need help with for them.

Contribute.json

pmac and peterbe, with feedback from the rest of Mozilla Webdev, have created contribute.json, a JSON schema for open-source project contribution data. The idea is to make contribute.json available at the root of every Mozilla site to make it easier for potential contributors and for third-party services to find details on how to contribute to that site. The schema is still a proposal, and feedback or suggestions are very welcome!

New Hires / Interns / Volunteers / Contributors

Here we introduce any newcomers to the Webdev group, including new employees, interns, volunteers, or any other form of contributor.

Name IRC Nick Project John Whitlock jwhitlock Web Platform Compatibility API Mark Lavin mlavin Mobile Partners Roundtable

The Roundtable is the home for discussions that don’t fit anywhere else.

How do you feel about Playdoh?

peterbe brought up the question of what to do about Playdoh, Mozilla’s Django-based project template for new sites. Many sites that used to be based on Playdoh are removing the components that tie them to the semi-out-of-date library, such as depending on playdoh-lib for library updates. The general conclusion was that many people want Playdoh to be rewritten or updated to address long-standing issues, such as:

  • Libraries are currently either installed in the repo or included via git submodules. A requirements.txt-based approach would be easier for users.
  • Many libraries included with Playdoh were made to implement features that Django has since included, making them redundant.
  • Django now supports project templates, making the current install method of using funfactory to clone Playdoh obsolete.

pmac has taken responsibility as a peer on the Playdoh module to spend some time extracting improvements from Bedrock into Playdoh.

Helping contributors via Cloud9

jgmize shared his experience making Bedrock run on the Cloud9 platform. The goal is to make it easy for contributors to spin up an instance of Bedrock using a free Cloud9 account, allowing them to edit and submit pull requests without having to go through the complex setup instructions for the site. jgmize has been dogfooding using Cloud9 as his main development environment for a few weeks and has had positive results using it.

If you’re interested in this approach, check out Cloud9 or ask jgmize for more information.

Unfortunately, we were unable to learn any mystic secrets from the ghosts that we were able to summon, but hey: there’s always next month!

If you’re interested in web development at Mozilla, or want to attend next month’s Extravaganza, subscribe to the dev-webdev@lists.mozilla.org mailing list to be notified of the next meeting, and maybe send a message introducing yourself. We’d love to meet you!

See you next month!

Categorieën: Mozilla-nl planet

Frederic Plourde: Gecko on Wayland

vr, 08/08/2014 - 15:44

266px-Wayland_Logo.svgAt Collabora, we’re always on the lookout for cool opportunities involving Wayland and we noticed recently that Mozilla had started to show some interest in porting Firefox to Wayland. In short, the Wayland display server is becoming very popular for being lightweight, versatile yet powerful and is designed to be a replacement for X11. Chrome and Webkit already got Wayland ports and we think that Firefox should have that too.

Some months ago, we wrote a simple proof-of-concept basically starting from actual Gecko’s GTK3 paths and stripping all the MOZ_X11 ifdefs out of the way. We did a bunch of quick hacks fixing broken stuff but rather easily and quickly (couple days), we got Firefox to run on Weston (Wayland official reference compositor). Ok, because of hard X11 dependencies, keyboard input was broken and decorations suffered a little, but that’s a very good start! Take a look at the below screenshot :)

firefox-on-wayland


Categorieën: Mozilla-nl planet

Pagina's