mozilla

Mozilla Nederland LogoDe Nederlandse
Mozilla-gemeenschap

Mozilla Marketing Engineering & Ops Blog: MozMEAO SRE Status Report - August 1, 2017

Mozilla planet - di, 01/08/2017 - 02:00

Here’s what happened on the MozMEAO SRE team from July 25th - August 1st.

Current work MDN Migration

@metadave and @escattone are working on migrating MDN from the SCL3 datacenter to AWS.

MDN misc Decommissioning the Virginia cluster

The last remaining services running on the Virginia cluster have been moved to other regions or hosting options:

Links
Categorieën: Mozilla-nl planet

Emma Humphries: Firefox Triage Report 2017-07-31

Mozilla planet - di, 01/08/2017 - 01:44

It's the weekly report on the state of triage in Firefox-related components.

Hotspots

The components with the most untriaged bugs have not changed substantially from last week, and the number of untriaged bugs has gone up.

**Rank** **Component** **Last Week** **This Week** ---------- ------------------------------ --------------- --------------- 1 Core: JavaScript Engine 431 446 2 Firefox for Android: General 406 410 3 Core: Build Config 399 421 4 Core: General 234 239 5 Firefox: General 234 238 6 Core: Graphics 199 194 7 Core: XPCOM -- 178 8 Core: Disability Access APIs 173 162 All Components 8,403 8,543

Please make sure you’ve made it clear what, if anything will happen with these bugs.

Not sure how to triage? Read https://wiki.mozilla.org/Bugmasters/Process/Triage.

Next Release

Wednesday is Merge Day for 56.

There are still 4,400 bugs filed during the Firefox 55 release cycle without decisions.

**Version** 55 55 55 55 ------------------------------------- ------------ ------------ ------------ ------------ **Date** 2017-07-10 2017-07-17 2017-07-24 2017-07-31 **Untriaged this Cycle** 4,525 4,451 4,317 4,479 **Unassigned Untriaged this Cycle** 3,742 3,682 3,517 3,674 **Affected this Cycle** 111 126 139 **Enhancements** 102 107 91 103 **Orphaned P1s** 199 193 183 192 **Stalled P1s** 195 173 159 179

What should we do with these bugs? Bulk close them? Make them into P3s? Bugs without decisions add noise to our system, cause despair in those trying to triage bugs, and leaves the community wondering if we listen to them.

Note

I’m traveling from the 1st to the 17th. I plan to send out updates on the 7th and 14th, but they may be delayed.

Methods and Definitions

In this report I talk about bugs in Core, Firefox, Firefox for Android, Firefox for IOs, and Toolkit which are unresolved, not filed from treeherder using the intermittent-bug-filer account*, and have no pending needinfos.

By triaged, I mean a bug has been marked as P1 (work on now), P2 (work on next), P3 (backlog), or P5 (will not work on but will accept a patch).

A triage decision is not the same as a release decision (status and tracking flags.)

https://mozilla.github.io/triage-report/#report

Age of Untriaged Bugs

The average age of a bug filed since June 1st of 2016 which has gone without triage.

https://mozilla.github.io/triage-report/#date-report

Untriaged Bugs in Current Cycle

Bugs filed since the start of the Firefox 55 release cycle (March 6th, 2017) which do not have a triage decision.

https://mzl.la/2u1R7gA

Recommendation: review bugs you are responsible for (https://bugzilla.mozilla.org/page.cgi?id=triage_owners.html) and make triage decision, or RESOLVE.

Untriaged Bugs in Current Cycle Affecting Next Release

Bugs marked status_firefox56 = affected and untriaged.

https://mzl.la/2u2GCcG

Enhancements in Release Cycle

Bugs filed in the release cycle which are enhancement requests, severity = enhancement, and untriaged.

​Recommendation: ​product managers should review and mark as P3, P5, or RESOLVE as WONTFIX.

High Priority Bugs without Owners

Bugs with a priority of P1, which do not have an assignee, have not been modified in the past two weeks, and do not have pending needinfos.

https://mzl.la/2u1VLem

Recommendation: review priorities and assign bugs, re-prioritize to P2, P3, P5, or RESOLVE.

Stalled High Priority Bugs

There 159 bugs with a priority of P1, which have an assignee, but have not been modified in the past two weeks.

https://mzl.la/2u2poMJ

Recommendation: review assignments, determine if the priority should be changed to P2, P3, P5 or RESOLVE.

* New intermittents are filed as P5s, and we are still cleaning up bugs after this change, See https://bugzilla.mozilla.org/show_bug.cgi?id=1381587, https://bugzilla.mozilla.org/show_bug.cgi?id=1381960, and https://bugzilla.mozilla.org/show_bug.cgi?id=1383923

If you have questions or enhancements you want to see in this report, please reply to me here, on IRC, or Slack and thank you for reading.



comment count unavailable comments
Categorieën: Mozilla-nl planet

Alexis Metaireau: Comment installer Kinto manuellement chez Alwaysdata

Mozilla planet - di, 01/08/2017 - 00:00
Qu'est-ce que Alwaysdata ?

Alwaysdata est un hébergeur français qui vous permet de payer pour un espace et d'y héberger autant de sites que vous le souhaitez en toute liberté.

L'équipe est à l'écoute et très sympathique ce qui en fait un service de grande qualité.

Une offre de découverte gratuite vous permet de tester le service avant de souscrire à un abonnement.

Créer un compte chez Alwaysdata

Pour créer un compte, rendez-vous ici: https://www.alwaysdata.com/fr/signup/

Vous allez renseigner vos coordonnées, votre email et choisir un mot de passe ainsi qu'un identifiant pour votre compte.

Cet identifiant vous sera nécessaire par la suite. Nous l'appelons ID_ALWAYSDATA.

Installation automatique de Kinto chez Alwaysdata

Avant toute chose, sachez qu'il existe un script qui s'occupe de faire l'installation pour vous automatiquement.

Vous pouvez le trouver ici : https://kinto.github.io/kinto-alwaysdata/

Installation manuelle de Kinto chez Alwaysdata Activez Python 3.6.0

Dans le menu Environnement > Python vérifiez que votre version de Python par défaut soit bien au moins Python 3.6.0

Configurer la version 3.6.0 de Python chez Alwaysdata. Créez une base de données PostgreSQL

Via l'administration de Alwaysdata vous pouvez créer un utilisateur et une base de données PostgreSQL, retenez bien l'identifiant et le mot de passe car vous en aurez besoin pour votre fichier de configuration kinto.ini

Créer un utilisateur et une base de données PostgreSQL Activer la connexion SSH

L'installation de Kinto chez Alwaysdata se fait à l'aide de la connexion SSH.

Commencez par vous connecter à l'administration et à activer votre compte SSH.

Activer l'utilisateur SSH. Créer le fichier de configuration de Kinto

Créez le fichier kinto/kinto.ini

Vous pouvez utiliser ce modèle de fichier ini en veillant bien à remplacer les champs de type {...} et à configurer votre base de donnée PostgreSQL (appellée ici DBname) avec votre identifiant et mot de passe. Il est conseillé de créer un utilisateur spécifique pour Kinto (ici DBuser) en plus de votre compte AlwaysData (ici ADuser).

  • {id_alwaysdata} -> ADuser_DBuser
  • {password} -> le mot de passe
  • {postgresql_host} -> postgresql-ADuser.alwaysdata.net
  • {prefixed_username} -> ADuser_DBname

Vous devez également générer une clé secrète pour {hmac_secret}. Pour ce faire vous pouvez utiliser la commande uuidgen.

Pas besoin de changer {bucket_id}/{collection_id}.

Créer le lanceur du serveur

Créez le fichier kinto/kinto.wsgi

Vous pouvez utiliser ce modèle de fichier WSGI

Veillez à ce que le fichier soit exécutable :

chmod +x kinto/kinto.wsgi Créer un environment virtuel Python et installez Kinto dedans

Vous pouvez vous connecter à votre compte SSH depuis le terminal :

ssh ID_ALWAYSDATA@ssh-ID_ALWAYSDATA.alwaysdata.net

Remplacez ID_ALWAYSDATA par l'identifiant de votre compte alwaysdata.

virtualenv kinto/venv/ kinto/venv/bin/pip install kinto[postgresql] kinto-attachment kinto/venv/bin/kinto migrate --ini kinto/kinto.ini Associer votre Kinto à un nom de domaine

Par défaut, Alwaysdata vous procure le sous domaine suivant: ID_ALWAYSDATA.alwaysdata.net

Mais vous pouvez également lier votre propre nom de domaine, soit en délégant la gestion du DNS à Alwaysdata soit en configurant la zone DNS pour pointer vers les serveurs de Alwaysdata.

Dans tous les cas Alwaysdata pourra vous fournir un certificat SSL gratuit via Let's Encrypt.

Ajoutez un nouveau site Configurer le site internet Informations

Entrez les informations concernant le site.

  • Nom : Kinto
  • Adresses: ID_ALWAYSDATA.alwaysdata.net
Nom du site et noms de domaine associés. Configuration

Configurer le site Python :

  • Type : Python WSGI
  • Chemin de l'application : /kinto/kinto.wsgi
  • Version de Python : Version par défaut
  • Répertoire du virtualenv : /kinto/venv/
  • Chemins statiques : /attachments/=attachments
Configuration du site Python SSL
  • Forcer le SSL : Oui
Configuration SSL Se connecter

Vous pouvez maintenant vous connecter au site et vérifier que tout fonctionne :

Configurer un administrateur

Connectez-vous à la kinto-admin : https://ID_ALWAYSDATA.alwaysdata.net/v1/admin/

Choisissez un login/password et connectez-vous. Vous allez obtenir un identifiant unique pour votre administrateur: basicauth:......

Vous pouvez ensuite modifier la configuration de votre serveur Kinto pour n'autoriser la création de bucket que par cet utilisateur (et vous protéger des usages abusifs de votre service.)

kinto.bucket_create_principals = basicauth:800748fb8963fd00...a3420db354570a4e kinto.bucket_write_principals = basicauth:800748fb8963fd00...a3420db354570a4e kinto.bucket_read_principals = basicauth:800748fb8963fd00...a3420db354570a4e kinto.collection_create_principals = basicauth:800748fb8963fd00...a3420db354570a4e kinto.collection_write_principals = basicauth:800748fb8963fd00...a3420db354570a4e kinto.collection_read_principals = basicauth:800748fb8963fd00...a3420db354570a4e kinto.group_create_principals = basicauth:800748fb8963fd00...a3420db354570a4e kinto.group_write_principals = basicauth:800748fb8963fd00...a3420db354570a4e kinto.group_read_principals = basicauth:800748fb8963fd00...a3420db354570a4e kinto.record_create_principals = basicauth:800748fb8963fd00...a3420db354570a4e kinto.record_write_principals = basicauth:800748fb8963fd00...a3420db354570a4e kinto.record_read_principals = basicauth:800748fb8963fd00...a3420db354570a4e

Si vous utilisez un autre système d'authentification, vous aurez probablement d'autres identifiants ne commençant pas par basicauth mais vous pouvez quand même utiliser ces identifiants dans votre fichier de configuration.

Vous pouvez également créer un groupe et utiliser ce groupe comme identifiant.

N'hésitez pas à nous demander de l'aide sur Slack ou IRC.

Categorieën: Mozilla-nl planet

David Humphrey: Service Workers and Cache Storage in Thimble

Mozilla planet - ma, 31/07/2017 - 20:03

tldr; During the spring and summer, I've been working on rewriting Thimble to use Service Workers and CacheStorage. Now that we've shipped it, I wanted to talk about how it works and what I learned.

On Twitter a few weeks back, I was reminiscing with Dave Herman and Brendan about web standards we worked on in Firefox a decade ago. Dave made the point that the 2017 web is still exciting and accelerating, especially with things like Service Workers. I couldn't agree more.

I've been waiting to be able to do what Service Workers provide forever. For years we heard about all the problems that they would solve, and I was a believer from day one. Imagine having a web server running in your browser, and able to serve any content type? Sign me up!

Now that they're here (Chrome and Firefox), most of what I see written relates to using Service Workers in order to build Progressive Web Apps (PWA). PWAs are great, but there are other things you can do with Service Workers. For example, you can use them to serve user generated content on the fly without needing a server.

Four years ago I started working on this problem. The goal was to be able to allow users to create arbitrary web sites/apps in a full editor, and then run them without needing a server. In a classroom setting, the network is often non-functional and I wanted to make it possible to just use a browser to do everything. I wanted a static web page to provide the following:

  • filesystem (we wrote a browser version of node's fs module called Filer)
  • full editor (we forked Adobe's Brackets editor and made it run in the browser)
  • "web server" to host static content from the filesystem
  • "web browser" to run the content (an iframe with a postMessage API)

My first attempt was done before Service Workers were readily available, and didn't use them. Also, because I wanted to support as many browsers as possible, I needed a "progressive" solution for providing these features: IE11 will never have Service Workers, for example, while Safari probably will (see how optimistic I am?).

This initial version made heavy use of Blob, URL.createObjectURL(), and DOMParser. The idea was to leverage the fact that all filesystem operations are asynchronous, which provides an opportunity to also create and cache Blob URLs for files, mapped to filenames. When a user tries to browse to index.html, we instead serve the cached Blob URL for that file, and the browser does the rest. For content within an HTML or CSS file, we simply rewrote all the relative links to use Blob URLs, such that image/background.png becomes blob:https%3A//mozillathimblelivepreview.net/264a3524-5316-47e5-a835-451e78247678.

It worked great for many years as part of Thimble, allowing students and teachers to learn web programming without having to do any installation or setup.

However, I was never satisfied with this approach, especially since I knew Service Workers could solve one or our largest problems: we couldn't reliably rewrite filenames to Blob URLs in dynamic JavaScript. Over and over again users would file bugs that would say, "I tried to use JS to load an image, and it didn't work!" Not being able to fully support JS at the network level was no good, and it frustrated me to not fully support all of the web.

So I slowly worked away at adding a secondary URL cache implementation based on Service Workers. My idea was to use the same pathways that we already had for writing Blob URLs for file content, but instead of rewriting links, we'd just let a Service Worker serve the content from our filesystem instead of the network.

There was one problem for which I couldn't see an elegant solution. I needed a way for the editor to share the content with the Service Worker, since we don't always serve pages from disk (e.g., IndexedDB): if a user is editing a file, we serve an instrumented version instead, which does things like live highlighting, hot reloads, etc. It's possible for a user to have many Thimble tabs open at once, but there will only be one Service Worker. I didn't want the Service Worker to have to poll every open Thimble window before it found the right content.

The solution turned out to be really easy. Instead of somehow posting data between the DOM windows and the Service Worker, I could instead use storage shared between both in the form of CacheStorage. Whoever had the brilliant insight to make this available to both window and the Service Worker, I salute your forward thinking!

Using CacheStorage I was now able to create and cache URLs in the editor's DOM every time content changed on disk, or whenever the live version of a resource was updated. Then, the browser could request the resource and the Service Worker could fulfill the request using the cached URL, without needing to know where it came from in the first place. It's beautifully simple and works flawlessly.

For example, here's the React version of the TodoMVC app running unmodified in Thimble, which uses lots of dynamic JS:

And here's a great HTML5 Game Workshop from Mozilla Hacks, which uses JS, audio, and dynamic images/sprites:

At a high level, the code on the editor/filesystem side looks like this:

var data; // Typed Array of bytes (PNG, CSS file, etc) var type; // some content type, maybe 'application/octet-binary' var blob = new Blob([data], {type: type}); var response = new Response(blob, { status: 200, statusText: "Served from Offline Cache" }); var headers = new Headers(); headers.append("Content-Type", type); var url; // How you want to address this, any valid URL. var request = new Request(url, { method: "GET", headers: headers }); var cacheName; // Some unique name for this cache, maybe a URL window.caches .open(cacheName) .then(function(cache) { cache.put(request, response); }) .catch(callback); };

Inside the Service Worker, I do something like this, looking for the requested URL in the cache, and serving it if possible:

self.addEventListener("fetch", function(event) { event.respondWith( caches.match(url) .then(function(response) { // Either we have this file's response cached, or we should go to the network return response || fetch(event.request); }) .catch(function(err) { return fetch(event.request); }) ); });

HTML, CSS, JavaScript all work now, and so does audio, video, JSON, and any new thing that gets added to the web next year. Everything just works, thanks to Service Workers.

We quietly enabled this new cache layer on Thimble a month ago, and it's been working well. We do feature detection on Service Workers and Cache Storage, and browsers that support them get the new cache implementation, while browsers that don't continue to use or old Blob URL based layer.

After so many years of workarounds and complex hacks to make this work, I was amazed at how little code it took to get this working with the Service Worker and Cache Storage APIs. This was my first major app that used the new Service Worker stuff, and I found it took me a long time to adjust my mental model for web dev (the Service Worker lifecycle is complex and sometimes surprising), and my debugging workflow; the final code isn't hard, but the adjustment in how you build and test it is fairly significant. I've also found that it's been somewhat frustrating for my teammates who don't know as much about Service Workers, and run into various cache-related bugs that can be annoying.

My experience is that Service Workers do indeed live up to the hype I heard a decade ago, and I'd highly recommend you consider using them in your own apps. Don't get hung up on thinking of them only as a way to cache resources coming from the server as you load your pages. Instead, you should also consider them in cases where your user or client app generates content on the fly. You can serve this dynamic content alongside your hosted content, blurring the line between client/server. There's a lot of interesting problems you can solve this way, and all of it can be done with or without network.

Also, if you're teaching or learning web dev, please give Thimble a try. We've been adding a lot of cool features beyond just this architectural change I've described, and we think it's one of the best online coding tools available right now.

Categorieën: Mozilla-nl planet

Air Mozilla: Mozilla Weekly Project Meeting, 31 Jul 2017

Mozilla planet - ma, 31/07/2017 - 20:00

Mozilla Weekly Project Meeting The Monday Project Meeting

Categorieën: Mozilla-nl planet

Air Mozilla: Mozilla Weekly Project Meeting, 31 Jul 2017

Mozilla planet - ma, 31/07/2017 - 20:00

Mozilla Weekly Project Meeting The Monday Project Meeting

Categorieën: Mozilla-nl planet

William Lachance: mozregression's new mascot

Mozilla planet - ma, 31/07/2017 - 17:32

Spent a few hours this morning on a few housekeeping issues with mozregression. The web site was badly in need of an update (it was full of references to obsolete stuff like B2G and codefirefox.com) and the usual pile of fixes motivated a new release of the actual software. But most importantly, mozregression now has a proper application icon / logo, thanks to Victoria Wang!

One of the nice parts about working at Mozilla is the flexibility it offers to just hack on stuff that’s important, whether or not it’s part of your formal job description. Maintaining mozregression is pretty far outside my current set of responsibilities (or even interests), but I keep it going because it’s a key tool used by developers team here and no one else seems willing to take it over. Fortunately, tools like appveyor and pypi keep the time suckage to a mostly-reasonable level.

Categorieën: Mozilla-nl planet

Hacks.Mozilla.Org: Tour the latest features of the CSS Grid Inspector, July 2017

Mozilla planet - ma, 31/07/2017 - 16:00

We began work on a developer tool to help with understanding and using CSS Grid over a year ago. In March, we shipped the first version of a Grid Inspector in the Firefox DevTools along with CSS Grid. Now significant new features are landing in Firefox Nightly. Here’s a tour of what’s arrived in July 2017.

Download Firefox Nightly (if you don’t have it already) to get access to the latest and greatest, and to keep up with the continuing improvements.

Categorieën: Mozilla-nl planet

Robert O'Callahan: Selecting A Compression Algorithm For rr

Mozilla planet - ma, 31/07/2017 - 05:56

rr's traces are large. Memory-mapped files account for a lot of that, but the most efficient way to handle them is "zero copy" file cloning so we don't want to compress them during recording. Most of the rest is recordings of data copied into tracee address spaces by the kernel, and plus snapshots of registers, and these data are extremely compressible — often containing long runs of zeroes, for example. For a long time rr has used zlib to compress the non-mapped-file trace data, and zlib's 'deflate' algorithm often achieves compression of 8x or more.

Of course zlib is pretty old and significantly better algorithms exist, so now seems like a good time to reevaluate that decision. I used the Squash framework to compare zlib to two contenders, brotli and zstd, on actual rr trace data: a single, fairly short Firefox run, 828MB uncompressed, treated as independent 1MB chunks because that's what rr does. Here are the results:

I've omitted compression levels that took more than 20 seconds to compress the data. Currently rr uses zlib level 6, which takes just over 12 seconds to compress the data. Data compression occurs in parallel with the rest of recording, and uses multiple cores when it needs to, so in practice is seldom a performance bottleneck.

On this data, both brotli and zstd beat zlib by a significant margin, so we're definitely leaving some performance on the table by sticking with zlib. In particular, given the same time budget, zstd can shave 14% off the size of the non-mapped-file trace data, and brotli can shave off 17%. Alternatively, for the same trace size we could use much less CPU time — zstd level 1 compresses slightly better than zlib level 6, at 10x the speed!

For rr I think brotli level 5 is an obvious choice. For some reason there's a sudden improvement in compression at level 5, where it passes zstd and reaches roughly its optimal compression given a reasonable time budget. At level 5 we're shaving 17% off the current size and also taking 32% off the CPU time.

Apart from the performance, brotli also has a better licensing story. zstd has Facebook's standard patent license, which terminates if you sue Facebook for any patent infringement, and some organisations aren't comfortable with that. Apparently people have done patent searches and haven't found any Facebook patents covering zstd, but that is not wholly reassuring (while also being mystifying — if they're not applying for relevant patents, why not make that clear?). On the other hand, Google has made a clear commitment to license brotli royalty-free with no such conditions. Of course there could be third-party patents, but if they became a problem it would be easy to switch rr's algorithm (especially compared to the trouble they would cause for Web sites and browsers!).

Of course there are lots of other compression algorithms I could evaluate, but I guess if there are any further gains to be had, they would be very minor.

Update Unfortunately Ubuntu doesn't have a brotli library package. (Fedora does.) So, using brotli would mean everyone building rr on Ubuntu has to build brotli themselves first, or we vendor brotli into rr (or we do something truly insane like have rr pull and build brotli at build time if necessary). None of these approaches are appealing :-(. I guess there's also "rewrite rr in Rust so we can use cargo to have reasonable dependency management", which is appealing but not currently practical.

I'm leaning towards vendoring brotli into rr.

Categorieën: Mozilla-nl planet

Robert O'Callahan: Upstream Stable Kernels Work With rr Again

Mozilla planet - za, 29/07/2017 - 01:22

Greg K-H has released stable Linux kernels 3.18.63, 4.4.79, 4.9.40, and 4.12.4, containing a (backout) fix for the regression that broke rr. 4.13-rc2 also contains the fix. 4.11 was just declared end-of-life so it will not ever be officially fixed.

Obviously distros still have to produce kernel updates containing the fix, so we're not quite out of the woods yet, but that should be soon.

I'm holding off doing the rr 4.6.0 release until distro updates that work with rr have been out for a little while. To the (limited) extent possible I'd like to avoid people trying rr while it doesn't work on their kernel.

Categorieën: Mozilla-nl planet

Air Mozilla: Localization Conference - 20170727

Mozilla planet - vr, 28/07/2017 - 23:21

Localization Conference - 20170727 Localization Conference

Categorieën: Mozilla-nl planet

The Mozilla Blog: How Could You Use a Speech Interface?

Mozilla planet - vr, 28/07/2017 - 18:10

Last month in San Francisco, my colleagues at Mozilla took to the streets to collect samples of spoken English from passers-by. It was the kickoff of our Common Voice Project, an effort to build an open database of audio files that developers can use to train new speech-to-text (STT) applications.

What’s the big deal about speech recognition?

Speech is fast becoming a preferred way to interact with personal electronics like phones, computers, tablets and televisions. Anyone who’s ever had to type in a movie title using their TV’s remote control can attest to the convenience of a speech interface. According to one study, it’s three times faster to talk to your phone or computer than to type a search query into a screen interface.

Plus, the number of speech-enabled devices is increasing daily, as Google Home, Amazon Echo and Apple HomePod gain traction in the market. Speech is also finding its way into multi-modal interfaces, in-car assistants, smart watches, lightbulbs, bicycles and thermostats. So speech interfaces are handy — and fast becoming ubiquitous.

The good news is that a lot of technical advancements have happened in recent years, so it’s simpler than ever to create production-quality STT and text-to-speech (TTS) engines. Powerful tools like artificial intelligence and machine learning, combined with today’s more advanced speech algorithms, have changed our traditional approach to development. Programmers no longer need to build phoneme dictionaries or hand-design processing pipelines or custom components. Instead, speech engines can use deep learning techniques to handle varied speech patterns, accents and background noise – and deliver better-than-ever accuracy.

The Innovation Penalty

There are barriers to open innovation, however. Today’s speech recognition technologies are largely tied up in a few companies that have invested heavily in them. Developers who want to implement STT on the web are working against a fractured set of APIs and support. Google Chrome supports an STT API that is different from the one Apple supports in Safari, which is different from Microsoft’s.

So if you want to create a speech interface for a web application that works across all browsers, you would need to write code that would work with each of the various browser APIs. Writing and then rewriting code to work with every browser isn’t feasible for many projects, especially if the code base is large or complex.

There is a second option: You can purchase access to a non-browser-based API from Google, IBM or Nuance. Fees for this can cost roughly one cent per invocation. If you go this route, then you get one stable API to write to. But at one cent per utterance, those fees can add up quickly, especially if your app is wildly popular and millions of people want to use it. This option has a success penalty built into it, so it’s not a solid foundation for any business that wants to grow and scale.

Opening Up Speech on the Web

We think now is a good time to try to open up the still-young field of speech technology, so more people can get involved, innovate, and compete with the larger players. To help with that, the Machine Learning team in Mozilla Research is working on an open source STT engine. That engine will give Mozilla the ability to support STT in our Firefox browser, and we plan to make it freely available to the speech developer community, with no access or usage fees.

Secondly, we want to rally other browser companies to support the Web Speech API, a W3C community group specification that can allow developers to write speech-driven interfaces that utilize any STT service they choose, rather than having to select a proprietary or commercial service. That could open up a competitive market for smart home hubs–devices like the Amazon Echo that could be configured to communicate with one another, and other systems, for truly integrated speech-responsive home environments.

Where Could Speech Take Us?

Voice-activated computing could do a lot of good. Home hubs could be used to provide safety and health monitoring for ill or elderly folks who want to stay in their homes. Adding Siri-like functionality to cars could make our roads safer, giving drivers hands-free access to a wide variety of services, like direction requests and chat, so eyes stay on the road ahead. Speech interfaces for the web could enhance browsing experiences for people with visual and physical limitations, giving them the option to talk to applications instead of having to type, read or move a mouse.

It’s fun to think about where this work might lead. For instance, how might we use silent speech interfaces to keep conversations private? If your phone could read your lips, you could share personal information without the person sitting next to you at a café or on the bus overhearing. Now that’s a perk for speakers and listeners alike.Speech recognition using lip-reading

Want to participate? We’re looking for more folks to participate in both open source projects: STT engine development and the Common Voice application repository.

If programming is not your bag, you can always donate a few sentences to the Common Voice Project. You might read: “It made his heart rise into his throat” or “I have the diet of a kid who won $20.” Either way, it’s quick and fun. And it helps us offer developers an open source option that’s robust and affordable.

The post How Could You Use a Speech Interface? appeared first on The Mozilla Blog.

Categorieën: Mozilla-nl planet

Firefox Test Pilot: Test Pilot on Campus

Mozilla planet - vr, 28/07/2017 - 16:33

Test Pilot can never have too many good ideas! That was the original thought behind co-hosting a 12-week course with Tatung University (TTU)’s Department of Media Design in Taiwan. We expected to cultivate new ideas by guiding students through a simplified UX design process, and we did!

Scope of collaboration

In 2017, the Campus Seeding program in Taipei encouraged us to grow our influence in schools, so the Taipei UX team decided to work with TTU’s Professor Chia-yin Yu and Professor Peirong Cheng as our first step to creating a win-win collaboration: students learns new things, Mozilla collects new ideas. After several meetings, we finalized the course information and the modality of cooperation. There were 12 graduate students from diverse backgrounds who were divided into four teams.

Taipei UX team gave three in-class lessons covering our design process, including user research, a design workshop, prototyping, and user testing.

Professors shared relevant knowledge to form a compact program. Students completed assignments in Google Docs so that we could review and give feedback online. At the end of this semester, we asked students to give presentations in the Mozilla Taipei office and invited Firefox UX designers to critique.

The simplified UX design process covered during the course

1st lesson: User Research

For the first lesson, I gave an introduction to the Firefox browser and Test Pilot to help students understand who we are and what we do. Next, Ruby Hsu, our Senior User Researcher, gave a User Research 101 lesson and demonstrated interviews, breaking the research method down into step-by-step exercises. After students practiced several rounds of interviews with each other, the assignment was to put what they learned to use and deliver a report which contained user needs and insights for the next lesson: design workshop.

Ruby demonstrating advanced interview skills

2nd lesson: Design Workshop

Juwei Huang, our Senior User Experience Designer, with support from me, UX designers Tina Hsieh and Tori Chen, prepared a series of brainstorming tools to help students diverge from their reports and converge to concrete design proposals. With students’ research reports posted on the wall, the four of us led each team to practice various brainstorming exercises, including affinity diagramming, How Might We, and 3–12–3 brainstorming.

Each team got a UX designer-mentor

3rd lesson: Prototyping & User Testing

In the last lesson, Mark Liang, our UX Designer/Prototyper, introduced different prototyping instruments. Mark also elaborated on the importance and performance of user testing from Ruby’s lesson. From instruction to execution, we guided students to build paper prototypes based on their proposals and tested them with each other.

Students having fun working on paper prototypes

Final challenge: Presentations

At the end of the semester, we invited students to the Mozilla Taipei office. We gave an office tour to talk about Mozilla’s vision, manifesto, and the history of Taipei office. More importantly, students experienced how we review and critique at Mozilla by presenting their final presentations and what they learned. The participation from various Mozillians was invaluable. John Gruen, Test Pilot Product Manager, reviewed presentations from the product perspective. Philipp Sackl, Firefox Product Design Lead and Michael Verdi, Firefox Senior Interaction Designer, shared more international perspectives, not to mention the helpful critique from Taipei UX team members.

Students listening to Mozilla staff during critique

Looking ahead

To wrap up, it was a fabulous journey for students and for us. Thanks to everyone who contributed time on this project and joined us to learn about design process. We’ve collected potential experiment ideas and compelling proposals for the next phase of Test Pilot, and we’ll keep collaborating with schools and students for more fresh ideas. We can never have too many good ideas, right?

Test Pilot on Campus was originally published in Firefox Test Pilot on Medium, where people are continuing the conversation by highlighting and responding to this story.

Categorieën: Mozilla-nl planet

Adrian Gaudebert: Processeer.io - a Powerful and Flexible Report Builder for JSON APIs

Mozilla planet - vr, 28/07/2017 - 14:08

More than 2 years ago, I wrote about a prototype I built, called Spectateur, that allowed users of Socorro to create custom reports for their needs. The idea was to make it easy to write a script that pulls data from a public API, transforms it and then displays it as a nice table or chart. I strongly believe in this idea, and have been making various progress on the concept over the last 2 years, until I finally put together a nicer, more complete prototype at the end of 2016. I called it Processeer, and it's available at processeer.io.

What is Processeer solving?

Processeer aims at leveraging 2 things: the burden of creating custom reports, and the difficulty of sharing them with other people. First, it simplifies the process of writing scripts that interpret data from APIs, and provides tools to show that data in a beautiful fashion. Second, it makes it easy to share both the results of such scripts and the scripts themselves with other people, encouraging them to reuse and improve existing reports.

Here are few comparison points that might help you understand what it is. Do you know about re:dash? It's kind of the same thing, except it doesn't talk to a database but to APIs, and there's some powerful composition possibilities in Processeer. Maybe you've heard of bl.ocks.org? Processeer wants to give you the same kind of easy-sharing reports.

How about a simple example? Let's say you're a Lord of the Rings fan and you want to know at a glance the repartition of each of Middle Earth's peoples in the community of the Ring as well as the number of Rings of Power each of those peoples has. Well, here's a nice line chart that shows just that.

Right, that's cheating: that last one doesn't even pull data from an API. Alright, how about something a bit more complex? This one tells you if a bug from bugzilla is still relevant or not, by showing the number of crash reports associated to that bug in the last week: Bug Signature Status. That Report is composed of 3 different Blocks. The first one pulls a list of signatures from crash-stats, the second one pulls the bug title from bugzilla, and the third one pulls from crash-stats the actual number of crash reports for each signature we received from the first Block. If you want to look at how it works, simply log in with a github account and you'll be able to edit the Report and Blocks.

Note that you can type in any bug number in the "bug" filter at the top, and click "Run" to rerun that Report for that bug.

How does it work?

Processeer has a few concepts that need to be explained. First of all, you need to have a data source that is accessible over HTTP and that is serving JSON. That includes, for example, the crash-stats public API, bugzilla's REST API, github's REST API, and many more.

Processeer's high level component is called the Report. It is an ordered list of Blocks (we'll learn about that in just a few moments). When you load a Report page, Processeer is going to run each Block in turn, passing the output of the first Block as the input of the second, and so on. The output of the last Block of the chain is then going to be parsed and displayed as nicely as possible, as a table or as a chart for example.

Blocks are the building elements of Processeer. It's where you put most of the logic: where to pull the data from, what to do with it, and what to return. The Models of a Block define how the data is pulled. You can set several Models in a Block. Each one is basically a construct to make an HTTP call to some API that returns JSON. The Controller is a JavaScript function that you write in order to transform the data you get from your models into what you want your Block to return. It can be a specially parsed object that Processeer will parse and turn into a nice looking chart or table, or it can be an object that should be passed to the next Block in a Report. And that is where Params come in. Params define the "input" of your Block: you can set default values for some arguments you expect, and if those arguments exist in the output of the previous Block, their values will be passed down for you to use in your current Block. That mechanism allows, for example, to have a block that pulls some data from an API, then have another Block that uses that very data in the query of another API request. It also makes Blocks more reusable, and encourages users to keep them as small and focused as possible.

Schema of how Processeer works, a recap of the previous paragraph.

Schema of how Processeer works, a recap of the previous paragraph.

How do I use it?

Go to processeer.io and log in using a github account. You will then be able to create new Blocks and Reports. There are some Blocks already, I invite you to take a look at them to see how the system works, and what you can do. There's some documentation on the Processeer github repository, as well as some basic examples to understand the possible outputs.

I am very happy to answer questions if you have any. You can reach out to me with github issues, by email via my contact page, or on IRC (for example in #processeer in Freenode's IRC server).

The future is unwritten

The future of Processeer is unclear. I don't have any users just yet, probably for a lack of advertisement and documentation. But it is hard to invest time into those things when you don't have any users. So, here's my message in a bottle that I am throwing into the sea. If you find Processeer useful and think it can help you, I would love to hear about it. I need fearless explorers ready to dig into it to make the best out of the amazing possibilities it offers.

And if you think it's a brilliant idea and want to contribute, it's all open source (not licensed yet though). :)

Categorieën: Mozilla-nl planet

Macro Castelluccio: Overview of the Code Coverage Architecture at Mozilla

Mozilla planet - vr, 28/07/2017 - 02:00

Firefox is a huge project, consisting of around 20K source files and 3M lines of code (if you only consider the Linux part!), supporting officially four operating systems, and being written in multiple programming languages (C/C++/JavaScript/Rust). We have around 200 commits landing per day in the mozilla-central repository, with developers committing even more often to the try repository. Usually, code coverage analysis is performed on a single language on small/medium size projects. Therefore, collecting code coverage information for such a project is not an easy task.

I’m going to present an overview of the current state of the architecture for code coverage builds at Mozilla.

Tests in code coverage builds are slower than in normal builds, especially when we will start disabling more compiler optimizations to get more precise results. Moreover, the amount of data generated is quite large, each report being around 20 MB. If we had one report for each test suite and each commit, we would have around ~100 MB x ~200 commits x ~20 test suites = ~400 GB per day. This means we are, at least currently, only running a code coverage build per mozilla-central push (which usually contain around ~50 to ~100 commits), instead of per mozilla-inbound commit.

View of linux64-ccov build and tests from Treeherder Figure 1: A linux64-ccov build (B) with associated tests, from https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&filter-searchStr=linux64-ccov&group_state=expanded.

Each test machine, e.g. bc1 (first chunk of the Mochitest browser chrome) in Figure 1, generates gcno/gcda files, which are parsed directly on the test machine to generate a LCOV report.

Because of the scale of Firefox, we could not rely on some existing tools like LCOV. Instead, we had to redevelop some tooling to make sure the whole process would scale. To achieve this goal, we developed grcov, an alternative to LCOV written in Rust (providing performance and parallelism), to parse the gcno/gcda files. With the standard LCOV, parsing the gcno/gcda files takes minutes as opposed to seconds with grcov (and, if you multiply that by the number of test machines we have, it becomes more than 24 hours vs around 5 minutes).

Let’s take a look at the current architecture we have in place:

Architecture view Figure 2: A high-level view of the architecture.

Both the Pulse Listener and the Uploader Task are part of the awesome Mozilla Release Engineering Services (https://github.com/mozilla-releng/services). The release management team has been contributing to this project to share code and efforts.

Pulse Listener

We are running a pulse listener process on Heroku which listens to the taskGroupResolved message, sent by TaskCluster when a group of tasks finishes (either successfully or not). In our case, the group of tasks is the linux64-ccov build and its tests (note: you can now easily choose this build on trychooser, run your own coverage build and generate your report. See this page for instructions).

The listener, once it receives the “group resolved” notification for a linux64-ccov build and related tests, spawns an “uploader task”.

The source code of the Pulse Listener can be found here.

Uploader Task

The main responsibility of the uploader task is aggregating the coverage reports from the test machines.

In order to do this, the task:

  1. Clones mozilla-central;
  2. Builds Firefox (using artifact builds for speed); this is currently needed in order to generate the mapping between the URLs of internal JavaScript components and modules (which use special protocols, such as chrome:// or resource://) to the corresponding files in the mozilla-central repository (e.g. resource://gre/modules/Services.jsmtoolkit/modules/Services.jsm);
  3. Rewrites the LCOV files generated by the JavaScript engine for JavaScript code, using the mapping generated in step 2 and also resolving preprocessed files (yes, we do preprocess some JavaScript source files with a C-style preprocessor);
  4. Runs grcov again to aggregate the LCOV reports from the test machines into a single JSON report, which is then sent to codecov.io and coveralls.io.

Both codecov.io and coveralls.io, in order to show source files with coverage overlay, take the contents of the files from GitHub. So, we can’t directly use our Mercurial repository, but we have to rely on our Git mirror hosted on GitHub (https://github.com/mozilla/gecko-dev). In order to map the mercurial changeset hash associated with the coverage build to a Git hash, we use a Mapper service.

Code coverage results on Firefox code can be seen on:

The source code of the Uploader Task can be found here.

Future Directions Reports per Test Suite and Scaling Issues

We are interested in collecting code coverage information per test suite. This is interesting for several reasons. First of all, we could suggest developers which suite they should run in order to cover the code they change with a patch. Moreover, we can evaluate the coverage of web platform tests and see how they fare against our built-in tests, with the objective to make web platform tests cover as much as possible.

Both codecov.io and coveralls.io support receiving multiple reports for a single build and showing the information both in separation and in aggregation (“flags” on codecov.io, “jobs” on coveralls.io). Unfortunately, both services are currently choking when we present them with too much data (our reports are huge, given that our project is huge, and if we send one per test suite instead of one per build… things blow).

Coverage per Push

Understanding whether the code introduced by a set of patches is covered by tests or not is very valuable for risk assessment. As I said earlier, we are currently only collecting code coverage information for each mozilla-central push, which means around 50-100 commits (e.g. https://hg.mozilla.org/mozilla-central/pushloghtml?changeset=07484bfdb96b), instead of for each mozilla-inbound push (often only one commit). This means we don’t have coverage information for each set of patches pushed by developers.

Given that most mozilla-inbound pushes in the same mozilla-central push will not change the same lines in the same files, we believe we can infer the coverage information for intermediate commits from the coverage information of the last commit.

Windows, macOS and Android Coverage

We are currently only collecting coverage information for Linux 64 bit. We are looking into expanding it to Windows,macOS and Android. Help is appreciated!

Support for Rust

Experimental support for gcov-style coverage collection landed recently in Rust. The feature needs to be ship in a stable release of Rust before we can use it; this issue is tracking its stabilization.

Categorieën: Mozilla-nl planet

Chris H-C: Another Advantage of Decreasing Data Latency: Flatter Graphs

Mozilla planet - do, 27/07/2017 - 22:22

I’ve muttered before about how difficult it can be to measure application crashes. The most important lesson is that you can’t just count the number of crashes, you must normalize it by some “usage” value in order to determine whether a crashy day is because the application got crashier or because the application was just being used more.

Thus you have a numerator (number of crashes) and a denominator (some proxy of application usage) to determine the crash rate: crashes-per-use.

The current dominant denominator for Firefox is “thousand hours that Firefox is open,” or “kilo-usage-hours (kuh).”

The biggest problem we’ve been facing lately is how our numerator (number of crashes) comes in at a different rate and time than our denominator (kilo-usage-hours) due to the former being transmitted nearly-immediately via “crash” ping and the former being transmitted occasionally via “main” ping.

With pingsender now sending most “main” pings as soon as they’re created, our client submission delay for “main” pings is now roughly in line with the client submission delay of “crash” pings.

What does this mean? Well, look at this graph from https://telemetry.mozilla.org/crashes:

Screenshot-2017-7-25 Crash Rates (Telemetry)

This is the Firefox Beta Main Crash Rate (number of main process crashes on Firefox Beta divided by the number of thousands of hours users had Firefox Beta running) over the past three months or so. The spike in the middle is when we switched from Firefox Beta 54 to Firefox Beta 55. (Most of that spike is a measuring artefact due to a delay between a beta being available and people installing it. Feel free to ignore it for our purposes.)

On the left in the Beta 54 data there is a seven-day cycle where Sundays are the lowest point and Saturday is the highest point.

On the right in the Beta 55 data, there is no seven-day cycle. The rate is flat. (It is a little high, but flat. Feel free to ignore its height for our purposes.)

This is because sending “main” pings with pingsender is behaviour that ships in Firefox 55. Starting with 55, instead of having most of our denominator data (usage hours) coming in one day late due to “main” ping delay, we have that data in-sync with the numerator data (main crashes), resulting in a flat rate.

You can see it in the difference between Firefox ESR 52 (yellow) and Beta 55 (green) in the kusage_hours graph also on https://telemetry.mozilla.org/crashes:

Screenshot-2017-7-27 Crash Rates (Telemetry)

On the left, before Firefox Beta 55’s release, they were both in sync with each other, but one day behind the crash counts. On the right, after Beta 55’s release, notice that Beta 55’s cycle is now one day ahead of ESR 52’s.

This results in still more graphs that are quite satisfying. To me at least.

It also, somewhat more importantly, now makes the crash rate graph less time-variable. This reduces cognitive load on people looking at the graphs for explanations of what Firefox users experience in the wild. Decision-makers looking at these graphs no longer need to mentally subtract from the graph for Saturday numbers, adding that back in somehow for Sundays (and conducting more subtle adjustments through the week).

Now the rate is just the rate. And any change is much more likely to mean a change in crashiness, not some odd day-of-week measurement you can ignore.

I’m not making these graphs to have them ignored.

(many thanks to :philipp for noticing this effect and forcing me to explain it)

:chutten


Categorieën: Mozilla-nl planet

Air Mozilla: Reps Weekly Meeting Jul. 27, 2017

Mozilla planet - do, 27/07/2017 - 18:00

Reps Weekly Meeting Jul. 27, 2017 This is a weekly call with some of the Reps to discuss all matters about/affecting Reps and invite Reps to share their work with everyone.

Categorieën: Mozilla-nl planet

Air Mozilla: Egencia Training: Canada site- Pacific Time

Mozilla planet - do, 27/07/2017 - 12:03

 Canada site- Pacific Time Training and demo of Egencia Canada site (For all residents in Canada)

Categorieën: Mozilla-nl planet

Air Mozilla: Egencia Training: UK site

Mozilla planet - do, 27/07/2017 - 11:56

 UK site Training and demo of Egencia UK site (For residents in the UK)

Categorieën: Mozilla-nl planet

Air Mozilla: Egencia Training: Singapore site

Mozilla planet - do, 27/07/2017 - 11:48

 Singapore site Training and demo of Egencia Singapore site (For residents in Taipei and APAC, excludes Australia and New Zealand) Here is the training video for Egencia...

Categorieën: Mozilla-nl planet

Pagina's