mozilla

Mozilla Nederland LogoDe Nederlandse
Mozilla-gemeenschap

Abonneren op feed Mozilla planet
Planet Mozilla - http://planet.mozilla.org/
Bijgewerkt: 4 min 28 sec geleden

Nicholas Nethercote: I want more users on the Nightly channel

do, 19/05/2016 - 07:28

I have been working recently on a new Platform Engineering initiative called Uptime, the goal of which is to reduce Firefox’s crash rate on both desktop and mobile. As a result I’ve been spending a lot of time looking at crash reports, particular on the Nightly channel. This in turn has increased my appreciation of how important Nightly channel users are.

A crash report from a Nightly user is much more useful than a crash report from a non-Nightly user, for two reasons.

  • If a developer lands a change that triggers crashes for Nightly users, they will get fast feedback via crash reports, often within a day or two.  This maximizes the likelihood of a fix, because the particular change will be fresh in the developer’s mind. Also, backing out changes is usually easy at this point. In contrast, finding out about a crash weeks or months later is less useful.
  • Because a new Nightly build is done every night, if a new crash signature appears, we have a fairly small regression window. This makes it easier to identify which change caused the new crashes.

Also, Nightly builds contain some extra diagnostics and checks that can also be helpful with identifying a range of problems. (See MOZ_DIAGNOSTIC_ASSERT for one example.)

If we could significantly increase the size of our Nightly user population, that would definitely help reduce crash rates. We would get data about a wider range of crashes. We would also get stronger signals for specific crash-causing defects. This is important because the number of crash reports received for each Nightly build is relatively low, and it’s often the case that a cluster of crash reports that come from two or more different users will receive more attention than a cluster that comes from a single user.

(You might be wondering how we distinguish those two cases. Each crash report doesn’t contain enough information to individually identify the user — unless the user entered their email address into the crash reporting form — but crash reports do contain enough information that you can usually tell if two different crash reports have come from two different users. For example, the installation time alone is usually enough, because it’s measured to the nearest second.)

All this is doubly true on Android, where the number of Nightly users is much smaller than on Windows, Mac and Linux.

Using the Nightly channel is not the best choice for everyone. There are some disadvantages.

  • Nightly is less stable than later channels, but not drastically so. The crash rate is typically 1.5–2.5 times higher than Beta or Release, though occasionally it spikes higher for a short period. So a Nightly user should be comfortable with the prospect of less stability.
  • Nightly gets updated every 24 hours, which some people would find annoying.

There are also advantages.

  • Nightly users get to experience new features and fixes immediately.
  • Nightly users get the satisfaction that they are helping produce a better Firefox. The frustration of any crash is offset by the knowledge that the information in the corresponding crash report is disproportionately valuable. Indeed, there’s a non-trivial likelihood that a single crash report from a Nightly user will receive individual attention from an engineer.

If you, or somebody you know, thinks that those advantages outweigh the disadvantages, please consider switching. Thank you.

Categorieën: Mozilla-nl planet

Robert O'Callahan: x86-64 SSE/AVX Register Usage

do, 19/05/2016 - 05:57

Following up on my previous post, I counted the fraction of instructions in Firefox opt/debug libxul.so that use each XMM/YMM register.

Observations:

  • As before, debug builds are heavily weighted towards use of the first few registers, and opt builds allocate across more registers as you'd expect.
  • In debug builds, usage of the higher-numbered registers (up to 7) is a combination of va_start spilling all parameter registers (0-7) to the stack, and handwritten-assembly. It looks like almost all the handwritten assembly in Firefox restricts itself to registers 0-7, presumably so it works in x86-32 as well. Maybe some of that code would benefit from being updated for x86-64 with more registers?
  • In opt builds there's a clear drop-off in usage after register 7, more than can be explained by handwritten assembly or va_start spilling (since those equally affect debug). It's not related to caller/callee-saves status because all MM registers are caller-saves on Linux. It appears that in some functions experiencing moderate register pressure, gcc has freely used registers 0-7 but avoided using 8-15. Maybe that's because the latter require longer instruction encodings in some cases. You don't see the same dropoff moving to the upper eight GP registers, which have the same encoding length issue, but that may because of callee-saves and generally increased register pressure.
  • In libxul at least, MM registers are used far less often than GP registers. Register 0, the most-used by far, is used by barely 1% of instructions, comparable to the least-used GP registers. Registers 8 to 15 are each used by less than 0.1% of instructions.

As before, these are static counts and I'd expect weighting instructions by dynamic frequency would change the results dramatically --- on the right workloads --- since most of the hand-written assembly in Firefox is hand-written specifically to optimize use of MM registers in hot loops.

Update One interesting takeaway is that you have eight huge registers (256 bits each, 512 soon) unused by most code. That creates some interesting possibilities...

Categorieën: Mozilla-nl planet

Air Mozilla: Engineering Onboarding - Intern Q&A: Bugzilla, Session 1

wo, 18/05/2016 - 21:00

 Bugzilla, Session 1 Intern Q&A: Bugzilla, Session 1

Categorieën: Mozilla-nl planet

The Mozilla Blog: Welcome Alex Salkever, Vice President of Marketing Communications

wo, 18/05/2016 - 20:03

I’m excited to announce that Alex Salkever joins the Mozilla leadership team today as the Vice President of Marketing Communications.

In this role, Alex Salkever will be responsible for driving strategic positioning and marketing communications campaigns. Alex will oversee the global communications, social media, user support and content marketing teams and work across the organization to develop impactful outbound communications for Mozilla and Firefox products.

Alex Salkever, MozillaAlex was most recently Chief Marketing Officer of Silk.co, a data publishing and visualization startup, where he led efforts focused on user growth and platform partnerships. Alex has held a variety of senior marketing, marketing communications and product marketing roles working on products in the fields of scientific instruments, cloud computing, telecommunications and Internet of Things. In these various capacities, Alex has managed campaigns across all aspects of marketing and product marketing including PR, content marketing, user acquisition, developer marketing and marketing analytics.

Alex also brings to Mozilla his experience as a former Technology Editor for BusinessWeek.com. Among his many accomplishments, Alex is the co-author of “The Immigrant Exodus”, a book named to The Economist Book of the Year List in the Business Books category in 2012.

Welcome Alex!

Background:

Alex’s bio & Mozillians profile

LinkedIn profile

High-resolution photo

Categorieën: Mozilla-nl planet

Air Mozilla: The Joy of Coding - Episode 57

wo, 18/05/2016 - 19:00

The Joy of Coding - Episode 57 mconley livehacks on real Firefox bugs while thinking aloud.

Categorieën: Mozilla-nl planet

Air Mozilla: Weekly SUMO Community Meeting May 18, 2016

wo, 18/05/2016 - 18:00

Weekly SUMO Community Meeting May 18, 2016 This is the SUMO weekly call

Categorieën: Mozilla-nl planet

Karl Dubost: Working AS a distributed team. No more remote workers

wo, 18/05/2016 - 08:09

When you say remoter workers, you already failed your intents of having an effective team. Companies setting up "friendly remote work environment" are most of the time failing at understanding the nature of the issues. The best way you can set up a distributed team is by forgetting about the workers and focusing on the work itself.

I'm working at Mozilla in the Web Compatibility Activity. I'm part of a now 4 persons team (Mozilla staff) and a couple of active contributors. For the Mozilla staff part:

  • Mike Taylor in Texas, USA
  • Hallvord Steen in Norway
  • Adam Stevenson in Ontario, Canada
  • Karl Dubost (myself) in Kanagawa, Japan
  • We have contributors in India, Japan, France, Romania, Brazil, Mexico, etc.

We are working together.

The most important part of creating a successful distributed team is when you stop thinking that there are remote workers in your team. You need to consider that the work can be done from anywhere by anyone competent for the job. This will give a good base for organizing the work in terms of process, protocols and tools to be productive and effective. That's the key, the only one.

Some tips:

  1. Choose open first: Opening a private discussion is a lot harder, than making a private comment on a discussion.
  2. Record Action Items: Have action items which are identifiable by all the team members you are working with (and broader when possible. See 1.). These items need an owner, an unambiguous actionable task, a target or context and a deadline.
  3. Record any meetings: When there is a meeting, write down detailed minutes on the spot. Give these minutes a unique and stable URI. The context might be useful for another team or a new employee later on.
  4. Share your work assets with others: Anything you produce, code, documents, etc. Give access to it. Share it as early as possible with again stable and public URIs.
  5. Share your worklog: This helps others to decide if they can request more things from you. This will help them to decide if they can make progress on their own job.
  6. Web Archived Mailing-Lists: Set your mailing-list archives in a way that makes it accessible to everyone (to the world if possible, to the entire company, and in some very rare contexts to only your team)

There are many others small tips to make this more effective, but these will go a long way in achieving your goals.

The amazing benefits of working that way is that it doesn't allow only for a distributed team, it makes the whole organization more robust by having a solid information flow management.

PS: I have been working in a distributed way at W3C, Opera and now Mozilla for the last 15 years. In my work history, I still consider the W3C (2000-2008) the best place for distributed work among staff. I don't know about W3C today.

Otsukare!

Categorieën: Mozilla-nl planet

Maja Frydrychowicz: A Week of Reflection and Programming for Fun

wo, 18/05/2016 - 06:00

I recently got to spend a week back at the heart of an excellent delightful inspiring technical community: Recurse Center or RC. This friendly group consists mostly of programmers from around the world who have, at some point, participated in RC’s three-month “retreat” in New York City to work on whatever projects happen to interest them. The retreat’s motto is “never graduate”, and so participants continue to support each other’s technical growth and curiosity forever and ever.

I’m an RC alum from 2014! RC’s retreat is how I ended up contributing to open source software and eventually gathering the courage to join Mozilla. Before RC, despite already having thousands of hours of programming and fancy math under my belt, I held myself back with doubts about whether I’m a “real programmer”, whatever that stereotype means. That subconscious negativity hasn’t magically disappeared, but I’ve had a lot of good experiences in the past few years to help me manage it. Today, RC helps me stay excited about learning all the things for the sake of learning all the things.

A retreat at RC looks something like this: you put your life more-or-less on hold, move to NYC, and spend three months tinkering in a big, open office with around fifty fellow (thoughtful, kind, enthusiastic) programmers. During my 2014 retreat, I worked mostly on lowish-level networking things in Python, pair programmed on whatever else people happened to be working on, gave and received code review, chatted with wise “residents”, attended spontaneous workshops, presentations and so on.

Every May, alumni are invited to return to the RC space for a week, and this year I got to go! (Thanks, Mozilla!) It was awesome! Exclamation points! This past week felt like a tiny version of the 3-month retreat. After two years away, I felt right at home — that says a lot about the warm atmosphere RC manages to cultivate. My personal goal for the week was just to work in a language that’s relatively new to me - JavaScript - but I also happened to have really interesting conversations about things like:

  • How to implement a basic debugger?
  • How to improve the technical interview process?
  • What holds developers back or slows them down? What unnecessary assumptions do we have about our tools and their limitations?

RC’s retreat is a great environment for growing as a developer, but I don’t want to make it sound like it’s all effortless whimsy. Both the hardest and most wonderful part of RC (and many other groups) is being surrounded by extremely impressive, positive people who never seem to struggle with anything. It’s easy to slip into showing off our knowledge or to get distracted by measuring ourselves against our peers. Sometimes this is impostor syndrome. Sometimes it’s the myth of the 10x developer. RC puts a lot of effort into being a safe space where you can reveal your ignorance and ask questions, but insecurity can always be a challenge.

Similarly, the main benefit of RC is learning from your peers, but the usual ways of doing this seem to be geared toward people who are outgoing and think out loud. These are valuable skills, but when we focus on them exclusively we don’t hear from people who have different defaults. There is also little structure provided by RC so you are free to self-organize and exchange ideas as you deem appropriate. The risk is that quiet people are allowed to hide in their quiet corners, and then everyone misses out on their contributions. I think RC makes efforts to balance this out, but the overall lack of structure means you really have to take charge of how you learn from others. I’m definitely better at this than I used to be.

RC is an experiment and it’s always changing. Although at this point my involvement is mostly passive, I’m glad to be a part of it. I love that I’ve been able to work closely with vastly different people, getting an inside look at their work habits and ways of thinking. Now, long after my “never-graduation”, the RC community continues to expose me to a variety of ideas about technology and learning in a way that makes us all get better. Continuous improvement, yeah!

Categorieën: Mozilla-nl planet

Armen Zambrano: Schedule a Linux64 TaskCluster task from the command line

di, 17/05/2016 - 22:57
I've created an experimental repository to play with TaskCluster scheduling using your personal temporary credentials.

If you want to try to schedule a real task from the command line feel free to give it a try:
https://github.com/armenzg/TC_developer_scheduling_experiments


Here's the output of scheduling a Linux64 debug task.
NOTE: It will not post to Treeherder
NOTE: It will open a new tab asking you to grant access to your TaskCluster temp credentials.

(TC_scheduling) armenzg@armenzg-thinkpad:~/repos/TC_developer_scheduling_experiments$ python schedule_linux64_task.py 
04:48:50 root Setting INFO level
04:48:50 mozci.taskcluster.tc We're going to open a new tab and authenticate you with TaskCluster.
-------------------------------------------------------
  Opening browser window to login.taskcluster.net
  Asking you to grant temporary credentials to:
     http://localhost:39025
-------------------------------------------------------
04:48:54 mozci.taskcluster.tc Inspect the task in https://tools.taskcluster.net/task-inspector/#bmt-5IqPTwmn8JrMzdofGg



Creative Commons License
This work by Zambrano Gasparnian, Armen is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.
Categorieën: Mozilla-nl planet

Air Mozilla: Connected Devices Weekly Program Update, 17 May 2016

di, 17/05/2016 - 19:00

Connected Devices Weekly Program Update Weekly project updates from the Mozilla Connected Devices team.

Categorieën: Mozilla-nl planet

Mozilla Addons Blog: Add-on Compatibility for Firefox 48

di, 17/05/2016 - 17:47

Firefox 48 will be released on August 2nd. Here’s the list of changes that went into this version that can affect add-on compatibility. There is more information available in Firefox 48 for Developers, so you should also give it a look.

General XPCOM and Modules New

Let me know in the comments if there’s anything missing or incorrect on these lists. If your add-on breaks on Firefox 48, I’d like to know.

The automatic compatibility validation and upgrade for add-ons on AMO will happen in a few weeks, so keep an eye on your email if you have an add-on listed on our site with its compatibility set to Firefox 47.

Categorieën: Mozilla-nl planet

About:Community: Reinventing Mozilla on Campus

di, 17/05/2016 - 15:28

Re-post from George Roter’s blog, “Reinventing Mozilla on Campus” .

Throughout history, University students, staff and professors have often shaped the leading edge of change and innovation. The history of the web is no different: the student-built Lynx browser was one of the first and Mosaic (Firefox’s distant ancestor!), pioneered by students and staff, opened the graphical web to millions.

I saw the impact that students and professors can make through my own experience at Engineers Without Borders Canada. Engineering students and professors on campuses across Canada and in Africa built remarkable ventures, reshaped curriculum, changed on-campus and government policy, and taught hundreds of thousands of young people about global development.

I fully believe in the potential of students, staff and professors on campuses around the world to have massive impact on Mozilla’s mission. As innovators, contributors and open web advocates. Engineers, scientists, lawyers, social scientists, economists and designers.

From what I know about my past experience and have heard in the past year working for Mozilla, our mission resonates tremendously with students and professors. The range of impact and involvement is considerable. Until now, we’ve only just scraped the surface of this potential.

We need to reinvent Mozilla on campus.

Our existing engagement on University campuses around the world is an assortment of largely disconnected programs and people. Firefox Student Ambassadors and Firefox Clubs. Mozilla Clubs. Code contribution by individual contributors. Maker Party. Mozilla Science Lab. Various professor and lab partnerships. Employee recruitment. Many of these are successful in their own right; there’s an opportunity learn from each of them, find connections, and imagine opportunity to scale their impact with a more coordinated approach.

Photo credit: Tanha Islam and Trisa Islam

The largest of these by student involvement, Firefox Student Ambassadors (FSAs) and Firefox Clubs, has been constrained by limited and variable employee support and a focus on marketing. Our student leaders have already been “hacking” this program to introduce advocacy, code contribution, support, localization, teaching and many other activities; official support for this has lagged.

Our team came into this year with a key hypothesis as part of our strategy: That we can supercharge participation with a reinvented campus program.

The Take Back the Web campus campaign focused on privacy and security has been our first effort to test this hypothesis. Already it’s showing great promise, with over 600 campus teams signed up (including hundreds of FSAs) to have impact in 3 areas. We’re focused on learning as much as we can from this campaign.

The campus campaign is a step toward reinvention. But I think it’s now time to take a step back to ask: What impact can we imagine with a coordinated effort on campuses around the world? What do students, staff and professors want and need to be involved with Mozilla’s mission? How might we evolve our existing programs? What programs and structures would we design, and how do they relate to one another? How can we invite people on campus to innovate with Mozilla?

These are the broad questions that will guide a process over the next 9 weeks. By July 15th we aim to have a clear articulation of the impact we can have, the programs we’ll invest in and how they relate to one another, and the opportunities for students, staff and professors to participate.

We’re hoping that this process of reinventing Mozilla on campus will be participatory, and we’re inviting many voices to contribute. Lucy Harris on the Participation Team will be stewarding this process and shaping the final options. Mark Surman, Mitchell Baker, Chris Lawrence, Katharina Borchert and I will be involved in making a final decision on the direction we take.

You can read more about the details of the process in this post, but let me summarize it and the opportunities you have to be involved:

Phase 1: Listening (May 16-27)

→ provide thoughts on existing programs and opportunities you see

Phase 2: Synthesis and options (May 27-June 10)

→ we’ll frame some tensions for you to weigh in on

→ we’ll shape a set of options for conversation during the London All Hands

Phase 3: Final input (June 10-24)

→ we’ll articulate a set of options for you to consider as we move forward, and will be diving deep into these and key questions during the Mozilla All Hands in London

Phase 4: Final Decision and Disseminate (June 24-July 15)

→ we’ll take all the input and decide on a direction for moving forward

Let me finish by reiterating the opportunity. University campuses are a hotbed of innovation and a locus for creating change. Mozilla can tap into this energy and catalyze involvement in unleashing the next wave of openness and opportunity in online life. Finally, our team is excited about helping to shape a direction we can take, and investing in a robust program of participation moving forward.

I’m excited for this journey of reinventing Mozilla on campus.

Categorieën: Mozilla-nl planet

George Roter: Reinventing Mozilla on campus

di, 17/05/2016 - 14:57

Throughout history, University students, staff and professors have often shaped the leading edge of change and innovation. The history of the web is no different: the student-built Lynx browser was one of the first and Mosaic (Firefox’s distant ancestor!), pioneered by students and staff, opened the graphical web to millions.

I saw the impact that students and professors can make through my own experience at Engineers Without Borders Canada. Engineering students and professors on campuses across Canada and in Africa built remarkable ventures, reshaped curriculum, changed on-campus and government policy, and taught hundreds of thousands of young people about global development.

I fully believe in the potential of students, staff and professors on campuses around the world to have massive impact on Mozilla’s mission. As innovators, contributors and open web advocates. Engineers, scientists, lawyers, social scientists, economists and designers.

From what I know about my past experience and have heard in the past year working for Mozilla, our mission resonates tremendously with students and professors. The range of impact and involvement is considerable. Until now, we’ve only just scraped the surface of this potential.

We need to reinvent Mozilla on campus.

Our existing engagement on University campuses around the world is an assortment of largely disconnected programs and people. Firefox Student Ambassadors and Firefox Clubs. Mozilla Clubs. Code contribution by individual contributors. Maker Party. Mozilla Science Lab. Various professor and lab partnerships. Employee recruitment. Many of these are successful in their own right; there’s an opportunity learn from each of them, find connections, and imagine opportunity to scale their impact with a more coordinated approach.

mg_3830

Photo credit: Tanha Islam and Trisa Islam [1]

The largest of these by student involvement, Firefox Student Ambassadors (FSAs) and Firefox Clubs, has been constrained by limited and variable employee support and a focus on marketing. Our student leaders have already been “hacking” this program to introduce advocacy, code contribution, support, localization, teaching and many other activities; official support for this has lagged.

Our team came into this year with a key hypothesis as part of our strategy: That we can supercharge participation with a reinvented campus program.

The Take Back the Web campus campaign focused on privacy and security has been our first effort to test this hypothesis. Already it’s showing great promise, with over 600 campus teams signed up (including hundreds of FSAs) to have impact in 3 areas. We’re focused on learning as much as we can from this campaign.

The campus campaign is a step toward reinvention. But I think it’s now time to take a step back to ask: What impact can we imagine with a coordinated effort on campuses around the world? What do students, staff and professors want and need to be involved with Mozilla’s mission? How might we evolve our existing programs? What programs and structures would we design, and how do they relate to one another? How can we invite people on campus to innovate with Mozilla?

These are the broad questions that will guide a process over the next 9 weeks. By July 15th we aim to have a clear articulation of the impact we can have, the programs we’ll invest in and how they relate to one another, and the opportunities for students, staff and professors to participate.

We’re hoping that this process of reinventing Mozilla on campus will be participatory, and we’re inviting many voices to contribute. Lucy Harris on the Participation Team will be stewarding this process and shaping the final options. Mark Surman, Mitchell Baker, Chris Lawrence, Katharina Borchert and I will be involved in making a final decision on the direction we take.

You can read more about the details of the process in this post, but let me summarize it and the opportunities you have to be involved:

Phase 1: Listening (May 16-27)

→ provide thoughts on existing programs and opportunities you see

Phase 2: Synthesis and options (May 27-June 10)

→ we’ll frame some tensions for you to weigh in on

→ we’ll shape a set of options for conversation during the London All Hands

Phase 3: Final input (June 10-24)

→ we’ll articulate a set of options for you to consider as we move forward, and will be diving deep into these and key questions during the Mozilla All Hands in London

Phase 4: Final Decision and Disseminate (June 24-July 15)

→ we’ll take all the input and decide on a direction for moving forward

 

Let me finish by reiterating the opportunity. University campuses are a hotbed of innovation and a locus for creating change. Mozilla can tap into this energy and catalyze involvement in unleashing the next wave of openness and opportunity in online life. Finally, our team is excited about helping to shape a direction we can take, and investing in a robust program of participation moving forward.

I’m excited for this journey of reinventing Mozilla on campus.

 

[1] Photo credit: Tanha Islam and Trisa Islam


Categorieën: Mozilla-nl planet

The Mozilla Blog: Mozilla Expands Its National Gigabit Project to Austin, TX

di, 17/05/2016 - 13:13

Mozilla will provide $150,000 in funding, and also grow the local maker community, to spur gigabit innovation in Texas’ capital

When you couple lightning-fast Internet with innovative projects in the realms of education and workforce development, amazing things can happen.

That’s the philosophy behind the Mozilla Gigabit Community Fund, our joint initiative with the National Science Foundation and US Ignite. The Mozilla Gigabit Community Fund brings funding and staffing to U.S. cities equipped with gigabit connectivity, the next-generation Internet that’s 250-times faster than most other connections. Our goal: Spark the creation of groundbreaking, gigabit-enabled educational technologies so that more people of all ages and backgrounds can read, write, and participate on this next-generation Web.

As we just announced at the Gigabit City Summit in Kansas City, we’re expanding our gigabit work to the city of Austin, TX in August 2016. Selected from a list of contenders from across the country, Austin stood out due to its existing city-wide digital inclusion plan, active developer community, and growing informal education landscape. Beginning this fall, Mozilla will provide $150,000 in grant funding to innovative and local projects and tools that leverage Austin’s Google Fiber network. Think: 4K streaming in classrooms, immersive virtual reality, and more.

(In the existing Mozilla Gigabit cities of Chattanooga, TN and Kansas City, projects include real-time water monitoring systems, 3D learning tools for classrooms, and specialized technology for first responder training. Read more about those projects here.)

Individuals from the Chattanooga gigabit project Hyperaudio participate in a New York City Maker Party.

Individuals from the Chattanooga gigabit project Hyperaudio participate in a New York City Maker Party.

Mozilla is also investing in the makers and educators who make Austin great. We’ll help create Gigabit Hive Austin — a network of individuals, schools, nonprofits, museums, and other local organizations passionate about teaching and learning the Web. Hive Austin will be one of 14 Mozilla Hive networks and communities across four continents that teach web literacy and 21st-century skills.

Mozilla will open the first round of grant applications in Austin this August, and accept applications through October 18, 2016. Applicants and projects don’t have to be from Austin originally, but must be piloted locally. Click here to learn about the RFP process.

This spring, Mozilla is also providing $134,000 in new gigabit funding in Chattanooga and Kansas City. Funds will support projects that explore gigabit and robotics, big data, the Internet of Things, and more. Learn more.

Over the next two years, Mozilla will be expanding its Gigabit work to two additional cities. Interested in becoming a future Gigabit Hive city? We will reopen the city application process in late 2016.

Categorieën: Mozilla-nl planet

Karl Dubost: [worklog] From 17s to 0.5s, testing for performance

di, 17/05/2016 - 10:21

Tune of the week: Hip-Hop Mixtapes.

Webcompat Life

Progress this week:

Today: 2016-05-23T15:18:18.501043 354 open issues ---------------------- needsinfo 3 needsdiagnosis 119 needscontact 27 contactready 43 sitewait 157 ----------------------

You are welcome to participate

Londong agenda.

French Mozilla Community seems interested by Web Compatibility work. Replying to a thread where we asked me more information.

Seen this http-prompt library. Shared the link with mike who replied "wow! gonna install this". A couple of minutes later I opened a new issue for the missing HEAD method. Then I thought "Oh shoot! I can solve this". I created a pull request, which was merged in the following hour. ❤️ opensource.

Webcompat issues

(a selection of some of the bugs worked on this week).

WebCompat.com dev
  • We got a performance regression with a code I had written recently. Maybe there is a solution. A bit later. Found. Reminder to myself: do performance testing for any I/O operations before pushing them to prod. On Thursday, I did a lot of testing. Maybe I should write a separate post about this. It was an interesting experiment. Upload performances went from 17s to 0.5s
  • We probably need to block the about: URIs in reporting. It doesn't make sense in the context of our work.
Reading List
  • Evaluating scripts using Marionette: "I finally managed to land a patch that makes evaluated scripts have lasting side-effects in the document. This means that scripts such as window.foo = 42 or document.documentElement.style.background = "pink" will take effect on the DOM. This is a requirement for WebDriver conformance, and a major inhibitor for people trying to replace FirefoxDriver with Marionette in their automation. "
  • Web Standards: "Standards are made by the people who show up, and not nearly enough people are showing up. We need publishers and designers and engineers, typographers and librarians and entrepreneurs, web folks and book folks and business folks. We certainly need more diversity of gender and color and class."
  • GridViz: A Firefox extension to visualize grid lines defined by the CSS Grid API
Follow Your Nose TODO
  • Document how to write tests on webcompat.com using test fixtures.
  • ToWrite: rounding numbers in CSS for width
  • ToWrite: Amazon prefetching resources with <object> for Firefox only.

Otsukare!

Categorieën: Mozilla-nl planet

James Long: What’s in a Continuation

di, 17/05/2016 - 02:00

Many people have heard the word “continuation” because it has something to do with node’s callback hell. I don’t think most people understand what continuations really are, though. They aren’t just a callback function used by async functions.

A continuation is a representation of the control flow of your program at any point in time, essentially the stack. In abstract terms, it represents "the rest of your program." In languages like Scheme that expose continuations as first-class values, you can capture the current continuation and invoke it later. When invoked, the current program state is replaced with the state at which the continuation was captured (i.e. the current stack is replaced with the stack from the continuation).

Continuations allow you to literally "jump" to different places in your code. They are a low-level primitive that gives you control over execution flow, allowing you implement everything from resumable exceptions to coroutines. Understanding continuations was the best thing I did as a young programmer; it forces you to understand how control flow works.

It would be neat to see something like continuations implemented in JavaScript engines because you can implement everything on top of them (note that I said something like, as continuations themselves are very hard to optimize). I'm a fan of low-level primitives for the same reasons as the Extensible Web Manifesto: let users evolve the language over time.

I recently ended up implementing continuations in JavaScript. It wasn’t on purpose; I was originally pursuing a way to arbitrarily pause JavaScript in user-land so I could write tutorials and interactive editors. I realized that to arbitrarily pause JS, I would need all the machinery necessary for continuations. The ability to save a stack and resume it later. Eventually I discovered the paper "Exceptional Continuations in JavaScript" and was able to achieve my in-browser stepping debugger by implementing continuations.

I wrote more about the backstory of my work at the end of this post. I did most of this work 2 years ago and I'm now polishing it up and publishing it.

function foo(x) { console.log(x); if (x <= 0) { return x; } else { return x + foo(x - 1); } } function main() { console.log(foo(3)); } main(); A working stepping debugger! Click on any line to add a breakpoint.

And just like that, I realized that I could expose continuations first-class to this special variant of JS, which is way more interesting than my stepping debugger. In this article, I will use my work to explain what continuations are and give you a chance to interact with them.

I will explain how it is implemented in the next post. A short version: it transforms all code into a state machine and uses exceptions to save the state of all functions on the stack. This means that every function is transformed into a big switch statement with every expression as separate cases, giving us the ability to arbitrarily jump around.

This transform is very similar to what regenerator does, which compiles JavaScript generators to ES5 code. In fact, that project is what motivated this work. Two years ago, I forked regenerator and implemented everything you see here. That means that it doesn't support a lot of recent ES6 features and it's missing a lot of bug fixes.

Visit the unwinder repo to see the code and try it yourself. Warning: this is very prototype quality and many things are ugly. There's a very good chance that you will hit bugs. However, with some polish work this has a chance to become a place where we can explore interesting patterns.

A few other caveats:

  • You cannot step through native frames, or use continuations when they are on the stack. If you use the native array forEach and capture the continuation in the callback, things will go badly. This requires all code to be compiled through this transformer if you want to use continuations (normal code can call out to native code just fine, however).

  • This technique favors performance of code that does not use continuations. Capturing continuations is not very fast, but if you are implementing something like a debugger, that doesn't matter. However, if you are implementing advanced control flow operators, you will likely hit performance problems. This is a good place to experiment with them though.

Introducing Continuations

Let's revisit the definition of a continuation; wikipedia describes it as "an abstract representation of the control state of a computer program. A continuation reifies the program control state…" The key words are control state. This means that when a continuation is created, it contains all the necessary information to resume the program exactly at the point in time which is was created.

This is how the stepping debugger works internally. The generated code looks for breakpoints, and when one is hit, it captures the current continuation and stops executing. Resuming is as simple as invoking the saved continuation.

Let's have some real fun though and expose continuations as first-class values! In Scheme, you use call-with-current-continuation to capture the current continuation, or the shorthand call/cc. Any experienced Scheme coder is familiar with code like this:

(define (foo) (let ([x (call/cc (lambda (cont) (display "captured continuation") (cont 5) (display "continuation called")))]) (display "returning x") x)) (display (foo))

I implemented a callCC function in my special JavaScript variant since it already has all the necessary machinery. Additionally, we can use the stepping debugger to study how continuations affect the control flow.

function foo() { var x = callCC(function(cont) { console.log("captured continuation"); cont(5); console.log("continuation called"); }); console.log("returning x", x); return x; } console.log(foo()); A basic example of continuations.

This is a very simple example of using a continuation. Click "Run & Ignore Breakpoints" to see what happens. We capture the continuation using callCC, which gives us the continuation as the function cont. We then log "captured continuation" and invoke cont. Note how "continuation called" is never logged. Why is that?

Now go back and click "Run" to hit the breakpoint on line 13, and continually click "Step" to step through the program to see what happened. What happens when cont is invoked?

It jumps back to line 3! The previous control flow is aborted and the stack when cont was captured is restored. Any arguments passed to continuations replace the call to callCC, as if callCC returned that value. The continuation represents a state of the program where the callCC function is waiting to return a value.

Note: While the interface for calling a continuation looks like calling a function, it's quite different. The call to a continuation never returns. While this may be confusing, we will look into alternative continuation interfaces in future posts. Also, continuations are a low-level interface and rarely used directly.

Continuations are like portals. If the image below represents your control flow, you can capture the current stack (the blue portal) and jump back to it at any time (the orange portals).

The control flow of a program. Jumping through the orange portals will reset the stack to where the blue portal was created.

However, just like in the game Portal, these portals do not traverse time. The only thing that a continuation saves is the stack, so any changes to variables will still be seen after jumping through a continuation. Look at what happens with a closure:

function foo() { var x = 5; var func = function() { return x; }; x = 6; return func; } console.log(foo()())

This will print 6 because closures reference the same variable that is mutated later. With continuations, it's the same thing except stack frames are pointing to the variables. We will show examples of this later.

Practical Uses

Now that you understand the general idea, let's put continuations to good use. This certainly seems powerful, but you might have trouble thinking of problems this solves. In fact, you might be thinking that this would just make programs harder to follow.

Abusing continuations definitely makes programs harder to follow. But there are constructs that continuations allow you to build that are generally helpful; break and continue might make your program a little harder to follow, but they solve real problems, just like other control flow operators. Additionally, in a future post we will talk about delimited continuations which force developers to use continuations in a clearer way.

The first exercise is to implement the JavaScript some method, which checks if any element in an array passes a predicate check. Importantly, it is "short-circuiting," meaning it stops iterating after finding the first passing element because it doesn't need to check the rest.

function some(predicate, arr) { var x = callCC(function(cont) { for (var idx = 0; idx < arr.length; idx++) { console.log('testing', arr[idx]); if(predicate(arr[idx])) { cont(true); } } return false; }); return x; } console.log(some(x => x >= 2, [1, 2, 3, 4])); An implementation of some using continuations.

If you run this, you'll notice that it doesn't check 3 or 4. It stops executing once 2 passes the predicate check. Step through the code and watch how it does that.

Of course, we could use break to stop the while loop. But this is a trivial example; it's common to call out to other functions within the loop where you can't break. The native control operators are quite limiting. Continuations, however, allow you to travel across stack frames.

For example, let's say you wanted to use forEach method instead of a for loop, because you already use that method everywhere else. Here is the example now:

// Note: don't use native forEach so you can step // into this one function forEach(arr, func) { for (var i=0; i < arr.length; i++) { func(arr[i]); } } function some(predicate, arr) { var x = callCC(function(cont) { forEach(arr, function(val) { console.log('testing', val); if(predicate(val)) { cont(true); } }); return false; }); return x; } console.log(some(x => x === 2, [1, 2, 3, 4])); An implementation of some using continuations across stack frames.

It works exactly the same way, even if we are calling the predicate within the function passed to forEach. It still short-circuits. Note how we didn't have to change anything about forEach; we are able to reuse the same method that we already use everywhere else.

This highlights a fundamental difference of continuations and anything currently in JavaScript: it suspends the entire stack. Generators suspend code as well, but their yield is shallow. It only suspends one frame, the generator itself.

While that makes code clearer, it leads to a proliferation of special syntax across all code and forces a lot of work on the developer over the lifetime of a project. Converting a single sync function to async requires a massive refactoring, changing the interface of every thing that uses it. I recommend reading "What Color is Your Function?" for a great description of this problem.

In the next post we will show how having a single function interface (no function* or async function) and deep stack control greatly improves the reusability and readability of code.

Exception Handling

Let's get real. The above exercises are pretty stupid. You wouldn't actually use continuations like that; there are much better constructs for looping over values and short-circuiting. Those examples were simple on purpose for illustrative purposes.

Now we will implement a new fundamental control construct: exceptions. This shows that continuations allow you implement things previously built-in to the language.

Users should be able to throw exceptions and install handlers to catch them. Installed handlers are dynamically scoped for a given section of code: any exception that occurs within a given block of code, even if it comes from an external function, should be caught.

Exception handlers must exist as a stack: you can install new handlers that override existing ones for a given period of time, but the previous ones are always restored once the newer ones are "popped" off the stack. So we must manage a stack.

The stack is a list of continuations, because when a throw happens we need to be able to jump back to where the try was created. They means in try we need to capture the current continuation, push it on the stack, run the code, and dispatch exceptions. Here is the full implementation of try/catch:

var tryStack = []; function Try(body, handler) { var ret = callCC(function(cont) { tryStack.push(cont); return body(); }); tryStack.pop(); if(ret.__exc) { return handler(ret.__exc); } return ret; } function Throw(exc) { if(tryStack.length > 0) { tryStack[tryStack.length - 1]({ __exc: exc }); } console.log("unhandled exception", exc); }

The key here is that continuations can be resumed with values. The return body() line will return the final value of the code. At that point no continuation was invoked; it just passes that value through. But if Throw is invoked, it will call the captured continuation with an exception value, which gets assigned to ret, and we check for that type of value and call the handler. (We could do more sophisticated detection of exception types.)

Note that we pop the current handler off the stack before calling it, meaning that any exceptions that occur within exception handlers will properly be passed up the handler stack.

Here's what it looks like using Try/Catch:

function bar(x) { if(x < 0) { Throw(new Error("error!")); } return x * 2; } function foo(x) { return bar(x); } Try( function() { console.log(foo(1)); console.log(foo(-1)); }, function(ex) { console.log("caught", ex); } );

Unfortunately JavaScript does not allow us to extend syntax (although this can be solved with sweet.js macros, as we'll show in future posts). Instead of using blocks we must pass functions into Try. The output of this code would be 2 \n caught "error!".

The above implementation and example code are loaded into the editor below, with a breakpoint already set at the Try block. Hit "Run & Ignore Breakpoints" to verify the output, and "Run" to break and step through the code to see how it unfolds.

var tryStack = []; function Try(body, handler) { var ret = callCC(function(cont) { tryStack.push(cont); return body(); }); console.log('ret is', JSON.stringify(ret)); tryStack.pop(); if(ret.__exc) { return handler(ret.__exc); } return ret; } function Throw(exc) { if(tryStack.length > 0) { tryStack[tryStack.length - 1]({ __exc: exc }); } console.log("unhandled exception", exc); } // Example code: function bar(x) { console.log('x is', x); if(x < 0) { Throw(new Error("error!")); } return x * 2; } function foo(x) { return bar(x); } Try( function() { console.log(foo(1)); console.log(foo(-1)); }, function(ex) { console.log("caught", ex); } ); When x is -1 in bar, it will throw an exception which will be handled by our handler. Step through the code to see.

There are far more complicated control constructs that you can implement using continuations, and we will look into many future in a future post.

Calling from the Outside

So far we have always invoked a continuation inside the callCC call. That means we are always only jumping up the stack, meaning we're trying to jump back to a previous stack frame.

There's a name for these kinds of continuations: escape continuations. These are a more limited continuation that can only be called within the dynamic extent of the function passed to callCC (in this case it would be callWithEscapeContinuation or callEC). A lot of things like exceptions can be implemented only with escape continuations.

The reason for the differentiation is performance. Escape continuations don't need to save the entire stack and they can assume that the stack frames at the point of the callEC call will always exist in memory whenever the continuation is invoked.

However, my implementation of continuations is full continuations. This is where things really start getting mind-bending. In future posts, we will use this technique to implement features like coroutines. It's worth looking at a simple example for now.

Within the callCC call, you can just return the continuation itself:

var value = callCC(cont => cont);

value will be the continuation, but we don't name it cont because it will be different values later in time when the continuation is invoked. value will be whatever value the continuation is invoked with. We can make this easily reusable by wrapping it into a function:

function currentContinuation() { return callCC(cont => cont); }

Now we can do things like:

function currentContinuation() { return callCC(cont => cont); } function foo() { var value = currentContinuation(); if(typeof value === "function") { console.log("got a continuation!"); // Do some stuff var x = 5; value(x * 2); } else { console.log("computation finished", value); } } foo(); We "fork" the control flow depending on if we got the continuation or a normal value.

This is really powerful because it shows that we can invoke a continuation from any point in time, and it all works.

The above example is trivial, so in the spirit of attempting to show more value, here is a more complex example. This implements a very basic form of a coroutine that can pause itself and resumed with a value.

function currentContinuation() { return callCC(cont => ({ __cont: cont })); } function pause() { var value = currentContinuation(); if(value.__cont) { throw value; } else { return value; } } function run(func) { try { return func(); } catch(e) { if(!e.__cont) { throw e; } var exc = e; return { send: function(value) { exc.__cont(value); } }; } }

When a coroutine calls pause, the continuation is thrown, the scheduler catches it, and returns an object that gives the caller the ability to resume it. A very simple program that uses this:

function foo() { var x = pause(); return x * 2; } var process = run(foo); if(process.send) { process.send(10); } else { console.log(process); }

The check for process.send is needed because our implementation is very naive. It saved the full continuation, which includes the top-level stack at the point when run is called. That means when the process is resumed, the top-level control is restored as well and we will see run return again.

Challenge: implement a version where process.send returns the final value instead of forcing the user to handle the return from run multiple times.

Here is the full program in an editor that lets you step through:

function currentContinuation() { return callCC(cont => ({ __cont: cont })); } function pause() { var value = currentContinuation(); if(value.__cont) { throw value; } else { return value; } } function run(func) { try { return func(); } catch(e) { if(!e.__cont) { throw e; } var exc = e; return { send: function(value) { exc.__cont(value); } }; } } function foo() { var x = pause(); return x * 2; } var process = run(foo); if(process.send) { process.send(10); } else { console.log(process); } The full program with a breakpoint already. Click "run" to study it.

In future posts, we will look at more robust techniques for implementing coroutines with continuations.

Closing Over Data

It's very important to understand that continuations only save the call stack, not any of the data that stack frames may reference. Restoring a continuation does not restore any of the variables that those stack frames use. In this way, think of each stack frame as a closure that simply references those variables, and any external changes will still be seen.

This is confusing for beginners, but hopefully this is a simple illustration:

function foo() { var x = 5; callCC(function(cont) { x = 6; cont(); }); console.log(x); } foo(); This will log 6 because the change to x is still seen after the continuation is restored. Capturing does not save the value of x.

It doesn't matter when the continuation is invoked. If we saved the continuation for later, changed some local variables, and returned from the function, when the continuation is invoked it will still see all the local variable changes. A continuation closes over its data.

Note: There may be bugs in my continuation implementation where this is not the case. In my implementation, I have to go out of my way to ensure that data is closed over instead of copied. If changes are need seen from continuations invoked later, that's a bug. The Backstory

There's a long history here, but I'll keep it short:

  • In 2011 I worked on an in-browser game editor and I wanted the ability to interactively debug code.

  • Around this time I implemented my own Scheme-inspired language, Outlet, and tried to make it debuggable. I did it with a continuation-passing-style (CPS) transformation, effectively implementing continuations, but this forced me to re-implement stacks and scopes. It was very slow (can't compete with native JS stacks & scopes). I blogged the details here:

  • In an attempt to use native JS function scoping, I thought about abusing generators to suspend functions. While I still needed to re-implement the stack, at least variables are native and the implementation is much simpler (generators were just landing in JS engines). I called it YPS and it works by yielding every single expression and running in a special machine. It was horrendously slow.

    • Later @amasad ended up doing exactly this and writing an awesome post about it.
  • In response to my generator-based suspension idea, @msimoni pointed me to the paper "Exceptional Continuations in JavaScript". I realized that what I wanted required all the machinery of continuations, and that paper outlines a technique to implement them without much run-time performance cost. Although capturing continuations is slow, all other code has a minimal perf hit.

That paper describes a really neat trick to implement continuations, which gives me the power to arbitrarily jump around code. Unfortunately it requires a sophisticated transformation, but right around this time regenerator came out which implemented a similar transformation! I forked regenerator, implemented continuations, got a stepping debugger working, and then realized that I could expose continuations first-class and be able to do all the things I'm about to show you. (That was around 2 years ago. This project sat on my computer for that time until I resurrected it a few weeks ago.)

Next: The Implementation Details

I was going to explore the implementation details in this post, but it's already so dense that I am pulling this out into a separate post. Check out the next post if you are interested in details!

Explore!

I think this is could be a fun playground for playing with various advanced control operators. I'm also pretty proud that I was able to get an in-browser stepping debugger working for interactive tutorials.

I will go into more advanced usages of continuations, particularly delimited continuations, in future posts.

If you are interested in this, check out unwinder!

Output: Stack: Run Run & Ignore Breakpoints Continue Step Hi

Categorieën: Mozilla-nl planet

Robert Kaiser: Tools I Wrote for Crash (Stats) Analysis

ma, 16/05/2016 - 22:30
Now that I'm off the job that dominated my life (and almost burned me out) for the last years, I finally have some time again to blog. And I'll start with stuff I actually did for that job, as I still am happy to help others to continue from where I left.

The more fun part of the stability management job was actually creating new analysis - and tools. And those tools are still helpful to people working on crash analysis or crash stats analysis now - so as my last task on the job, I wrote some documentation for the tools I had created.

One of the first things I created (and which was part of the original job description when I started) was a prototype for detecting crash "explosiveness", i.e. a detector for crashes that are rising significantly in volume. This turned out to be quite helpful for me and others to use, and the newest reports of it are listed in my Report Overview. I probably should talk about it in more detail at some point, but I did write up a plan on the wiki for the tool, and the (PHP) code is on hg.m.o (that was the language I knew best and gave me the fastest result for a prototype). I had plans to port/rewrite it in python, but didn't get to it. Calixte, who is looking after most of "my" tools now, is working on that though, and I have already promised to review his work as a volunteer so we can make sure we have this helpful capability in better code (and hopefully better UI in the end) for future use.

In general, I have created one-line docs for all the PHP scripts I had in the Mercurial repository, and put them into the run-reports script that is called by a daily cron job. Outside of the explosiveness script, most of those have been obsoleted by Socorro Super Search (yay for Adrian's work and for the ElasticSearch backend!) nowadays.

Also, the scripts that generate the summed-up data for Are We Stable Yet dashboard and graphs (also see an older blog post discussing the graphs) have been ported to python (thanks Peter for helping me to get started there) - and those are available in the Magdalena repository on GitHub. You'll see that this repository doesn't just have more modern code, using python instead of PHP and the public Socorro API instead of private PostgreSQL access, it also has a decent README documenting what it and every script in it does. :)

The most important tools for people analyzing crash stats are in the Datil repository on GitHub (and its deployment on crash-analysis), though. I used all those 4 dashboards/tools daily in the last months to determine what to report to Release Managers and other parties, find out what we need to file as bugs and/or push to get fixed. Datil, like Magdalena, has good docs right in the repository now, readable directly on GitHub.

So, what's there?
Well, the before-mentioned "Are We Stable Yet" dashboard and graphs, for sure (see the longtermgraph docs for what graphs you can get and a legend of what the lines mean).
There's also a tool/prototype for "what's important" weighed top crash lists that I called "Top Crash Score", see the score docs for what it does and examples on how to use that tool.
And finally, I created a search query comparison tools that did let me answer questions like "which crashes happen more with or without multi-process support (e10s) being active?" or "which crashes have vanished with the new beta and which have appeared (instead)?" - which was incredibly helpful to me at least. Read the searchcompare docs for more details and examples.

I probably won't spend a lot of time with those tools any more, neither in usage nor in development, but I'm still happy about people using them, giving me feedback, and I'm also happy to review and merge pull requests that feel like making sense to me!
Categorieën: Mozilla-nl planet

Pagina's