mozilla

Mozilla Nederland LogoDe Nederlandse
Mozilla-gemeenschap

Microsoft Browsers Can't Keep Up with Mozilla Firefox and Google Chrome - Softpedia News

Nieuws verzameld via Google - di, 17/05/2016 - 13:39

Softpedia News

Microsoft Browsers Can't Keep Up with Mozilla Firefox and Google Chrome
Softpedia News
Microsoft introduced Edge browser in Windows 10 with much fanfare, hoping that this way the company can become more relevant in the browser business and have a strong competitor against the likes of Google Chrome and Mozilla Firefox. And while ...
Firefox usage overtakes Microsoft's IE and Edge desktop browsers globally for ...VentureBeat
Firefox overtakes Microsoft's IE and Edge browsers, but Chrome continues to ...TechCrunch
Firefox is now more popular than Internet Explorer and Edge combinedTechRadar
The Christian Times
alle 13 nieuwsartikelen »
Categorieën: Mozilla-nl planet

The Mozilla Blog: Mozilla Expands Its National Gigabit Project to Austin, TX

Mozilla planet - di, 17/05/2016 - 13:13

Mozilla will provide $150,000 in funding, and also grow the local maker community, to spur gigabit innovation in Texas’ capital

When you couple lightning-fast Internet with innovative projects in the realms of education and workforce development, amazing things can happen.

That’s the philosophy behind the Mozilla Gigabit Community Fund, our joint initiative with the National Science Foundation and US Ignite. The Mozilla Gigabit Community Fund brings funding and staffing to U.S. cities equipped with gigabit connectivity, the next-generation Internet that’s 250-times faster than most other connections. Our goal: Spark the creation of groundbreaking, gigabit-enabled educational technologies so that more people of all ages and backgrounds can read, write, and participate on this next-generation Web.

As we just announced at the Gigabit City Summit in Kansas City, we’re expanding our gigabit work to the city of Austin, TX in August 2016. Selected from a list of contenders from across the country, Austin stood out due to its existing city-wide digital inclusion plan, active developer community, and growing informal education landscape. Beginning this fall, Mozilla will provide $150,000 in grant funding to innovative and local projects and tools that leverage Austin’s Google Fiber network. Think: 4K streaming in classrooms, immersive virtual reality, and more.

(In the existing Mozilla Gigabit cities of Chattanooga, TN and Kansas City, projects include real-time water monitoring systems, 3D learning tools for classrooms, and specialized technology for first responder training. Read more about those projects here.)

Individuals from the Chattanooga gigabit project Hyperaudio participate in a New York City Maker Party.

Individuals from the Chattanooga gigabit project Hyperaudio participate in a New York City Maker Party.

Mozilla is also investing in the makers and educators who make Austin great. We’ll help create Gigabit Hive Austin — a network of individuals, schools, nonprofits, museums, and other local organizations passionate about teaching and learning the Web. Hive Austin will be one of 14 Mozilla Hive networks and communities across four continents that teach web literacy and 21st-century skills.

Mozilla will open the first round of grant applications in Austin this August, and accept applications through October 18, 2016. Applicants and projects don’t have to be from Austin originally, but must be piloted locally. Click here to learn about the RFP process.

This spring, Mozilla is also providing $134,000 in new gigabit funding in Chattanooga and Kansas City. Funds will support projects that explore gigabit and robotics, big data, the Internet of Things, and more. Learn more.

Over the next two years, Mozilla will be expanding its Gigabit work to two additional cities. Interested in becoming a future Gigabit Hive city? We will reopen the city application process in late 2016.

Categorieën: Mozilla-nl planet

Mozilla Expands Its National Gigabit Project to Austin, TX

Mozilla Blog - di, 17/05/2016 - 13:13

Mozilla will provide $150,000 in funding, and also grow the local maker community, to spur gigabit innovation in Texas’ capital

When you couple lightning-fast Internet with innovative projects in the realms of education and workforce development, amazing things can happen.

That’s the philosophy behind the Mozilla Gigabit Community Fund, our joint initiative with the National Science Foundation and US Ignite. The Mozilla Gigabit Community Fund brings funding and staffing to U.S. cities equipped with gigabit connectivity, the next-generation Internet that’s 250-times faster than most other connections. Our goal: Spark the creation of groundbreaking, gigabit-enabled educational technologies so that more people of all ages and backgrounds can read, write, and participate on this next-generation Web.

As we just announced at the Gigabit City Summit in Kansas City, we’re expanding our gigabit work to the city of Austin, TX in August 2016. Selected from a list of contenders from across the country, Austin stood out due to its existing city-wide digital inclusion plan, active developer community, and growing informal education landscape. Beginning this fall, Mozilla will provide $150,000 in grant funding to innovative and local projects and tools that leverage Austin’s Google Fiber network. Think: 4K streaming in classrooms, immersive virtual reality, and more.

(In the existing Mozilla Gigabit cities of Chattanooga, TN and Kansas City, projects include real-time water monitoring systems, 3D learning tools for classrooms, and specialized technology for first responder training. Read more about those projects here.)

Individuals from the Chattanooga gigabit project Hyperaudio participate in a New York City Maker Party.

Individuals from the Chattanooga gigabit project Hyperaudio participate in a New York City Maker Party.

Mozilla is also investing in the makers and educators who make Austin great. We’ll help create Gigabit Hive Austin — a network of individuals, schools, nonprofits, museums, and other local organizations passionate about teaching and learning the Web. Hive Austin will be one of 14 Mozilla Hive networks and communities across four continents that teach web literacy and 21st-century skills.

Mozilla will open the first round of grant applications in Austin this August, and accept applications through October 18, 2016. Applicants and projects don’t have to be from Austin originally, but must be piloted locally. Click here to learn about the RFP process.

This spring, Mozilla is also providing $134,000 in new gigabit funding in Chattanooga and Kansas City. Funds will support projects that explore gigabit and robotics, big data, the Internet of Things, and more. Learn more.

Over the next two years, Mozilla will be expanding its Gigabit work to two additional cities. Interested in becoming a future Gigabit Hive city? We will reopen the city application process in late 2016.

Categorieën: Mozilla-nl planet

Karl Dubost: [worklog] From 17s to 0.5s, testing for performance

Mozilla planet - di, 17/05/2016 - 10:21

Tune of the week: Hip-Hop Mixtapes.

Webcompat Life

Progress this week:

Today: 2016-05-23T15:18:18.501043 354 open issues ---------------------- needsinfo 3 needsdiagnosis 119 needscontact 27 contactready 43 sitewait 157 ----------------------

You are welcome to participate

Londong agenda.

French Mozilla Community seems interested by Web Compatibility work. Replying to a thread where we asked me more information.

Seen this http-prompt library. Shared the link with mike who replied "wow! gonna install this". A couple of minutes later I opened a new issue for the missing HEAD method. Then I thought "Oh shoot! I can solve this". I created a pull request, which was merged in the following hour. ❤️ opensource.

Webcompat issues

(a selection of some of the bugs worked on this week).

WebCompat.com dev
  • We got a performance regression with a code I had written recently. Maybe there is a solution. A bit later. Found. Reminder to myself: do performance testing for any I/O operations before pushing them to prod. On Thursday, I did a lot of testing. Maybe I should write a separate post about this. It was an interesting experiment. Upload performances went from 17s to 0.5s
  • We probably need to block the about: URIs in reporting. It doesn't make sense in the context of our work.
Reading List
  • Evaluating scripts using Marionette: "I finally managed to land a patch that makes evaluated scripts have lasting side-effects in the document. This means that scripts such as window.foo = 42 or document.documentElement.style.background = "pink" will take effect on the DOM. This is a requirement for WebDriver conformance, and a major inhibitor for people trying to replace FirefoxDriver with Marionette in their automation. "
  • Web Standards: "Standards are made by the people who show up, and not nearly enough people are showing up. We need publishers and designers and engineers, typographers and librarians and entrepreneurs, web folks and book folks and business folks. We certainly need more diversity of gender and color and class."
  • GridViz: A Firefox extension to visualize grid lines defined by the CSS Grid API
Follow Your Nose TODO
  • Document how to write tests on webcompat.com using test fixtures.
  • ToWrite: rounding numbers in CSS for width
  • ToWrite: Amazon prefetching resources with <object> for Firefox only.

Otsukare!

Categorieën: Mozilla-nl planet

James Long: What’s in a Continuation

Mozilla planet - di, 17/05/2016 - 02:00

Many people have heard the word “continuation” because it has something to do with node’s callback hell. I don’t think most people understand what continuations really are, though. They aren’t just a callback function used by async functions.

A continuation is a representation of the control flow of your program at any point in time, essentially the stack. In abstract terms, it represents "the rest of your program." In languages like Scheme that expose continuations as first-class values, you can capture the current continuation and invoke it later. When invoked, the current program state is replaced with the state at which the continuation was captured (i.e. the current stack is replaced with the stack from the continuation).

Continuations allow you to literally "jump" to different places in your code. They are a low-level primitive that gives you control over execution flow, allowing you implement everything from resumable exceptions to coroutines. Understanding continuations was the best thing I did as a young programmer; it forces you to understand how control flow works.

It would be neat to see something like continuations implemented in JavaScript engines because you can implement everything on top of them (note that I said something like, as continuations themselves are very hard to optimize). I'm a fan of low-level primitives for the same reasons as the Extensible Web Manifesto: let users evolve the language over time.

I recently ended up implementing continuations in JavaScript. It wasn’t on purpose; I was originally pursuing a way to arbitrarily pause JavaScript in user-land so I could write tutorials and interactive editors. I realized that to arbitrarily pause JS, I would need all the machinery necessary for continuations. The ability to save a stack and resume it later. Eventually I discovered the paper "Exceptional Continuations in JavaScript" and was able to achieve my in-browser stepping debugger by implementing continuations.

I wrote more about the backstory of my work at the end of this post. I did most of this work 2 years ago and I'm now polishing it up and publishing it.

function foo(x) { console.log(x); if (x <= 0) { return x; } else { return x + foo(x - 1); } } function main() { console.log(foo(3)); } main(); A working stepping debugger! Click on any line to add a breakpoint.

And just like that, I realized that I could expose continuations first-class to this special variant of JS, which is way more interesting than my stepping debugger. In this article, I will use my work to explain what continuations are and give you a chance to interact with them.

I will explain how it is implemented in the next post. A short version: it transforms all code into a state machine and uses exceptions to save the state of all functions on the stack. This means that every function is transformed into a big switch statement with every expression as separate cases, giving us the ability to arbitrarily jump around.

This transform is very similar to what regenerator does, which compiles JavaScript generators to ES5 code. In fact, that project is what motivated this work. Two years ago, I forked regenerator and implemented everything you see here. That means that it doesn't support a lot of recent ES6 features and it's missing a lot of bug fixes.

Visit the unwinder repo to see the code and try it yourself. Warning: this is very prototype quality and many things are ugly. There's a very good chance that you will hit bugs. However, with some polish work this has a chance to become a place where we can explore interesting patterns.

A few other caveats:

  • You cannot step through native frames, or use continuations when they are on the stack. If you use the native array forEach and capture the continuation in the callback, things will go badly. This requires all code to be compiled through this transformer if you want to use continuations (normal code can call out to native code just fine, however).

  • This technique favors performance of code that does not use continuations. Capturing continuations is not very fast, but if you are implementing something like a debugger, that doesn't matter. However, if you are implementing advanced control flow operators, you will likely hit performance problems. This is a good place to experiment with them though.

Introducing Continuations

Let's revisit the definition of a continuation; wikipedia describes it as "an abstract representation of the control state of a computer program. A continuation reifies the program control state…" The key words are control state. This means that when a continuation is created, it contains all the necessary information to resume the program exactly at the point in time which is was created.

This is how the stepping debugger works internally. The generated code looks for breakpoints, and when one is hit, it captures the current continuation and stops executing. Resuming is as simple as invoking the saved continuation.

Let's have some real fun though and expose continuations as first-class values! In Scheme, you use call-with-current-continuation to capture the current continuation, or the shorthand call/cc. Any experienced Scheme coder is familiar with code like this:

(define (foo) (let ([x (call/cc (lambda (cont) (display "captured continuation") (cont 5) (display "continuation called")))]) (display "returning x") x)) (display (foo))

I implemented a callCC function in my special JavaScript variant since it already has all the necessary machinery. Additionally, we can use the stepping debugger to study how continuations affect the control flow.

function foo() { var x = callCC(function(cont) { console.log("captured continuation"); cont(5); console.log("continuation called"); }); console.log("returning x", x); return x; } console.log(foo()); A basic example of continuations.

This is a very simple example of using a continuation. Click "Run & Ignore Breakpoints" to see what happens. We capture the continuation using callCC, which gives us the continuation as the function cont. We then log "captured continuation" and invoke cont. Note how "continuation called" is never logged. Why is that?

Now go back and click "Run" to hit the breakpoint on line 13, and continually click "Step" to step through the program to see what happened. What happens when cont is invoked?

It jumps back to line 3! The previous control flow is aborted and the stack when cont was captured is restored. Any arguments passed to continuations replace the call to callCC, as if callCC returned that value. The continuation represents a state of the program where the callCC function is waiting to return a value.

Note: While the interface for calling a continuation looks like calling a function, it's quite different. The call to a continuation never returns. While this may be confusing, we will look into alternative continuation interfaces in future posts. Also, continuations are a low-level interface and rarely used directly.

Continuations are like portals. If the image below represents your control flow, you can capture the current stack (the blue portal) and jump back to it at any time (the orange portals).

The control flow of a program. Jumping through the orange portals will reset the stack to where the blue portal was created.

However, just like in the game Portal, these portals do not traverse time. The only thing that a continuation saves is the stack, so any changes to variables will still be seen after jumping through a continuation. Look at what happens with a closure:

function foo() { var x = 5; var func = function() { return x; }; x = 6; return func; } console.log(foo()())

This will print 6 because closures reference the same variable that is mutated later. With continuations, it's the same thing except stack frames are pointing to the variables. We will show examples of this later.

Practical Uses

Now that you understand the general idea, let's put continuations to good use. This certainly seems powerful, but you might have trouble thinking of problems this solves. In fact, you might be thinking that this would just make programs harder to follow.

Abusing continuations definitely makes programs harder to follow. But there are constructs that continuations allow you to build that are generally helpful; break and continue might make your program a little harder to follow, but they solve real problems, just like other control flow operators. Additionally, in a future post we will talk about delimited continuations which force developers to use continuations in a clearer way.

The first exercise is to implement the JavaScript some method, which checks if any element in an array passes a predicate check. Importantly, it is "short-circuiting," meaning it stops iterating after finding the first passing element because it doesn't need to check the rest.

function some(predicate, arr) { var x = callCC(function(cont) { for (var idx = 0; idx < arr.length; idx++) { console.log('testing', arr[idx]); if(predicate(arr[idx])) { cont(true); } } return false; }); return x; } console.log(some(x => x >= 2, [1, 2, 3, 4])); An implementation of some using continuations.

If you run this, you'll notice that it doesn't check 3 or 4. It stops executing once 2 passes the predicate check. Step through the code and watch how it does that.

Of course, we could use break to stop the while loop. But this is a trivial example; it's common to call out to other functions within the loop where you can't break. The native control operators are quite limiting. Continuations, however, allow you to travel across stack frames.

For example, let's say you wanted to use forEach method instead of a for loop, because you already use that method everywhere else. Here is the example now:

// Note: don't use native forEach so you can step // into this one function forEach(arr, func) { for (var i=0; i < arr.length; i++) { func(arr[i]); } } function some(predicate, arr) { var x = callCC(function(cont) { forEach(arr, function(val) { console.log('testing', val); if(predicate(val)) { cont(true); } }); return false; }); return x; } console.log(some(x => x === 2, [1, 2, 3, 4])); An implementation of some using continuations across stack frames.

It works exactly the same way, even if we are calling the predicate within the function passed to forEach. It still short-circuits. Note how we didn't have to change anything about forEach; we are able to reuse the same method that we already use everywhere else.

This highlights a fundamental difference of continuations and anything currently in JavaScript: it suspends the entire stack. Generators suspend code as well, but their yield is shallow. It only suspends one frame, the generator itself.

While that makes code clearer, it leads to a proliferation of special syntax across all code and forces a lot of work on the developer over the lifetime of a project. Converting a single sync function to async requires a massive refactoring, changing the interface of every thing that uses it. I recommend reading "What Color is Your Function?" for a great description of this problem.

In the next post we will show how having a single function interface (no function* or async function) and deep stack control greatly improves the reusability and readability of code.

Exception Handling

Let's get real. The above exercises are pretty stupid. You wouldn't actually use continuations like that; there are much better constructs for looping over values and short-circuiting. Those examples were simple on purpose for illustrative purposes.

Now we will implement a new fundamental control construct: exceptions. This shows that continuations allow you implement things previously built-in to the language.

Users should be able to throw exceptions and install handlers to catch them. Installed handlers are dynamically scoped for a given section of code: any exception that occurs within a given block of code, even if it comes from an external function, should be caught.

Exception handlers must exist as a stack: you can install new handlers that override existing ones for a given period of time, but the previous ones are always restored once the newer ones are "popped" off the stack. So we must manage a stack.

The stack is a list of continuations, because when a throw happens we need to be able to jump back to where the try was created. They means in try we need to capture the current continuation, push it on the stack, run the code, and dispatch exceptions. Here is the full implementation of try/catch:

var tryStack = []; function Try(body, handler) { var ret = callCC(function(cont) { tryStack.push(cont); return body(); }); tryStack.pop(); if(ret.__exc) { return handler(ret.__exc); } return ret; } function Throw(exc) { if(tryStack.length > 0) { tryStack[tryStack.length - 1]({ __exc: exc }); } console.log("unhandled exception", exc); }

The key here is that continuations can be resumed with values. The return body() line will return the final value of the code. At that point no continuation was invoked; it just passes that value through. But if Throw is invoked, it will call the captured continuation with an exception value, which gets assigned to ret, and we check for that type of value and call the handler. (We could do more sophisticated detection of exception types.)

Note that we pop the current handler off the stack before calling it, meaning that any exceptions that occur within exception handlers will properly be passed up the handler stack.

Here's what it looks like using Try/Catch:

function bar(x) { if(x < 0) { Throw(new Error("error!")); } return x * 2; } function foo(x) { return bar(x); } Try( function() { console.log(foo(1)); console.log(foo(-1)); }, function(ex) { console.log("caught", ex); } );

Unfortunately JavaScript does not allow us to extend syntax (although this can be solved with sweet.js macros, as we'll show in future posts). Instead of using blocks we must pass functions into Try. The output of this code would be 2 \n caught "error!".

The above implementation and example code are loaded into the editor below, with a breakpoint already set at the Try block. Hit "Run & Ignore Breakpoints" to verify the output, and "Run" to break and step through the code to see how it unfolds.

var tryStack = []; function Try(body, handler) { var ret = callCC(function(cont) { tryStack.push(cont); return body(); }); console.log('ret is', JSON.stringify(ret)); tryStack.pop(); if(ret.__exc) { return handler(ret.__exc); } return ret; } function Throw(exc) { if(tryStack.length > 0) { tryStack[tryStack.length - 1]({ __exc: exc }); } console.log("unhandled exception", exc); } // Example code: function bar(x) { console.log('x is', x); if(x < 0) { Throw(new Error("error!")); } return x * 2; } function foo(x) { return bar(x); } Try( function() { console.log(foo(1)); console.log(foo(-1)); }, function(ex) { console.log("caught", ex); } ); When x is -1 in bar, it will throw an exception which will be handled by our handler. Step through the code to see.

There are far more complicated control constructs that you can implement using continuations, and we will look into many future in a future post.

Calling from the Outside

So far we have always invoked a continuation inside the callCC call. That means we are always only jumping up the stack, meaning we're trying to jump back to a previous stack frame.

There's a name for these kinds of continuations: escape continuations. These are a more limited continuation that can only be called within the dynamic extent of the function passed to callCC (in this case it would be callWithEscapeContinuation or callEC). A lot of things like exceptions can be implemented only with escape continuations.

The reason for the differentiation is performance. Escape continuations don't need to save the entire stack and they can assume that the stack frames at the point of the callEC call will always exist in memory whenever the continuation is invoked.

However, my implementation of continuations is full continuations. This is where things really start getting mind-bending. In future posts, we will use this technique to implement features like coroutines. It's worth looking at a simple example for now.

Within the callCC call, you can just return the continuation itself:

var value = callCC(cont => cont);

value will be the continuation, but we don't name it cont because it will be different values later in time when the continuation is invoked. value will be whatever value the continuation is invoked with. We can make this easily reusable by wrapping it into a function:

function currentContinuation() { return callCC(cont => cont); }

Now we can do things like:

function currentContinuation() { return callCC(cont => cont); } function foo() { var value = currentContinuation(); if(typeof value === "function") { console.log("got a continuation!"); // Do some stuff var x = 5; value(x * 2); } else { console.log("computation finished", value); } } foo(); We "fork" the control flow depending on if we got the continuation or a normal value.

This is really powerful because it shows that we can invoke a continuation from any point in time, and it all works.

The above example is trivial, so in the spirit of attempting to show more value, here is a more complex example. This implements a very basic form of a coroutine that can pause itself and resumed with a value.

function currentContinuation() { return callCC(cont => ({ __cont: cont })); } function pause() { var value = currentContinuation(); if(value.__cont) { throw value; } else { return value; } } function run(func) { try { return func(); } catch(e) { if(!e.__cont) { throw e; } var exc = e; return { send: function(value) { exc.__cont(value); } }; } }

When a coroutine calls pause, the continuation is thrown, the scheduler catches it, and returns an object that gives the caller the ability to resume it. A very simple program that uses this:

function foo() { var x = pause(); return x * 2; } var process = run(foo); if(process.send) { process.send(10); } else { console.log(process); }

The check for process.send is needed because our implementation is very naive. It saved the full continuation, which includes the top-level stack at the point when run is called. That means when the process is resumed, the top-level control is restored as well and we will see run return again.

Challenge: implement a version where process.send returns the final value instead of forcing the user to handle the return from run multiple times.

Here is the full program in an editor that lets you step through:

function currentContinuation() { return callCC(cont => ({ __cont: cont })); } function pause() { var value = currentContinuation(); if(value.__cont) { throw value; } else { return value; } } function run(func) { try { return func(); } catch(e) { if(!e.__cont) { throw e; } var exc = e; return { send: function(value) { exc.__cont(value); } }; } } function foo() { var x = pause(); return x * 2; } var process = run(foo); if(process.send) { process.send(10); } else { console.log(process); } The full program with a breakpoint already. Click "run" to study it.

In future posts, we will look at more robust techniques for implementing coroutines with continuations.

Closing Over Data

It's very important to understand that continuations only save the call stack, not any of the data that stack frames may reference. Restoring a continuation does not restore any of the variables that those stack frames use. In this way, think of each stack frame as a closure that simply references those variables, and any external changes will still be seen.

This is confusing for beginners, but hopefully this is a simple illustration:

function foo() { var x = 5; callCC(function(cont) { x = 6; cont(); }); console.log(x); } foo(); This will log 6 because the change to x is still seen after the continuation is restored. Capturing does not save the value of x.

It doesn't matter when the continuation is invoked. If we saved the continuation for later, changed some local variables, and returned from the function, when the continuation is invoked it will still see all the local variable changes. A continuation closes over its data.

Note: There may be bugs in my continuation implementation where this is not the case. In my implementation, I have to go out of my way to ensure that data is closed over instead of copied. If changes are need seen from continuations invoked later, that's a bug. The Backstory

There's a long history here, but I'll keep it short:

  • In 2011 I worked on an in-browser game editor and I wanted the ability to interactively debug code.

  • Around this time I implemented my own Scheme-inspired language, Outlet, and tried to make it debuggable. I did it with a continuation-passing-style (CPS) transformation, effectively implementing continuations, but this forced me to re-implement stacks and scopes. It was very slow (can't compete with native JS stacks & scopes). I blogged the details here:

  • In an attempt to use native JS function scoping, I thought about abusing generators to suspend functions. While I still needed to re-implement the stack, at least variables are native and the implementation is much simpler (generators were just landing in JS engines). I called it YPS and it works by yielding every single expression and running in a special machine. It was horrendously slow.

    • Later @amasad ended up doing exactly this and writing an awesome post about it.
  • In response to my generator-based suspension idea, @msimoni pointed me to the paper "Exceptional Continuations in JavaScript". I realized that what I wanted required all the machinery of continuations, and that paper outlines a technique to implement them without much run-time performance cost. Although capturing continuations is slow, all other code has a minimal perf hit.

That paper describes a really neat trick to implement continuations, which gives me the power to arbitrarily jump around code. Unfortunately it requires a sophisticated transformation, but right around this time regenerator came out which implemented a similar transformation! I forked regenerator, implemented continuations, got a stepping debugger working, and then realized that I could expose continuations first-class and be able to do all the things I'm about to show you. (That was around 2 years ago. This project sat on my computer for that time until I resurrected it a few weeks ago.)

Next: The Implementation Details

I was going to explore the implementation details in this post, but it's already so dense that I am pulling this out into a separate post. Check out the next post if you are interested in details!

Explore!

I think this is could be a fun playground for playing with various advanced control operators. I'm also pretty proud that I was able to get an in-browser stepping debugger working for interactive tutorials.

I will go into more advanced usages of continuations, particularly delimited continuations, in future posts.

If you are interested in this, check out unwinder!

Output: Stack: Run Run & Ignore Breakpoints Continue Step Hi

Categorieën: Mozilla-nl planet

Robert Kaiser: Tools I Wrote for Crash (Stats) Analysis

Mozilla planet - ma, 16/05/2016 - 22:30
Now that I'm off the job that dominated my life (and almost burned me out) for the last years, I finally have some time again to blog. And I'll start with stuff I actually did for that job, as I still am happy to help others to continue from where I left.

The more fun part of the stability management job was actually creating new analysis - and tools. And those tools are still helpful to people working on crash analysis or crash stats analysis now - so as my last task on the job, I wrote some documentation for the tools I had created.

One of the first things I created (and which was part of the original job description when I started) was a prototype for detecting crash "explosiveness", i.e. a detector for crashes that are rising significantly in volume. This turned out to be quite helpful for me and others to use, and the newest reports of it are listed in my Report Overview. I probably should talk about it in more detail at some point, but I did write up a plan on the wiki for the tool, and the (PHP) code is on hg.m.o (that was the language I knew best and gave me the fastest result for a prototype). I had plans to port/rewrite it in python, but didn't get to it. Calixte, who is looking after most of "my" tools now, is working on that though, and I have already promised to review his work as a volunteer so we can make sure we have this helpful capability in better code (and hopefully better UI in the end) for future use.

In general, I have created one-line docs for all the PHP scripts I had in the Mercurial repository, and put them into the run-reports script that is called by a daily cron job. Outside of the explosiveness script, most of those have been obsoleted by Socorro Super Search (yay for Adrian's work and for the ElasticSearch backend!) nowadays.

Also, the scripts that generate the summed-up data for Are We Stable Yet dashboard and graphs (also see an older blog post discussing the graphs) have been ported to python (thanks Peter for helping me to get started there) - and those are available in the Magdalena repository on GitHub. You'll see that this repository doesn't just have more modern code, using python instead of PHP and the public Socorro API instead of private PostgreSQL access, it also has a decent README documenting what it and every script in it does. :)

The most important tools for people analyzing crash stats are in the Datil repository on GitHub (and its deployment on crash-analysis), though. I used all those 4 dashboards/tools daily in the last months to determine what to report to Release Managers and other parties, find out what we need to file as bugs and/or push to get fixed. Datil, like Magdalena, has good docs right in the repository now, readable directly on GitHub.

So, what's there?
Well, the before-mentioned "Are We Stable Yet" dashboard and graphs, for sure (see the longtermgraph docs for what graphs you can get and a legend of what the lines mean).
There's also a tool/prototype for "what's important" weighed top crash lists that I called "Top Crash Score", see the score docs for what it does and examples on how to use that tool.
And finally, I created a search query comparison tools that did let me answer questions like "which crashes happen more with or without multi-process support (e10s) being active?" or "which crashes have vanished with the new beta and which have appeared (instead)?" - which was incredibly helpful to me at least. Read the searchcompare docs for more details and examples.

I probably won't spend a lot of time with those tools any more, neither in usage nor in development, but I'm still happy about people using them, giving me feedback, and I'm also happy to review and merge pull requests that feel like making sense to me!
Categorieën: Mozilla-nl planet

Mozilla vs FBI per la sicurezza di Tor - Data manager online

Nieuws verzameld via Google - ma, 16/05/2016 - 20:16

Data manager online

Mozilla vs FBI per la sicurezza di Tor
Data manager online
Il braccio di ferro tra le compagnie hi-tech e l'FBI è destinato a continuare, solcando fronti e ambiti differenti. L'ultimo caso riguarda una richiesta mossa da Mozilla, sviluppatrice di Firefox, nei confronti dell'organo federale che, nel recente ...

Google Nieuws
Categorieën: Mozilla-nl planet

Mozilla Issues Legal Challenge to FBI to Disclose Firefox Flaw - eWeek

Nieuws verzameld via Google - ma, 16/05/2016 - 20:01

eWeek

Mozilla Issues Legal Challenge to FBI to Disclose Firefox Flaw
eWeek
Open-source organization Mozilla wants the FBI to reveal details about software flaws that might impact its Firefox Web browser. Denelle Dixon-Thayer, Mozilla's chief legal and business officer, is optimistic that a legal challenge will be successful ...
The FBI and the Mozilla Foundation Lock Horns over Known Security HoleLinux Journal
Mozilla Pushes Feds To Disclose Firefox ExploitNewsFactor Network
FBI, Russia Team Up to Destroy Firefox, Hillary and AppleClapway

alle 8 nieuwsartikelen »
Categorieën: Mozilla-nl planet

Air Mozilla: Mozilla Weekly Project Meeting, 16 May 2016

Mozilla planet - ma, 16/05/2016 - 20:00

Mozilla Weekly Project Meeting The Monday Project Meeting

Categorieën: Mozilla-nl planet

Varun Joshi: Multifile Responses in Balrog

Mozilla planet - ma, 16/05/2016 - 19:59

Apart from Firefox, Balrog is also used by Mozilla to provide updates for the Gecko Media Plugin (GMP) package. The Gecko Media Plugin package contains various plugins for media support, like the OpenH264 codec and the Widevine plugin. To handle updates to these, we have a speical GMP blob that lists updates to the plugins. Updates to every plugin are included in one blob. This leads to problems when there are multiple versions of a plugin that we can use. For example, we might serve OpenH264 version 1.5.3 on Firefox 42 on Windows and version 1.5.2 on Firefox 40. We have to maintain a blob for each possible combination of versions. With an increase in the number of versions available, this method of serving updates might become intractable.

Ben and I discussed various strategies we could use to tackle this. A gist of what went through our minds is on the bug page. What we eventually went ahead with was to add a blob type that got its contents from other blobs. We call this blob type SuperBlob.

SuperBlob

A SuperBlob is basically just a redirection mechanism. It just contains the names of the products that we wish to include in the generated XML.:

{ "name": "fake", "schema_version": 4000, "products": [ "c", "d" ] }

This superblob is called fake. It just asks to look at the products named c and d. A rule can be set to point at this SuperBlob if we wish the response product to have all the files listed in products c and d.

How does this work?

The web interface, while processing requests, checks if the rule evaluates to a SuperBlob. If it is, it gets the product names from the SuperBlob and gets the corresponding blobs by evaluating the query with the requested product changed. So, in our example, the web interface will evaluate the query with the product name changed to c and d and obtain the resultant blobs. It will pick up the header and footer XML from the output of processing the blob obtained from the first product and the inner XML will include the concatenated inner XMLs of all the blobs obtained from the products listed in the SuperBlob.

So, if product a gave:

<updates> <update type="minor" version="None" extensionVersion="2.5" buildID="25"> <patch type="complete" URL="http://a.com/b" hashFunction="sha512" hashValue="23" size="27777777"/> </update> </updates>

and product b gave:

<updates> <update type="minor" version="None" extensionVersion="2.5" buildID="25"> <patch type="complete" URL="http://a.com/public" hashFunction="sha512" hashValue="23" size="22"/> </update> </updates>

the SuperBlob will give:

<updates> <update type="minor" version="None" extensionVersion="2.5" buildID="25"> <patch type="complete" URL="http://a.com/public" hashFunction="sha512" hashValue="23" size="22"/> <patch type="complete" URL="http://a.com/b" hashFunction="sha512" hashValue="23" size="27777777"/> </update> </updates>

So, we can now have one fixed rule for all GMP responses and have several rules for each constituent plugin without having to worry about the combinations like we had to do earlier.

While working on this, I also removed the createXML method in favour of three methods that return the header, the inner XML and the footer respectively. This helped in seperating the various components of the XML output without having to parse it. The XML generation logic has moved to the client view.

Categorieën: Mozilla-nl planet

Tarek Ziadé: Consistent Load Balancing

Mozilla planet - ma, 16/05/2016 - 19:50

This is an interesting problem to look at. Let's say you have 1 million users and 5 storage servers. You are building an application that needs to pick one of the 5 storage servers depending on the user id.

You want your app to do it as consistently as possible. e.g. a given user should always land on the same backend.

When a request comes in:

  • each request is identified by a unique user id.
  • the app should always call the same storage server given that id.
  • you want all your users to be distributed equally across your storage servers.
  • when a storage server is removed from the list, you want users to stick with the servers they initially got. For users on the removed server, they should be dispatched equally on other servers.
  • when a server is added, you want the minimal numbers of users to be moved around.
  • The application is stateless about this, so when I deploy a new node and give it the list of the storage servers, it should be able to start distributing users among them without prior knowledge.

Point 4 and 6 discards a simple round-robin solution.

The solution to that problem is to build a deterministic function that projects a user id into the space composed of the servers. (yeah well, clustering I guess.)

There are two known algorithms to do that. The consistent hashing algorithm and the rendezvous hashing.

Consistent Hashing

Consistent Hashing is a hashing that can be used to minimize the shuffling of users when a server is removed or added.

This is how it's implemented:

  • each server name is converted into a unique number
  • that number is projected on an modulo interval (a circle)
  • every user is also converted into a unique number and projected on the circle
  • the server that's the closest to the user is picked

If you want nice drawing go here.

This is an elegant solution because removing a server keeps the rest stable, and adding one server shuffles a minimal number of users.

The conversion from a name to a integer is key here: you have to be deterministic but in the same time try to have the numbers randomly and kind-of-evenly distributed on the circle.

Here's how you can do it using MD5:

import hashlib def hash(key): return long(hashlib.md5(key).hexdigest(), 16)

Using a classical hash like MD5 gives us the random part, but depending on the server name you might well end up with two servers that are very close to each other on the circle

And the result will be that when the users are converted into numbers, a very small amount of users will go to some servers.

One solution to fix that is to add replicas: for each server, instead of projecting a single number on the circle, we'll project 100. For example, "server1" becomes "server1:1", "server1:2", .., "server1:100" and those values are transformed into numbers.

Using replicas is very efficient to make sure users are spread evenly.

RendezVous

The other algorithm is called RendezVous and is based on a similar idea where servers are converted into numbers with a hash.

The difference is that instead of projecting servers and their replicas on a circle, the algorithm uses weights. To find which server a user should use, for each combination of server and user, a number is created with a classical hash function.

The server that's picked is the one with the highest number.

The Python code looks like this:

def get_server(user): high_score = -1 winner = None for server in server: score = hash(server + user) if score > high_score: high_score, winner = score, ip elif score == high_score: high_score, winner = score, max(server, winner) return winner

The advantage of this method is that you don't have to create replicas to worry about distribution. In fact, according to my tests, RendezVous is doing a better job than Consistent Hashing for distributing users.

One key decision is to decide which hashing algorithm you want to use.

It's all about the hashing!

Both RendezVous and Consistent hashing are using a classical hashing function to convert the servers and users into numbers - and picking one was not obvious.

I came across this amazing stackexchange post that shed some light on different hashing, their strengths and weaknesses. You should read it, you will learn a lot.

The take away for my experiment was that some hashing functions are doing a better job at randomizing values in the hash space. Some very simple functions are also colliding quite often, which can be a problem.

So I tried a bunch of them and benchmarked them.

It's interesting to note that I had much better results for my use case using RendezVous & sha256 than RendezVous & Murmur, when the latter is usually what people use with RendezVous.

I ended up removing Murmur from my tests, the results where too bad.

Anyways, here's the full implementation I did, based on snippets I found here and there, and the result:

The gist

And the winnner is : RendezVous and sha256

Of course, that entirely depends on how many servers & users you have.

Categorieën: Mozilla-nl planet

Mozilla grants boost local 3D modeling, literacy, STEM - Startland News (registration) (blog)

Nieuws verzameld via Google - ma, 16/05/2016 - 19:49

Mozilla grants boost local 3D modeling, literacy, STEM
Startland News (registration) (blog)
“Grantees will utilize the awarded funds to build, pilot and scale gigabit-enabled applications and associated curricula that have immediate, measurable impact on classrooms and informal learning organizations,” Mozilla wrote in a blog post. “Through ...

Google Nieuws
Categorieën: Mozilla-nl planet

Chris Cooper: RelEng & RelOps Weekly highlights - May 16, 2016

Mozilla planet - ma, 16/05/2016 - 19:18

REQUEST FOR URGENT BUSINESS RELATIONSHIP

FIRST, I MUST SOLICIT YOUR STRICTEST CONFIDENCE IN THIS MIGRATION. THIS IS BY VIRTUE OF ITS NATURE AS BEING UTTERLY CONFIDENTIAL AND ‘TOP SECRET’. I AM SURE AND HAVE CONFIDENCE OF YOUR ABILITY AND RELIABILITY TO HANDLE A TASK OF THIS GREAT MAGNITUDE INVOLVING A PENDING MIGRATION REQUIRING MAXIMUM CONFIDENCE.

WE ARE TOP OFFICIAL OF THE RELENG TEAM WHO ARE INTERESTED IN MIGRATION OF TASKS INTO TASKCLUSTER WITH JOBS WHICH ARE PRESENTLY TRAPPED IN BUILDBOT. IN ORDER TO COMMENCE THIS BUSINESS WE SOLICIT YOUR ASSISTANCE TO ENABLE US MIGRATE INTO YOUR TASKGRAPH THE SAID TRAPPED JOBS.

<marked as spam><deleted>

Modernize infrastructure:

The ongoing work to get TaskCluster building Firefox on Windows reached an important milestone with our first Mozharness based build going green in the TreeHerder dashboard. This represents underlying effort in documenting the dependency chain for Windows builders and producing simple manifests that give greater transparency to changes in this area.

Alin, Amy, and Van brought 192 new OS X 10.10.5 Mac minis online. This should eliminate our Yosemite backlog and allow us to enable more e10s tests.

Q, catlee, and jmaher migrated the first batch of Windows 7 tests to AWS this week. Currently we’re running these suites for all branches of gecko 49 and higher: Web platform tests + reftests, gtest, cppunit, jittest, jsreftest, crashtest. This will reduce our reliance on hardware and allow us to scale dynamically. They are still working on greening larger sets of tests which are more sensitive to their operating environment. Once we have moved a significant portion of tests, we can add additional e10s tests on w7 as well.

Improve Release Pipeline:

In the interest of making progress on migrating Nightly builds to TaskCluster, we had a meeting last week to discuss the security requirements around our nightly release process. Based on the discussions in that meeting, Aki is now iterating on a “one graph” solution for Nightlies (as opposed to a two-graph approach where signing is separate). If this approach works, i.e. we can’t find major security holes in the proposed model, it will simplify our process greatly.

Improve CI Pipeline:

We have achieved our first deprecation milestone in the TaskCluster migration by turning off Linux 64-bit debug builds/tests for aurora/trunk branches in buildbot. These bugs are only generated in TaskCluster now.

Aki added scriptworker to readthedocs: http://scriptworker.readthedocs.io/en/latest/

Operational:

We had a planned tree closing window over the weekend where, amongst other things, we upgraded our database hosts. It went smoothly, so you probably didn’t even notice.

Our new buildduty associates from Taipei were able to successfully loan machines to developers for the first time last week. Welcome aboard!

Release:

Lots of releases published or in-flight: Firefox 47 betas 3-5, Fennec 47.0b4, and Thunderbird 38.8.0 and 45.1.0. As always, you can find more details in our post-mortem wiki pages:

See you next week, freelance writer!

Categorieën: Mozilla-nl planet

Will Kahn-Greene: Anatomy of a class/function decorator and context manager

Mozilla planet - ma, 16/05/2016 - 18:00

Over the weekend, I wanted to implement something that acted as both a class and function decorator, but could also be used as a context manager. I needed this flexibility for overriding configuration values making it easier to write tests. I wanted to use it in the following ways:

  1. as a function decorator:

    @config_override(DEBUG='False') def test_something(): ...
  2. as a class decorator that would decorate all methods that start with test_:

    @config_override(DEBUG='False') class TestSomething: def test_something(self): ...
  3. as a context manager that allowed for multiple layer of overriding:

    def test_something(): with config_override(DEBUG='False'): with config_override(SOMETHING_ELSE='ou812'): ...

This kind of need comes up periodically, but infrequently enough that I forget how I wrote it the last time around.

This post walks through how I structured it.

Read more… (3 mins to read)

Categorieën: Mozilla-nl planet

Mozilla Sparks Gigabit Innovation In Chattanooga With $75000 In Grants - The Chattanoogan

Nieuws verzameld via Google - ma, 16/05/2016 - 17:21

Mozilla Sparks Gigabit Innovation In Chattanooga With $75000 In Grants
The Chattanoogan
As part of the Mozilla Community Gigabit Fund, Mozilla is supporting local Chattanooga organizations that are leveraging the city's gigabit Internet to create amazing things. Six organizations will be receiving $75,000 in grant funds this year. They ...

Categorieën: Mozilla-nl planet

Mozilla and Linux Foundation Advance New Trends in Open Source Funding - The VAR Guy

Nieuws verzameld via Google - ma, 16/05/2016 - 15:40

The VAR Guy

Mozilla and Linux Foundation Advance New Trends in Open Source Funding
The VAR Guy
The Mozilla Foundation has long injected money into the open source ecosystem through partnerships with other projects and grants. But it formalized that mission last year by launching MOSS, which originally focused on supporting open source projects ...

Categorieën: Mozilla-nl planet

Giorgos Logiotatidis: Takis - A util that blocks until a port is open.

Mozilla planet - ma, 16/05/2016 - 15:20

Over at Mozilla's Engagement Engineering we use Docker to ship our websites. We build the docker images in CI and then we run tests against them. Our tests usually need a database or a cache server which you can get it running simply with a single command:

docker run -d mariadb

The problem is that this container will take some time to initialize and become available to accept connections. Depending on what your test and how you run your tests this delay can cause a test failure to due database connection timeouts.

We used to wait on executing our tests with sleep command but that -besides being an ugly hack- will not always work. For example you may set sleep timeout to 10 seconds and due to CI server load database initialization takes 11 seconds. And nobody wants a non-deterministic test suite.

Meet Takis. Takis checks once per second if a host:port is open. Once it's open, it just returns. It blocks the execution of your pipeline until services become available. No messages or other output to get in the way of your build logs. No complicated configuration either: It reads CHECK_PORT and optionally CHECK_HOST environment variables, waits and eventually returns.

Takis is build using Go and it's fully statically linked as Adriaan explains in this intriguing read. You can download it and directly use it in your scripts

~$ wget https://github.com/glogiotatidis/takis/raw/master/bin/takis ~$ chmod +x takis ~$ CHECK_PORT=3306 ./takis

or use it's super small Docker image

docker run -e CHECK_PORT=3306 -e CHECK_HOST=database.example.com giorgos/takis

For example here's how we use it to build Snippets Service in TravisCI:

script: - docker run -d --name mariadb -e MYSQL_ALLOW_EMPTY_PASSWORD=yes -e MYSQL_DATABASE=snippets mariadb:10.0 # Wait mariadb to initialize. - docker run --link mariadb:db -e CHECK_PORT=3306 -e CHECK_HOST=db giorgos/takis - docker run --env-file .env --link mariadb:db mozorg/snippets:latest coverage run ./manage.py test

My colleague Paul also build urlwait, a Python utility and library with similar functionality that can be nicely added to your docker-compose workflow to fight the same problem. Neat!

Categorieën: Mozilla-nl planet

Top 10 most read: Mozilla-FBI legal case, DVLA digital driving licence plans, top ... - V3.co.uk

Nieuws verzameld via Google - ma, 16/05/2016 - 12:08

V3.co.uk

Top 10 most read: Mozilla-FBI legal case, DVLA digital driving licence plans, top ...
V3.co.uk
An interesting week in the technology world saw several notable stories hit the headlines and prove of interest to V3 readers. Of particular note was the legal challenge by Mozilla to try to force the FBI to disclose how it hacked the Tor browser ...

en meer »
Categorieën: Mozilla-nl planet

The FBI and the Mozilla Foundation Lock Horns over Known Security Hole - Linux Journal

Nieuws verzameld via Google - ma, 16/05/2016 - 11:34

V3.co.uk

The FBI and the Mozilla Foundation Lock Horns over Known Security Hole
Linux Journal
The Mozilla Foundation and the FBI recently have clashed over security weaknesses. The FBI is aware of a weakness in the Tor browser that may affect Firefox—it's a weakness the FBI has exploited during an investigation. Mozilla wants the FBI to reveal ...
Top 10 most read: Mozilla-FBI legal case, DVLA digital driving licence plans, top ...V3.co.uk

alle 2 nieuwsartikelen »
Categorieën: Mozilla-nl planet

Daniel Stenberg: curl user poll 2016

Mozilla planet - ma, 16/05/2016 - 10:25

It is time for our annual survey on how you use curl and libcurl. Your chance to tell us how you think we’ve done and what we should do next. The survey will close on midnight (central European time) May 27th, 2016.

If you use curl or libcurl from time to time, please consider helping us out with providing your feedback and opinions on a few things:

http://goo.gl/forms/e4CoSDEKde

It’ll take you a couple of minutes and it’ll help us a lot when making decisions going forward. Thanks a lot!

The poll is hosted by Google and that short link above will take you to:

https://docs.google.com/forms/d/1JftlLZoOZLHRZ_UqigzUDD0AKrTBZqPMpnyOdF2UDic/viewform

Categorieën: Mozilla-nl planet

Pagina's