mozilla

Mozilla Nederland LogoDe Nederlandse
Mozilla-gemeenschap

Abonneren op feed Mozilla planet
Planet Mozilla - http://planet.mozilla.org/
Bijgewerkt: 10 uur 2 min geleden

Mozilla Open Policy & Advocacy Blog: It’s the final countdown: US surveillance reform nearing the finish line

do, 28/05/2015 - 07:02

The USA FREEDOM Act was born out of the shocking revelations of overbroad government surveillance that began two years ago. In its short life (it was first introduced in 2013) the USA FREEDOM Act has gone through many iterations; along with many others we have supported this legislation in our push for broader surveillance reform. The USA FREEDOM Act, although not perfect and clearly a product of compromise, deserved our support. Yet, even after overwhelming support in the House, the Senate failed to do its job late last Friday night and failed to pass the USA FREEDOM Act (defeated by a 57-42 vote). Just three more votes were needed to move this legislation forward. The word “disappointing” doesn’t really express the proper sentiment.

Facing expiration (at midnight on May 31, 2015) of three legal authorities authorizing US surveillance programs, the Senate also declined on Friday night to advance a two-month reauthorization of these authorities (by a vote of 45-54). The strategic decision-making here is still confounding. Why wait until the night before a holiday recess when the House has already left to take these votes to the floor? Now, Senate Majority Leader Mitch McConnell – left with few options — is taking the unusual step of calling a Sunday May 31st session to try again to pass something just hours before these authorities sunset. This won’t be without several challenges.

Any paths for emergency reauthorization or any “compromises” will face filibuster threats from, among others, Senator Rand Paul, who took to the floor for 10 and half hours last week to lambast the surveillance state and refused Majority Leader McConnell’s attempts on Friday evening to extend these expiring provisions by even one day. Senator Paul’s hardline opposition, combined with the nuances of the Senate’s arcane procedures and the presence of a number of other privacy champions in the Senate, means it will be nearly impossible to pass legislation in such a short window of time.

On the House side, Representatives passed the USA FREEDOM Act overwhelmingly. The House is scheduled to return from recess and vote no earlier than 6:30PM on June 1st – well after the authorities sunset. There may be some last minute wrangling during this recess week, but without some agreement from the House to take up an interim reauthorization prior to the May 31st expiration, it appears that the Senate must either pass the House version of USA FREEDOM or let these authorities sunset. Put another way, if the Senate doesn’t pass the House version of USA FREEDOM, there will be a gap in authorized collection by the intelligence community. Justice Department officials have indicated that the NSA has already begun shutting down the 215 programs so as to be in compliance if a sunset occurs.

So what actually happens if these provisions sunset? It is tempting to view a sunset as a privacy win, as the government would be restrained from bulk collection of our telephone records after all. While there’s truth to that, we can and should expect to see the national security hawks and the intelligence community push hard to reinstate some version of these authorities. A likely possibility is a Senate effort to pass a weaker “reform” bill, a watered down USA FREEDOM Act. For example, several senators have been vocal about their belief in the need for a data retention mandate — which we’ve spoken out against before — so if these authorities expire, we fear post-sunset legislative efforts will likely include such a requirement as a pathway to 60 votes in the Senate. Already Senate Intelligence Chair Richard Burr and Vice Chair Diane Feinstein have each floated bills to this end. We oppose both of these bills; they do not provide the reforms that we need.

However, as Harley Geiger of CDT notes in a recent post on Lawfare: “A few hours’ sunset may not make much difference to beltway insiders, but a post-sunset vote re-instating Sec. 215 can correctly be seen by voters as a huge expansion in government surveillance power after that power had gone dormant.” While the Senate may look to compromise after sunset, the House may well lean the other direction. House Judiciary Chairman Goodlatte and Ranking Member Conyers have indicated in a recent statement, “If the Senate chooses to allow these authorities to expire, they should do so knowing that sunset may be permanent.”

Yet, permanent sunset of Section 215 would not satisfy the Senate, and it would not be a panacea for civil liberties. The government could still conduct bulk collection in a few different ways:

  1. The government could use the FISA pen register/trap and trace statute (PR/TT), for example, which the government has previously used to conduct bulk collection of email metadata (the program allegedly ended in 2011 due to a lack of resources and demonstrated efficacy). It’s worth noting that while bulk collection under PR/TT is curtailed in the USA FREEDOM Act, this statute doesn’t otherwise have a sunset.
  2. Through other legal pathways — the vast majority of mass surveillance occurs under other authorities such as Executive Order 12333 and Section 702 of the FISA Amendments Act. While the USA FREEDOM Act doesn’t address these authorities either, we believe passing this legislation will serve as a better foundation for further reform than a sunset.
  3. Through grandfathering — Section 215’s sunset provision has a clause saying that investigations that were begun before the sunset occurred may continue. The FBI doesn’t put a time limit on its investigations, whatever broad investigation they’ve been using to justify bulk collection could foreseeably continue indefinitely. While the recent 2nd Circuit Court of Appeals opinion might mitigate this a bit, it’s unclear if the Foreign Intelligence Surveillance Court will apply that court’s reasoning to the government.

Privacy and security of users on the Internet is fundamental. Mozilla stands in continued support of the USA FREEDOM Act, as do hundreds of thousands of users, tech companies, privacy advocates, the Director of National Intelligence, the Attorney General, and the White House. Indeed, a bipartisan majority of both chambers of Congress support this legislation. We urge the House to stand behind their overwhelming vote to approve this legislation and to demand reforms that are at least as strong as the USA FREEDOM Act. Ultimately, this bill failed on a procedural vote on Friday night by just three votes. When the Senate reconvenes this Sunday, we hope three senators will step up to help enact the real reforms that users demand.

 

 

Categorieën: Mozilla-nl planet

Mike Conley: The Joy of Coding (Ep. 16): Wacky Morning DJ

do, 28/05/2015 - 04:12

I’m on vacation this week, but the show must go on! So I pre-recorded a shorter episode of The Joy of Coding last Friday.

In this episode1, I focused on a tool I wrote that I alluded to in the last episode, which is a soundboard to use during Joy of Coding episodes.

I demo the tool, and then I explain how it works. After I finished the episode, I pushed to repository to GitHub, and you can check that out right here.

So I’ll see you next week with a full length episode! Take care!

  1. Which, several times, I mistakenly refer to as the 15th episode, and not the 16th. Whoops. 

Categorieën: Mozilla-nl planet

Air Mozilla: Kids' Vision - Mentorship Series

do, 28/05/2015 - 00:00

Kids' Vision - Mentorship Series Mozilla hosts Kids Vision Bay Area Mentor Series

Categorieën: Mozilla-nl planet

Air Mozilla: Quality Team (QA) Public Meeting

wo, 27/05/2015 - 22:30

Quality Team (QA) Public Meeting This is the meeting where all the Mozilla quality teams meet, swap ideas, exchange notes on what is upcoming, and strategize around community building and...

Categorieën: Mozilla-nl planet

Benjamin Smedberg: Yak Shaving

wo, 27/05/2015 - 22:22

Yak shaving tends to be looked down on. I don’t necessarily see it that way. It can be a way to pay down technical debt, or learn a new skill. In many ways I consider it a sign of broad engineering skill if somebody is capable of solving a multi-part problem.

It started so innocently. My team has been working on unifying the Firefox Health Report and Telemetry data collection systems, and there was a bug that I thought I could knock off pretty easily: “FHR data migration: org.mozilla.crashes”. Below are the roadblocks, mishaps, and sideshows that resulted, and I’m not even done yet:

Tryserver failure: crashes

Constant crashes only on Linux opt builds. It turned out this was entirely my fault. The following is not a safe access pattern because of c++ temporary lifetimes:

nsCSubstringTuple str = str1 + str2; Fn(str);
Backout #1: talos xperf failure

After landing, the code was backed out because the xperf Talos test detected main-thread I/O. On desktop, this was a simple ordering problem: we always do that I/O during startup to initialize the crypto system; I just moved it slightly earlier in the startup sequence. Why are we initializing the crypto system? To generate a random number. Fixed this by whitelisting the I/O. This involved landing code to the separate Talos repo and then telling the main Firefox tree to use the new revision. Much thanks to Aaron Klotz for helping me figure out the right steps.

Backout #2: test timeouts

Test timeouts if the first test of a test run uses the PopupNotifications API. This wasn’t caught during initial try runs because it appeared to be a well-known random orange. I was apparently changing the startup sequence just enough to tickle a focus bug in the test harness. It so happened that the particular test which runs first depends on the e10s or non-e10s configuration, leading to some confusion about what was going on. Fortunately, I was able to reproduce this locally. Gavin Sharp and Neil Deakin helped get the test harness in order in bug 1138079.

Local test failures on Linux

I discovered that several xpcshell tests were failing locally on Linux which were working fine on tryserver. After some debugging, I discovered that the tests thought I wasn’t using Linux, because the cargo-culted test for Linux was let isLinux = ("@mozilla.org/gnome-gconf-service;1" in Cc). This means that if gconf is disabled at build time or not present at runtime, the tests will fail. I installed GConf2-devel and rebuilt my tree and things were much better.

Incorrect failure case in the extension manager

While debugging the test failures, I discovered an incorrect codepath in GMPProvider.jsm for clients which are not Windows, Mac, or Linux (Android and the non-Linux Unixes).

Android performance regression

The landing caused an Android startup performance regression, bug 1163049. On Android, we don’t initialize NSS during startup, and the earlier initialization of the addon manager caused us to generate random Sync IDs for addons. I first fixed this by using Math.random() instead of good crypto, but Richard Newman suggested that I just make Sync generation lazy. This appears to work and will land when there is an open tree.

mach bootstrap on Fedora doesn’t work for Android

As part of debugging the performance regression, I built Firefox for Android for the first time in several years. I discovered that mach bootstrap for Android isn’t implemented on Fedora Core. I manually installed packages until it built properly. I have a list of the packages I installed and I’ll file a bug to fix mach bootstrap when I get a chance.

android build-tools not found

A configure check for the android build-tools package failed. I still don’t understand exactly why this happened; it has something to do with a version that’s too new and unexpected. Nick Alexander pointed me at a patch on bug 1162000 which fixed this for me locally, but it’s not the “right” fix and so it’s not checked into the tree.

Debugging on Android (jimdb)

Binary debugging on Android turned out to be difficult. There are some great docs on the wiki, but those docs failed to mention that you have to pass the configure flag –enable-debug-symbols. After that, I discovered that pending breakpoints don’t work by default with Android debugging, and since I was debugging a startup issue that was a critical failure. I wrote an ask.mozilla.org question and got a custom patch which finally made debugging work. I also had to patch the implementation of DumpJSStack() so that it would print to logcat on Android; this is another bug that I’ll file later when I clean up my tree.

Crash reporting broken on Mac

I broke crash report submission on mac for some users. Crash report annotations were being truncated from unicode instead of converted from UTF8. Because JSON.stringify doesn’t produce ASCII, this was breaking crash reporting when we tried to parse the resulting data. This was an API bug that existed prior to the patch, but I should have caught it earlier. Shoutout to Ted Mielczarek for fixing this and adding automated tests!

Semi-related weirdness: improving the startup performance of Pocket

The Firefox Pocket integration caused a significant startup performance issue on some trees. The details are especially gnarly, but it seems that by reordering the initialization of the addon manager, I was able to turn a performance regression into a win by accident. Probably something to do with I/O wait, but it still feels like spooky action at a distance. Kudos to Joel Maher, Jared Wein and Gijs Kruitbosch for diving into this under time pressure.

Experiences like this are frustrating, but as long as it’s possible to keep the final goal in sight, fixing unrelated bugs along the way might be the best thing for everyone involved. It will certainly save context-switches from other experts to help out. And doing the Android build and debugging was a useful learning experience.

Perhaps, though, I’ll go back to my primary job of being a manager.

Categorieën: Mozilla-nl planet

Air Mozilla: Security Services / MDN Update

wo, 27/05/2015 - 21:00

Security Services / MDN Update An update on the roadmap and plan for Minion and some new MDN projects coming up in Q3 and the rest of 2015

Categorieën: Mozilla-nl planet

Air Mozilla: Product Coordination Meeting

wo, 27/05/2015 - 20:00

Product Coordination Meeting Duration: 10 minutes This is a weekly status meeting, every Wednesday, that helps coordinate the shipping of our products (across 4 release channels) in order...

Categorieën: Mozilla-nl planet

Armen Zambrano: Welcome adusca!

wo, 27/05/2015 - 19:10
It is my privilege to announce that adusca (blog) joined Mozilla (since Monday) as an Outreachy intern for the next 4 months.

adusca has an outstanding number of contributions over the last few months including Mozilla CI Tools (which we're working on together).

Here's a bit about herself from her blog:
Hi! I’m Alice. I studied Mathematics in college. I was doing a Master’s degree in Mathematical Economics before getting serious about programming.She is also a graduate from Hacker's School.

Even though Alice has not been a programmer for many years, she has shown already lots of potential. For instance, she wrote a script to generate scheduling relations for buildbot; for this and many other reasons I tip my hat.

adusca will initially help me out with creating a generic pulse listener to handle job cancellations and retriggers for Treeheder. The intent is to create a way for Mozilla CI tools to manage scheduling on behalf of TH, make the way for more sophisticated Mozilla CI actions and allow other people to piggy back to this pulse service and trigger their own actions.

If you have not yet had a chance to welcome her and getting to know her, I highly encourage you to do so.

Welcome Alice!

Creative Commons License
This work by Zambrano Gasparnian, Armen is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.
Categorieën: Mozilla-nl planet

Mark Surman: Mozilla Academy Strategy Update

wo, 27/05/2015 - 18:13

One of MoFo’s main goals for 2015 is to come up with an ambitious learning and community strategy. The codename for this is ‘Mozilla Academy’. As a way to get the process rolling, I wrote a long post in March outlining what we might include in that strategy. Since then, I’ve been putting together a team to dig into the strategy work formally.

This post is an update on that process in FAQ form. More substance and meat is coming in future posts. Also, there is lots of info on the wiki.

Q1. What are we trying to do?

Our main goal is alignment: to get everyone working on Mozilla’s learning and leadership development programs pointed in the same direction. The three main places we need to align are:

  1. Purpose: help people learn and hone the ability to read | write | participate.
  2. Process: people learn and improve by making things (in a community of like-minded peers).
  3. Poetry: tie back to ‘web = public resource’ narrative. Strong Mozilla brand.

At the end of the year, we will have a unified strategy that connects Mozilla’s learning and leadership development offerings (Webmaker, Hive, Open News, etc.). Right now, we do good work in these areas, but they’re a bit fragmented. We need to fix that by creating a coherent story and common approaches that will increase the impact these programs can have on the world.

Q2. What is ‘Mozilla Academy’?

That’s what we’re trying to figure out. At the very least, Mozilla Academy will be a clearly packaged and branded harmonization of Mozilla’s learning and leadership programs. People will be able to clearly understand what we’re doing and which parts are for them. Mozilla Academy may also include a common set of web literacy skills, curriculum format and learning approaches that we use across programs. We are also reviewing the possibility of a shared set of credentials or roles for people participating in Mozilla Academy.

Q3. Who is ‘Mozilla Academy’ for?

Over the past few weeks, we’ve started to look at who we’re trying to serve with our existing programs (blog post on this soon). Using the ‘scale vs depth’ graph in the Mozilla Learning plan as a framework, we see three main audiences:

  • 1.4 billion Facebook users. Or, whatever metric you use to count active people on the internet. We can reach some percentage of these people with software or marketing that invite people to ‘read | write | participate’. We probably won’t get them to want to ‘learn’ in an explicit way. They will learn by doing. Which is fine. Webmaker and SmartOn currently focus on this group.
  • People who actively want to grow their web literacy and skills. These are people interested enough in skills or technology or Mozilla that they will choose to participate in an explicit learning activity. They include everyone from young people in afterschool programs to web developers who might be interested in taking a course with Mozilla. Mozilla Clubs, Hive and MDN’s nascent learning program currently focus on this group.
  • People who want to hone their skills *and* have an impact on the world. These are people who already understand the web and technology at some level, but want to get better. They are also interested in doing something good for the web, the world or both. They include everyone from an educator wanting to create digital curriculum to a developer who wants to make the world of news or science better. Hive, ReMo and our community-based fellowships currently serve these people.

A big part of the strategy process is getting clear on these audiences. From there we can start to ask questions like: who can Mozilla best serve?; where can we have the most impact?; can people in one group serve or support people in another? Once we have the answers to these questions we can decide where to place our biggest bets (we need to do this!). And we can start raising more money to support our ambitious plans.

Q4. What is a ‘strategy’ useful for?

We want to accomplish a few things as a result of this process. A. A way to clearly communicate the ‘what and why’ of Mozilla’s learning and leadership efforts. B. A framework for designing new programs, adjusting program designs and fundraising for program growth. C. Common approaches and platforms we can use across programs. These things are important if we want Mozilla to stay in learning and leadership for the long haul, which we do.

Q5. What do you mean by ‘common approaches’?

There are a number of places where we do similar work in different ways. For example, Mozilla Clubs, Hive, Mozilla Developer Network, Open News and Mozilla Science Lab are all working on curriculum but do not yet have a shared curriculum model or repository. Similarly, Mozilla runs four fellowship programs but does not have a shared definition of a ‘Mozilla Fellow’. Common approaches could help here.

Q6. Are you developing a new program for Mozilla?

That’s not our goal. We like most of the work we’re doing now. As outlined in the 2015 Mozilla Learning Plan, our aim is to keep building on the strongest elements of our work and then connect these elements where it makes sense. We may modify, add or cut program elements in the future, but that’s not our main focus.

Q7. Are you set on the ‘Mozilla Academy’ name?

It’s pretty unlikely that we will use that name. Many people hate it. However, we needed a moniker to use during the strategy process. For better or for worse, that’s the one we chose.

Q8. What’s the timing for all of this?

We will have a basic alignment framework around ‘purpose, process and poetry’ by the end of June. We’ll work with the team at the Mozilla All Hands in Whistler. We will develop specific program designs, engage in a  broad conversation and run experiments. By October, we will have an updated version of the Mozilla Learning plan, which will lay out our work for 2016+.

As indicated above, the aim of this post is to give a process update. There is much more info on the process, who’s running it and what all the pieces are in the Mozilla Learning strategy wiki FAQ. The wiki also has info on how to get involved. If you have additional questions, ask them here. I’ll respond to the comments and also add my answers to the wiki.

In terms of substance, I’m planning a number of posts in coming weeks on topics like the essence of web literacy, who our audiences are and how we think about learning. People leading Mozilla Academy working groups will also be posting on substantive topics like our evolving thinking around the web literacy map and fellows programs. And, of course, the wiki will be growing with substantive strategy documents covering many of the topics above.


Filed under: education, mozilla, webmakers
Categorieën: Mozilla-nl planet

Manish Goregaokar: Wrapper Types in Rust: Choosing Your Guarantees

wo, 27/05/2015 - 16:59

In my previous post I talked a bit about why the RWlock pattern is important for accessing data, which is why Rust enforces this pattern either at compile time or runtime depending on the abstractions used.

It occurred to me that there are many such abstractions in Rust, each with their unique guarantees. The programmer once again has the choice between runtime and compile time enforcement. It occurred to me that this plethora of “wrapper types”1 could be daunting to newcomers; in this post I intend to give a thorough explanation of what some prominent ones do and when they should be used.

I’m assuming the reader knows about ownership and borrowing in Rust. Nevertheless, I will attempt to keep the majority of this post accessible to those not yet familiar with these concepts. Aside from the two links into the book above, these two blog posts cover the topic in depth.

Basic pointer types Box<T>

Box<T> is an “owned pointer”. While it can hand out borrowed references to the data, it is the only owner of the data. In particular, when something like the following occurs:

let x = Box::new(1); let y = x; // x no longer accessible here

Here, the box was moved into y. As x no longer owns it, the compiler will no longer allow the programmer to use x after this. A box can similarly be moved out of a function by returning, and when a box (one which hasn’t been moved) goes out of scope, destructors are run, deallocating the inner data.

This abstraction is a low cost abstraction for dynamic allocation. If you want to allocate some memory on the heap and safely pass a pointer to that memory around, this is ideal. Note that you will only be allowed to share borrowed references to this by the regular borrowing rules, checked at compile time.

Interlude: Copy

Move semantics are not special to Box<T>; it is a feature of all types which are not Copy.

A Copy type is one where all the data it logically encompasses (usually, owns) is part of its stack representation2. Most types containing pointers to other data are not Copy, since there is additional data elsewhere, and simply copying the stack representation may accidentally share ownership of that data in an unsafe manner.

Types like Vec<T> and String which also have data on the heap are also not Copy. Types like the integer/boolean types are Copy

&T and raw pointers are Copy. Even though they do point to further data, they do not “own” that data. Whereas Box<T> can be thought of as “some data which happens to be dynamically allocated”, &T is thought of as “a borrowing reference to some data”. Even though both are pointers, only the first is considered to be “data”. Hence, a copy of the first should involve a copy of the data (which is not part of its stack representation), but a copy of the second only needs a copy of the reference. &mut T is not Copy because mutable aliases cannot be shared, and &mut T “owns” the data it points to somewhat since it can mutate.

Practically speaking, a type can be Copy if a copy of its stack representation doesn’t violate memory safety.

&T and &mut T

These are immutable and mutable references respectively. They follow the “read-write lock” pattern described in my previous post, such that one may either have only one mutable reference to some data, or any number of immutable ones, but not both. This guarantee is enforced at compile time, and has no visible cost at runtime. In most cases such pointers suffice for sharing cheap references between sections of code.

These pointers cannot be copied in such a way that they outlive the lifetime associated with them.

*const T and *mut T

These are C-like raw pointers with no lifetime or ownership attached to them. They just point to some location in memory with no other restrictions. The only guarantee that these provide is that they cannot be dereferenced except in code marked unsafe.

These are useful when building safe, low cost abstractions like Vec<T>, but should be avoided in safe code.

Rc<T>

This is the first wrapper we will cover that has a runtime cost.

Rc<T> is a reference counted pointer. In other words, this lets us have multiple “owning” pointers to the same data, and the data will be freed (destructors will be run) when all pointers are out of scope.

Internally, it contains a shared “reference count”, which is incremented each time the Rc is cloned, and decremented each time one of the Rcs goes out of scope. The main responsibility of Rc<T> is to ensure that destructors are called for shared data.

The internal data here is immutable, and if a cycle of references is created, the data will be leaked. If we want data that doesn’t leak when there are cycles, we need a garbage collector. I do not know of any existing GCs in Rust, but I am working on one with Michael Layzell and there’s another cycle collecting one being written by Nick Fitzgerald.

Guarantees

The main guarantee provided here is that the data will not be destroyed until all references to it are out of scope.

This should be used when you wish to dynamically allocate and share some data (read-only) between various portions of your program, where it is not certain which portion will finish using the pointer last. It’s a viable alternative to &T when &T is either impossible to statically check for correctness, or creates extremely unergonomic code where the programmer does not wish to spend the development cost of working with.

This pointer is not thread safe, and Rust will not let it be sent or shared with other threads. This lets one avoid the cost of atomics in situations where they are unnecessary.

There is a sister smart pointer to this one, Weak<T>. This is a non-owning, but also non-borrowed, smart pointer. It is also similar to &T, but it is not restricted in lifetime — a Weak<T> can be held on to forever. However, it is possible that an attempt to access the inner data may fail and return None, since this can outlive the owned Rcs. This is useful for when one wants cyclic data structures and other things.

Cost

As far as memory goes, Rc<T> is a single allocation, though it will allocate two extra words as compared to a regular Box<T> (for “strong” and “weak” refcounts).

Rc<T> has the computational cost of incrementing/decrementing the refcount whenever it is cloned or goes out of scope respectively. Note that a clone will not do a deep copy, rather it will simply increment the inner reference count and return a copy of the Rc<T>

Cell types

“Cells” provide interior mutability. In other words, they contain data which can be manipulated even if the type cannot be obtained in a mutable form (for example, when it is behind an &-ptr or Rc<T>).

The documentation for the cell module has a pretty good explanation for these.

These types are generally found in struct fields, but they may be found elsewhere too.

Cell<T>

Cell<T> is a type that provides zero-cost interior mutability, but only for Copy types. Since the compiler knows that all the data owned by the contained value is on the stack, there’s no worry of leaking any data behind references (or worse!) by simply replacing the data.

It is still possible to violate your own invariants using this wrapper, so be careful when using it. If a field is wrapped in Cell, it’s a nice indicator that the chunk of data is mutable and may not stay the same between the time you first read it and when you intend to use it.

let x = Cell::new(1); let y = &x; let z = &x; x.set(2); y.set(3); z.set(4); println!("{}", x.get());

Note that here we were able to mutate the same value from various immutable references.

This has the same runtime cost as the following:

let mut x = 1; let y = &mut x; let z = &mut x; x = 2; *y = 3; *z = 4; println!("{}", x;

but it has the added benefit of actually compiling successfully.

Guarantees

This relaxes the “no aliasing with mutability” restriction in places where it’s unnecessary. However, this also relaxes the guarantees that the restriction provides; so if one’s invariants depend on data stored within Cell, one should be careful.

This is useful for mutating primitives and other Copy types when there is no easy way of doing it in line with the static rules of & and &mut.

Gábor Lehel summed up the guarantees provided by Cell in a rather succinct manner:

The basic guarantee we need to ensure is that interior references can’t be invalidated (left dangling) by mutation of the outer structure. (Think about references to the interiors of types like Option, Box, Vec, etc.) &, &mut, and Cell each make a different tradeoff here. & allows shared interior references but forbids mutation; &mut allows mutation xor interior references but not sharing; Cell allows shared mutability but not interior references.

[because Cell operates by copying, it can’t create interior references]

This comment by Eddy also touches on the guarantees of Cell and the alternatives

Cost

There is no runtime cost to using Cell<T>, however if one is using it to wrap larger (Copy) structs, it might be worthwhile to instead wrap individual fields in Cell<T> since each write is a full copy of the struct.

RefCell<T>

RefCell<T> also provides interior mutability, but isn’t restricted to Copy types.

Instead, it has a runtime cost. RefCell<T> enforces the RWLock pattern at runtime (it’s like a single-threaded mutex), unlike &T/&mut T which do so at compile time. This is done by the borrow() and borrow_mut() functions, which modify an internal reference count and return smart pointers which can be dereferenced immutably and mutably respectively. The refcount is restored when the smart pointers go out of scope. With this system, we can dynamically ensure that there are never any other borrows active when a mutable borrow is active. If the programmer attempts to make such a borrow, the thread will panic.

let x = RefCell::new(vec![1,2,3,4]); { println!("{:?}", *x.borrow()) } { let ref = x.borrow_mut(); ref.push(1); }

Similar to Cell, this is mainly useful for situations where it’s hard or impossible to satisfy the borrow checker. Generally one knows that such mutations won’t happen in a nested form, but it’s good to check.

For large, complicated programs, it becomes useful to put some things in RefCells to make things simpler. For example, a lot of the maps in the ctxt struct in the rust compiler internals are inside this wrapper. These are only modified once (during creation, which is not right after initialization) or a couple of times in well-separated places. However, since this struct is pervasively used everywhere, juggling mutable and immutable pointers would be hard (perhaps impossible) and probably form a soup of &-ptrs which would be hard to extend. On the other hand, the RefCell provides a cheap (not zero-cost) way of safely accessing these. In the future, if someone adds some code that attempts to modify the cell when it’s already borrowed, it will cause a (usually deterministic) panic which can be traced back to the offending borrow.

Similarly, in Servo’s DOM we have a lot of mutation, most of which is local to a DOM type, but some of which crisscrosses the DOM and modifies various things. Using RefCell and Cell to guard all mutation lets us avoid worrying about mutability everywhere, and it simultaneously highlights the places where mutation is actually happening.

Note that RefCell should be avoided if a mostly simple solution is possible with & pointers.

Guarantees

RefCell relaxes the static restrictions preventing aliased mutation, and replaces them with dynamic ones. As such the guarantees have not changed.

Cost

RefCell does not allocate, but it contains an additional “borrow state” indicator (one word in size) along with the data.

At runtime each borrow causes a modification/check of the refcount.

Synchronous types

Many of the types above cannot be used in a threadsafe manner. Particularly, Rc<T> and RefCell<T>, which both use non-atomic ref counts, cannot be used this way. This makes them cheaper to use, but one needs thread safe versions of these too. They exist, in the form of Arc<T> and Mutex<T>/RWLock<T>

Note that the non-threadsafe types cannot be sent between threads, and this is checked at compile time. I’ll touch on how this is done in a later blog post.

There are many useful wrappers for concurrent programming in the sync module, but I’m only going to cover the major ones.

Arc<T>

Arc<T> is just a version of Rc<T> that uses an atomic reference count (hence, “Arc”). This can be sent freely between threads.

C++’s shared_ptr is similar to Arc, however in C++s case the inner data is always mutable. For semantics similar to that from C++, we should use Arc<Mutex<T>>, Arc<RwLock<T>>, or Arc<UnsafeCell<T>>3 (UnsafeCell<T> is a cell type that can be used to hold any data and has no runtime cost, but accessing it requires unsafe blocks). The last one should only be used if one is certain that the usage won’t cause any memory unsafety. Remember that writing to a struct is not an atomic operation, and many functions like vec.push() can reallocate internally and cause unsafe behavior (so even monotonicity may not be enough to justify UnsafeCell)

Guarantees

Like Rc, this provides the (thread safe) guarantee that the destructor for the internal data will be run when the last Arc goes out of scope (barring any cycles).

Cost

This has the added cost of using atomics for changing the refcount (which will happen whenever it is cloned or goes out of scope). When sharing data from an Arc in a single thread, it is preferable to share & pointers whenever possible.

Mutex<T> and RwLock<T>

Mutex<T> and RwLock<T> provide mutual-exclusion via RAII guards. For both of these, the mutex is opaque until one calls lock() on it, at which point the thread will block until a lock can be acquired, and then a guard will be returned. This guard can be used to access the inner data (mutably), and the lock will be released when the guard goes out of scope.

{ let guard = mutex.lock(); // guard dereferences mutably to the inner type *guard += 1; } // lock released when destructor runs

RwLock has the added benefit of being efficient for multiple reads. It is always safe to have multiple readers to shared data as long as there are no writers; and RwLock lets readers acquire a “read lock”. Such locks can be acquired concurrently and are kept track of via a reference count. Writers must obtain a “write lock” which can only be obtained when all readers have gone out of scope.

Guarantees

Both of these provide safe shared mutability across threads, however they are prone to deadlocks. Some level of additional protocol safety can be obtained via the type system. An example of this is rust-sessions, an experimental library which uses session types for protocol safety.

Costs

These use internal atomic-like types to maintain the locks, and these are similar pretty costly (they can block all memory reads across processors till they’re done). Waiting on these locks can also be slow when there’s a lot of concurrent access happening.

Composition

A common gripe when reading Rust code is with stuff like Rc<RefCell<Vec<T>>> and more complicated compositions of such types.

Usually, it’s a case of composing together the guarantees that one needs, without paying for stuff that is unnecessary.

For example, Rc<RefCell<T>> is one such composition. Rc itself can’t be dereferenced mutably; because Rc provides sharing and shared mutability isn’t good, so we put RefCell inside to get dynamically verified shared mutability. Now we have shared mutable data, but it’s shared in a way that there can only be one mutator (and no readers) or multiple readers.

Now, we can take this a step further, and have Rc<RefCell<Vec<T>>> or Rc<Vec<RefCell<T>>>. These are both shareable, mutable vectors, but they’re not the same.

With the former, the RefCell is wrapping the Vec, so the Vec in its entirety is mutable. At the same time, there can only be one mutable borrow of the whole Vec at a given time. This means that your code cannot simultaneously work on different elements of the vector from different Rc handles. However, we are able to push and pop from the Vec at will. This is similar to an &mut Vec<T> with the borrow checking done at runtime.

With the latter, the borrowing is of individual elements, but the overall vector is immutable. Thus, we can independently borrow separate elements, but we cannot push or pop from the vector. This is similar to an &mut [T]4, but, again, the borrow checking is at runtime.

In concurrent programs, we have a similar situation with Arc<Mutex<T>>, which provides shared mutability and ownership.

When reading code that uses these, go in step by step and look at the guarantees/costs provided.

When choosing a composed type, we must do the reverse; figure out which guarantees we want, and at which point of the composition we need them. For example, if there is a choice between Vec<RefCell<T>> and RefCell<Vec<T>>, we should figure out the tradeoffs as done above and pick one.

Discuss: HN, Reddit

  1. I’m not sure if this is the technical term for them, but I’ll be calling them that throughout this post.

  2. By “stack representation” I mean the data on the stack when a value of this type is held on the stack. For example, a Vec<T> has a stack representation of a pointer and two integers (length, capacity). While there is more data behind the indirection of the pointer, it is not part of the stack-held portion of the Vec. Looking at this a different way, a type is Copy if a memcopy of the data copies all the data owned by it.

  3. Arc<UnsafeCell<T>> actually won’t compile since UnsafeCell<T> isn’t Send or Sync, but we can wrap it in a type and implement Send/Sync for it manually to get Arc<Wrapper<T>> where Wrapper is struct Wrapper<T>(UnsafeCell<T>).

  4. &[T] and &mut [T] are slices; they consist of a pointer and a length and can refer to a portion of a vector or array. &mut [T] can have its elements mutated, however its length cannot be touched.

Categorieën: Mozilla-nl planet

Mozilla Addons Blog: Update on Extension Signing and New Developer Agreement

wo, 27/05/2015 - 16:10

If you have an active extension listing on AMO you probably got a message from us already, explaining how we will automatically sign your add-on and provide it to your users via automatic updates. The automatic signing process will run this week, in batches, and we will notify you when your add-on is signed. Please take some time to test the signed version in the current release version of Firefox and either Developer Edition or Nightly (where Firefox already warns about unsigned extensions).

If you’re unfamiliar with extension signing, please read the original announcement for context.

Next week, we will activate two new features on AMO: signing of new add-on versions after they are reviewed, and add-on submission for developers who wish to have their add-ons signed but don’t want them listed on AMO. We will post another update once this happens. When this is done, all extension developers will be able to have their extensions signed, with enough time to update their users before signing becomes a requirement in release versions of Firefox.

New Developer Agreement

Since we will be signing add-ons that won’t be listed on AMO, we have updated the Add-on Distribution Developer Agreement to cover the new ways in which we will handle add-ons. This document hadn’t been touched for years, so we took our time and significantly updated its contents to reflect how we do things now. Hopefully it is also easier to read than its previous version.

Note that the new agreement will go into effect on June 1st. The version that is displayed on AMO when you submit a new add-on will be updated then, and all active developers on AMO will be notified about it.

If you want to stay up to date with changes related to extension signing, you can follow this blog or check in regularly to the wiki page, where I update the timeline information as it becomes clearer.

Categorieën: Mozilla-nl planet

Air Mozilla: Bugzilla Development Meeting

wo, 27/05/2015 - 16:00

Bugzilla Development Meeting Help define, plan, design, and implement Bugzilla's future!

Categorieën: Mozilla-nl planet

Gervase Markham: The Jeeves Test

wo, 27/05/2015 - 14:56

What is a browser for? What should it do, or not do? What should it be?

People within the Mozilla project have been recently discussing the user value of some new features in Firefox. I think a person’s view of questions of this nature will depend on their view of the role of the browser. One option is the “featureless window on the web” view – the browser is nothing, the site is everything. But as one participant said, this leads to all the value-add and features being provided by the sites, which is not a recipe for user control, or for using the browser to advance the Mozilla mission.

I think the best vision for Firefox is as your “Internet butler” – quiet and refined, highly capable, provides what you need even before you know you need it, who gently guides you out of trouble but generally does his thing without you needing to think about him or provide explicit direction or management.

Bertie using an early voice interface prototype

So I’d like to propose the “Jeeves Test” for evaluating feature proposals for Firefox. It works like this: imagine Bertie Wooster, relaxing in an armchair in his apartment, with a cigarette, a gin and tonic, and a tablet computer. Then take the user value proposition of an idea, write it in appropriately deferential language, and see if you can imagine Jeeves whispering it into his ear. If you can’t, perhaps it’s not something we want to do.

To make that a bit more concrete, here are some examples of things that might pass: “Here’s an English translation of this Serbian page, sir”, or “For your safety, sir, access to this malware page has been blocked”. And here are some which might not: “For your convenience, sir, I’ve exempted aol.com from your popup blocker”, or “You’ll be pleased to find, sir, that the user interface has been substantially rearranged”.

There may be occasions where we’d want to do something which doesn’t obviously pass the Jeeves Test, if the effects on the broader web ecosystem of making the change are significantly positive. Some of the things we do to improve web security but which have a short-term compatibility impact might fall into that category. “Let me ensure this site doesn’t load for you, sir” generally doesn’t go down well, after all. But in those cases, that longer-term or broader value has to be clearly articulated – before we make the change – if we are to avoid an exasperated “Dash it, Jeeves… why?” from our userbase.

Categorieën: Mozilla-nl planet

Mozilla Release Management Team: Firefox 38.0.5 rc to rc2

wo, 27/05/2015 - 13:54

A small update for next week release. We took a patch to fix a JS top crash and 3 changes to fix an important regression in WebRTC.

  • 6 changesets
  • 10 files changed
  • 85 insertions
  • 35 deletions

ExtensionOccurrences cpp5 xml1 txt1 mn1 html1 h1

ModuleOccurrences js5 mobile1 media1 dom1 config1 browser1

List of changesets:

tbirdbldAutomated checkin: version bump for thunderbird 38.0b6 release. DONTBUILD CLOSED TREE a=release - 7eb2d1522ac2 Margaret LeibovicBug 1166392 - Include about:reader strings on Android. r=mfinkle, a=sledru - a468d1edeac8 Tooru FujisawaBug 1159973 - Abort parsing when TokenStream::SourceCoords hits OOM. r=jorendorff, a=sylvestre - 414d430012ee Paul AdenotBug 1166183 - Back out the direct listener removal landed by mistake in Bug 1141781. r=jesup, a=sledru - 9525d8723413 Paul AdenotBug 1166183 - Work around Bug 934512 in track_peerConnection_replaceTrack.html. r=pehrson, a=sledru - 6ee39f753d19 Andreas PehrsonBug 1166183 - Reset PipelineListener's flag after ReplaceTrack(). r=bwc, a=sledru - 19b9ffda6e2f

Categorieën: Mozilla-nl planet

Byron Jones: happy bmo push day!

wo, 27/05/2015 - 09:11

the following changes have been pushed to bugzilla.mozilla.org:

  • [1168357] Unable to reset mentor in modal bug view
  • [1146782] migrate autocomplete from yui to jquery
  • [1165355] sanitizeme.pl script should remove webservice api keys from database to remove sensitive information
  • [1158010] provide a standard and simple way to render relative dates, in perl and javascript
  • [1168303] send-request-nags.pl is triggering oom on bugzillaadm.private.scl3
  • [1162854] hitting back after making an edit should keep edit mode open
  • [1164850] add preferences to request nagging watching facility (reviews only, extended period, and skip encryption)
  • [1164780] updating an attachment with pre-existing review requests shouldn’t enforce the “last logged in to bugzilla” limit
  • [1162427] the “reset assignee to default” checkbox / functionality should be visible

discuss these changes on mozilla.tools.bmo.


Filed under: bmo, mozilla
Categorieën: Mozilla-nl planet

Marcia Knous: QA BuddyUp team meets again

wo, 27/05/2015 - 00:14
The QA BuddyUp Team met once again prior to the Mozilla Balkans meetup in Bucharest. Thanks to Softvision, we had a great workspace to use for the day, and we had a lot of good discussion about the next steps for the project as well as our approach to automation. Pictured below are the members of the team (from left to right): Ada, Madalina, Ioana, fredy, Christos and Bebe.  You can see the notes from our discussion here. The day we met there were some challenges with working on automation, but
Categorieën: Mozilla-nl planet

Daniel Stenberg: picturing curl’s future

di, 26/05/2015 - 22:43

development graph

There will be more stuff over time in the cURL project. Exactly which stuff and how long time it takes for everything, we don’t know. It depends largely on who works on what and how much time said persons can spend on implementing the stuff they work on…

I suspect we might be able to do things slightly faster over time, which is why the red arrow isn’t just a straight line.

I drew this little picture inspired from discussions with friends after a talk I did about curl and how development works in an open source project such as this. We know we will work on things that will improve the products but we don’t see exactly what very far in advance. I tweeted this picture a few days ago, and it turned out very popular.

Categorieën: Mozilla-nl planet

Doug Belshaw: Web Literacy: what happens beyond peak centralisation and software with shareholders?

di, 26/05/2015 - 21:22

There’s no TIDE podcast this week, so I thought I’d record a blog post today. Here’s the abstract:

We’re at peak centralisation of our data in online services, with data as the new oil. It’s a time of ‘frictionless sharing’, but also a time when we’re increasingly having decisions made on our behalf by algorithms. Education is now subject to a land grab by ‘software with shareholders’ looking to profit from collecting, mining, packaging, and selling learner data. This article explores some of the issues at stake, as well as pointing towards the seeds of a potential solution.

The Code Acts in Education blog I mention in the introduction to this piece can be found here and Ben Williamson is @BenPatrickWill on Twitter.

Comments (once you’ve listened!) much appreciated. I’ve still got time to re-work this… :)

(no audio? click here!)

References

Belshaw, D.A.J. (2014a). Software with shareholders (or, the menace of private public spaces). Doug Belshaw’s blog. 23 April 2014. http://dougbelshaw.com/blog/2014/04/23/software-with-shareholders.

Belshaw, D.A.J. (2014b). Curate or Be Curated: Why Our Information Environment is Crucial to a Flourishing Democracy, Civil Society. DMLcentral. 23 October 2014. http://dmlcentral.net/blog/doug-belshaw/curate-or-be-curated-why-our-information-environment-crucial-flourishing-democracy.

Dixon-Thayer, D. (2015). Mozilla View on Zero-Rating. Open Policy & Advocacy Blog. Mozilla. 5 May 2015. https://blog.mozilla.org/netpolicy/2015/05/05/mozilla-view-on-zero-rating.

Flew, T. (2008). New Media: An Introduction (3rd ed.). Melbourne: Oxford University Press.

Gillula, J. & Malcolm, J. (2015). Internet.org Is Not Neutral, Not Secure, and Not the Internet. Deeplinks Blog. Electronic Frontier Foundation. 18 May 2015. https://www.eff.org/deeplinks/2015/05/internetorg-not-neutral-not-secure-and-not-internet.

Kramer, A.D.I., Guillory, J.E., Hancock, J.T. (2014) Experimental evidence of massive-scale emotional contagion through social networks. Proceedings of the National Academy of Sciences of the United Sates of America. 111(24).

McNeal, G.S. (2014). Facebook Manipulated User News Feeds To Create Emotional Responses. Forbes. 28 June 2014. http://www.forbes.com/sites/gregorymcneal/2014/06/28/facebook-manipulated-user-news-feeds-to-create-emotional-contagion

Mozilla. (2015). Web Literacy Map v1.1. https://teach.mozilla.org/teach-like-mozilla/web-literacy

Thorp, J. (2012). Big Data Is Not the New Oil. Harvard Business Review. 30 November 2012. https://hbr.org/2012/11/data-humans-and-the-new-oil.

Image CC BY-NC Graham Chastney

Categorieën: Mozilla-nl planet

Air Mozilla: SpiderMonkey garbage collection update

di, 26/05/2015 - 20:59

SpiderMonkey garbage collection update An overview of the progress we've made in the last two years development on the SpiderMonkey GC implementing generational collection and starting work on compacting.

Categorieën: Mozilla-nl planet

Chris AtLee: RelEng 2015 (part 1)

di, 26/05/2015 - 20:51

Last week, I had the opportunity to attend RelEng 2015 - the 3rd International Workshop of Release Engineering. This was a fantastic conference, and I came away with lots of new ideas for things to try here at Mozilla.

I'd like to share some of my thoughts and notes I took about some of the sessions. As of yet, the speakers' slides aren't collected or linked to from the conference website. Hopefully they'll get them up soon! The program and abstracts are available here.

For your sake (and mine!) I've split up my notes into a few separate posts. This post covers the introduction and keynote.

tl;dr

"Continuous deployment" of web applications is basically a solved problem today. What remains is for organizations to adopt best practices. Mobile/desktop applications remain a challenge.

Cisco relies heavily on collecting and analyzing metrics to inform their approach to software development. Statistically speaking, quality is the best driver of customer satisfaction. There are many aspects to product quality, but new lines of code introduced per release gives a good predictor of how many new bugs will be introduced. It's always challenging to find enough resources to focus on software quality; being able to correlate quality to customer satisfaction (and therefore market share, $$$) is one technique for getting organizational support for shipping high quality software. Common release criteria such as bugs found during testing, or bug fix rate, are used to inform stakeholders as to the quality of the release.

Introductory Session

Bram Adams and Foutse Khomh kicked things off with an overview of "continuous deployment" over the last 5 years. Back in 2009 we were already talking about systems where pushing to version control would trigger tens of thousands of tests, and do canary deployments up to 50 times a day.

Today we see companies like Facebook demonstrating that continuous deployment of web applications is basically a solved problem. Many organizations are still trying to implement these techniques. Mobile [and desktop!] applications still present a challenge.

Keynote

Pete Rotella from Cisco discussed how he and his team measured and predicted release quality for various projects at Cisco. His team is quite focused on data and analytics.

Cisco has relatively long release cycles compared to what we at Mozilla are used to now. They release 2-3 times per year, with each release representing approximately 500kloc of new code. Their customers really like predictable release cycles, and also don't like releases that are too frequent. Many of their customers have their own testing / validation cycles for releases, and so are only willing to update for something they deem critical.

Pete described how he thought software projects had four degrees of freedom in which to operate, and how quality ends up being the one sacrificed most often in order to compensate for constraints in the others:

  • resources (people / money): It's generally hard to hire more people or find room in the budget to meet the increasing demands of customers. You also run into the mythical man month problem by trying to throw more people at a problem.

  • schedule (time): Having standard release cycles means organizations don't usually have a lot of room to push out the schedule so that features can be completed properly.

    I feel that at Mozilla, the rapid release cycle has helped us out to some extent here. The theory is that if your feature isn't ready for the current version, it can wait for the next release which is only 6 weeks behind. However, I do worry that we have too many features trying to get finished off in aurora or even in beta.

  • content (features): Another way to get more room to operate is to cut features. However, it's generally hard to cut content or features, because those are what customers are most interested in.

  • quality: Pete believes this is where most organizations steal resources for to make up for people/schedule/content constraints. It's a poor long-term play, and despite "quality is our top priority" being the Official Party Line, most organizations don't invest enough here. What's working against quality?

    • plethora of releases: lots of projects / products / special requests for releases. Attempts to reduce the # of releases have failed on most occasions.
    • monetization of quality is difficult. Pete suggests tying the cost of a poor quality release to this. How many customers will we lose with a buggy release?
    • having RelEng and QA embedded in Engineering teams is a problem; they should be independent organizations so that their recommendations can have more weight.
    • "control point exceptions" are common. e.g. VP overrides recommendations of QA / RelEng and ships the release.

Why should we focus on quality? Pete's metrics show that it's the strongest driver of customer satisfaction. Your product's customer satisfaction needs to be more than 4.3/5 to get more than marginal market share.

How can RelEng improve metrics?

  • simple dashboards
  • actionable metrics - people need to know how to move the needle
  • passive - use existing data. everybody's stretched thin, so requiring other teams to add more metadata for your metrics isn't going to work.
  • standardized quality metrics across the company
  • informing engineering teams about risk
  • correlation with customer experience.

Interestingly, handling the backlog of bugs has minimal impact on customer satisfaction. In addition, there's substantial risk introduced whenever bugs are fixed late in a release cycle. There's an exponential relationship between new lines of code added and # of defects introduced, and therefore customer satisfaction.

Another good indicator of customer satisfaction is the number of "Customer found defects" - i.e. the number of bugs found and reported by their customers vs. bugs found internally.

Pete's data shows that if they can find more than 80% of the bugs in a release prior to it being shipped, then the remaining bugs are very unlikely to impact customers. He uses lines of code added for previous releases, and historical bug counts per version to estimate number of bugs introduced in the current version given the new lines of code added. This 80% figure represents one of their "Release Criteria". If less than 80% of predicted bugs have been found, then the release is considered risky.

Another "Release Criteria" Pete discussed was the weekly rate of fixing bugs. Data shows that good quality releases have the weekly bug fix rate drop to 43% of the maximum rate at the end of the testing cycle. This data demonstrates that changes late in the cycle have a negative impact on software quality. You really want to be fixing fewer and fewer bugs as you get closer to release.

I really enjoyed Pete's talk! There are definitely a lot of things to think about, and how we might apply them at Mozilla.

Categorieën: Mozilla-nl planet

Pagina's