mozilla

Mozilla Nederland LogoDe Nederlandse
Mozilla-gemeenschap

The Firefox Frontier: [Watch] Where Do You Draw The Line for TMI Online?

Mozilla planet - wo, 02/08/2017 - 20:21

As part of our tireless crusade to raise awareness around the issue of personal privacy on the web and to advocate for a free and open internet, we recently teamed … Read more

The post [Watch] Where Do You Draw The Line for TMI Online? appeared first on The Firefox Frontier.

Categorieën: Mozilla-nl planet

Air Mozilla: The Joy of Coding - Episode 108

Mozilla planet - wo, 02/08/2017 - 19:00

The Joy of Coding - Episode 108 mconley livehacks on real Firefox bugs while thinking aloud.

Categorieën: Mozilla-nl planet

Air Mozilla: Weekly SUMO Community Meeting August 2, 2017

Mozilla planet - wo, 02/08/2017 - 18:00

Weekly SUMO Community Meeting August 2, 2017 This is the sumo weekly call

Categorieën: Mozilla-nl planet

Botond Ballo: Trip Report: C++ Standards Meeting in Toronto, July 2017

Mozilla planet - wo, 02/08/2017 - 16:00
Summary / TL;DR

Project What’s in it? Status C++17 See below Draft International Standard published; on track for final publication by end of 2017 Filesystems TS Standard filesystem interface Part of C++17 Library Fundamentals TS v1 optional, any, string_view and more Part of C++17 Library Fundamentals TS v2 source code information capture and various utilities Published! Concepts TS Constrained templates Merged into C++20 with some modifications Parallelism TS v1 Parallel versions of STL algorithms Part of C++17 Parallelism TS v2 Task blocks, library vector types and algorithms and more Under active development Transactional Memory TS Transaction support Published! Uncertain whether this is headed towards C++20 Concurrency TS v1 future.then(), latches and barriers, atomic smart pointers Published! Parts of it headed for C++20 Concurrency TS v2 See below Under active development Networking TS Sockets library based on Boost.ASIO Voted for publication! Ranges TS Range-based algorithms and views Voted for publication! Coroutines TS Resumable functions, based on Microsoft’s await design Voted for publication! Modules TS A component system to supersede the textual header file inclusion model Preliminary Draft voted out for balloting by national standards bodies Numerics TS Various numerical facilities Under active development Graphics TS 2D drawing API Under active design review Reflection Code introspection and (later) reification mechanisms Introspection proposal passed core language and library design review; next stop is wording review. Targeting a Reflection TS. Contracts Preconditions, postconditions, and assertions Proposal passed core language and library design review; next stop is wording review.

Some of the links in this blog post may not resolve until the committee’s post-meeting mailing is published. If you encounter such a link, please check back in a few days.

Introduction

A couple of weeks ago I attended a meeting of the ISO C++ Standards Committee (also known as WG21) in Toronto, Canada (which, incidentally, is where I’m based). This was the second committee meeting in 2017; you can find my reports on previous meetings here (November 2016, Issaquah) and here (February 2017, Kona). These reports, particularly the Kona one, provide useful context for this post.

With the C++17 Draft International Standard (DIS) being published (and its balloting by national standards bodies currently in progress), this meeting was focused on C++20, and the various Technical Specifications (TS) we have in flight.

What’s the status of C++17?

From a technical point of view, C++17 is effectively done.

Procedurally, the DIS ballot is still in progress, and will close in August. Assuming it’s successful (which is widely expected), we will be in a position to vote to publish the final standard, whose content would be the same as the DIS with possible editorial changes, at the next meeting in November. (In the unlikely event that the DIS ballot is unsuccessful, we would instead publish a revised document labelled “FDIS” (Final Draft International Standard) at the November meeting, which would need to go through one final round of balloting prior to publication. In this case the final publication would likely happen in calendar year 2018, but I think the term “C++17” is sufficiently entrenched by now that it would remain the colloquial name for the standard nonetheless.)

C++20

With C++17 at the DIS stage, C++20 now has a working draft and is “open for business”; to use a development analogy, C++17 has “branched”, and the standard’s “trunk” is open for new development. Indeed, several changes have been voted into the C++20 working draft at this meeting.

Technical Specifications

This meeting was a particularly productive one for in-progress Technical Specifications. In addition to Concepts (which had already been published previously) being merged into C++20, three TSes – Coroutines, Ranges, and Networking – passed a publication vote this week, and a fourth, Modules, was sent out for its PDTS ballot (a ballot process that allows national standards bodies to vote and comment on the proposed TS, allowing the committee to incorporate their feedback prior to sending out a revised document for publication).

Coroutines TS

The Coroutines TS – which contains a stackless coroutine design, sometimes called co_await after one of the keywords it uses – had just been sent out for its PDTS ballot at the previous meeting. The results were in before this meeting began – the ballot had passed, with some comments. The committee made it a priority to get through all the comments at this meeting and draft any resulting revisions, so that the revised TS could be voted for final publication, which happened (successfully) during the closing plenary session.

Meanwhile, an independent proposal for stackful coroutines with a library-only interface is making its way through the Library groups. Attempts to unify the two varieties of coroutines into a single design seem to have been abandoned for now; the respective proposal authors maintain that the two kinds of coroutines are useful for different purposes, and could reasonably co-exist (no pun intended) in the language.

Ranges TS

The Ranges TS was sent out for its PDTS ballot two meetings ago, but due to the focus on C++17, the committee didn’t get through all of the resulting comments at the last meeting. That work was finished at this meeting, and this revised TS also successfully passed a publication vote.

Networking TS

Like the Ranges TS, the Networking TS was also sent out for its PDTS ballot two meetings ago, and resolving the ballot comments was completed at this meeting, leading to another successful publication vote.

Modules TS

Modules had come close to being sent out for its PDTS ballot at the previous meeting, but didn’t quite make it due to some procedural mis-communication (detailed in my previous report if you’re interested).

Modules is kind of in an interesting state. There are two relatively mature implementations (in MSVC and Clang), whose development either preceded or was concurrent with the development of the specification. Given this state of affairs, I’ve seen the following dynamic play out several times over the past few meetings:

  • a prospective user, or someone working on a new implementation (such as the one in GCC), comes to the committee seeking clarification about what happens in a particular scenario (like this one)
  • the two existing implementers consult their respective implementations, and give different answers
  • the implementers trace the difference in outcome back to a difference in the conceptual model of Modules that they have in their mind
  • the difference in the conceptual model, once identified, is discussed and reconciled by the committee, typically in the Evolution Working Group (EWG)
  • the implementers work with the Core Working Group (CWG) to ensure the specification wording reflects the new shared understanding

Of course, this is a desirable outcome – identifying and reconciling differences like this, and arriving at a specification that’s precise enough that someone can write a new implementation based purely on the spec, is precisely what we want out of a standards process. However, I can’t help but wonder if there isn’t a more efficient way to identify these differences – for example, by the two implementers actually studying each other’s implementations (I realize that’s complicated by the fact that one is proprietary…), or at least discussing their respective implementation strategies in depth.

That said, Modules did make good progress at this meeting. EWG looked at several proposal changes to the spec (I summarize the technical discussion below), CWG worked diligently to polish the spec wording further, and in the end, we achieved consensus to – finally – send out Modules for its PDTS ballot!

Parallelism TS v2

The Parallelism TS v2 (working draft here) picked up a new feature, vector and wavefront policies. Other proposals targeting it, like vector types and algorithms, are continuing to work their way through the library groups.

Concurrency TS v2

SG 1, the Study Group that deals with concurrency and parallelism, reviewed several proposals targeting the Concurrency TS v2 (which still does not yet have a working draft) at this meeting, including a variant of joining_thread with cooperative cancellation, lock-free programming techniques for reclamation, and stackful coroutines (which I’ve mentioned above in connection with the Coroutines TS).

Executors are still likely slated for a separate TS. The unified proposal presented at the last meeting has been simplified as requested, to narrow its scope to something manageable for initial standardization.

Merging Technical Specifications Into C++20

We have some technical specifications that are already published and haven’t been merged into C++17, and are thus candidates for merging into C++20. I already mentioned that Concepts was merged with some modifications (details below).

Parts of the Concurrency TS are slated to be merged into C++20: latches, with barriers to hopefully follow after some design issues are ironed out, and an improved version of atomic shared pointers. future.then() is going to require some more iteration before final standardization.

The Transactional Memory TS currently has only one implementation; the Study Group that worked on it hopes for some more implementation and usage experience prior to standardization.

The Library Fundamentals TS v2 seems to be in good shape to be merged into C++20, though I’m not sure of the exact status / concrete plans.

In addition to the TSes that are already published, many people are eager to see the TSes that were just published (Coroutines, Ranges, and Networking), as well as Modules, make it into C++20 too. I think it’s too early to try and predict whether they will make it. From a procedural point of view, there is enough time for all of these to complete their publication process and be merged in the C++20 timeframe. However, it will really depend on how much implementation and use experience these features get between now and the C++20 feature-complete date (sometime in 2019), and what the feedback from that experience is.

Future Technical Specifications

Finally, I’ll give some mention to some planned future Technical Specifications that don’t have an official project or working draft yet:

Reflection

A proposal for static introspection (sometimes called “reflexpr” after the keyword it uses; see its summary, design, and specification for details) continues to head towards a Reflection TS. It has been approved by SG 7 (the Reflection and Metaprogramming Study Group) and the Evolution Working Group at previous meetings. This week, it was successfully reviewed by the Library Evolution Working Group, allowing it to move on to the Core and Library groups going forward.

Meanwhile, SG 7 is continuing to look at more forward-looking reflection and metaprogramming topics, such as a longer-term vision for metaprogramming, and a proposal for metaclasses (I talk more about these below).

Graphics

The Graphics TS, which proposes to standardize a set of 2D graphics primitives inspired by cairo, continues to be under active review by the Library Evolution Working Group; the latest draft spec can be found here. The proposal is close to being forwarded to the Library Working Group, but isn’t quite there yet.

While I wasn’t present for its discussion in LEWG, I’m told that one of the changes that have been requested is to give the library a stateless interface. This matches the feedback I’ve heard from Mozilla engineers knowledgeable about graphics (and which I’ve tried to relay, albeit unsuccessfully, at a previous meeting).

Evolution Working Group

I’ll now write in a bit more detail about the technical discussions that took place in the Evolution Working Group, the subgroup that I sat in for the duration of the week.

All proposals discussed in EWG at this meeting were targeting C++20 (except for Modules, where we discussed changes targeting the Modules TS). I’ve categorized them into the usual “accepted”, “further work encouraged”, and “rejected” categories:

Accepted proposals:

  • Default member initializers for bitfields. Previously, bit-fields couldn’t have default member initializers; now they can, with the “natural” syntax, int x : 5 = 42; (brace initialization is also allowed). A disambiguation rule was added to deal with parsing ambiguities (since e.g. an = could conceivably be part of the bitfield width expression).
  • Tweaking the rules for constructor template argument deduction. At the last meeting, EWG decided that for wrapper types like tuple, copying should be preferable to wrapping; that is, something like tuple t{tuple{1, 2}}; should deduce the type of t as tuple<int, int> rather than tuple<tuple<int, int>>. However, it had been unclear whether this guidance applied to types like vector that had std::initializer_list constructors. EWG clarified that copying should indeed be preferred to wrapping for those types, too. (The paper also proposed several other tweaks to the rules, which did not gain consensus to be approved just yet; the author will come back with a revised paper for those.)
  • Resolving a language defect related to defaulted copy constructors. This was actually a proposal that I co-authored, and it was prompted by me running into this language defect in Mozilla code (it prevented the storing of an nsAutoPtr inside a std::tuple). It’s also, to date, my first proposal to be approved by EWG!
  • A simpler solution to the problem that allowing the template keyword in unqualified-ids aimed to solve. While reviewing that proposal, the Core Working Group found that the relevant lookup rules could be tweaked so as to avoid having to use the template keyword at all. The proposed rules technically change the meaning of certain existing code patterns, but only ones that are very obscure and unlikely to occur in the wild. EWG was, naturally, delighted with this simplification.
  • An attribute to mark unreachable code. This proposal aims to standardize existing practice where a point in the code that the author expects cannot be reached is marked with __builtin_unreachable() or __assume(false). The initial proposal was to make the standardized version an [[unreachable]] attribute, but based on EWG’s feedback, this was revised to be a std::unreachable() library function instead. The semantics is that if such a call is reached during execution, the behaviour is undefined. (EWG discussed at length whether this facility should be tied to the Contracts proposal. The outcome was that it should not be; since “undefined behaviour” encompasses everything, we can later change the specified behaviour to be something like “call the contract violation handler” without that being a breaking change.) The proposal was sent to LEWG, which will design the library interface more precisely, and consider the possibility of passing in a compile-time string argument for diagnostic purposes.
  • Down with typename! This paper argued that in some contexts where typename is currently required to disambiguate a name nested in a dependent scope as being a type, the compiler can actually disambiguate based on the context, and proposed removing the requirement of writing typename in such contexts. The proposal passed with flying colours. (It was, however, pointed out that the proposal prevents certain hypothetical future extensions. For example, one of the contexts in question is y in using x = y;: that can currently only be a type. However, suppose we later want to add expression aliases to C++; this proposal rules out re-using the using x = y; syntax for them.)
  • Removing throw(). Dynamic exception specifications have been deprecated since C++11 (superseded by noexcept), and removed altogether in C++17, with the exception of throw() as an alias for noexcept(true). This paper proposed removing that last vestige, too, and EWG approved it. (The paper also proposed removing some other things that were deprecated in C++17, which were rejected; I mention those in the list of rejected proposals below.)
  • Ranged-based for statement with initializer. This introduces a new form of range-for: for (T var = init; U elem : <range-expression>); here, var is a variable that lives for the duration of the loop, and can be referenced by <range-expression> (whereas elem is the usual loop variable that takes on a new value on every iteration). This is useful for both scope hygiene (it avoids polluting the enclosing scope with var) and resolving a category of lifetime issues with range-for. EWG expressed concerns about parseability (parsers will now need to perform more lookahead to determine which form of loop they are parsing) and readability (the “semicolon colon” punctuation in a loop header of this form can look deceptively like the “semicolon semicolon” punctuation in a traditional for loop), but passed the proposal anyways.
  • Some changes to the Modules TS (other proposed changes were deferred to Modules v2) – I talk about these below
  • Changes to Concepts – see below

Proposals for which further work is encouraged:

  • Non-throwing container operations. This paper acknowledges the reality that many C++ projects are unable to or choose not to use exceptions, and proposes that standard library container types which currently rely on exceptions to report memory allocation failure, provide an alternative API that doesn’t use exceptions. Several concrete alternatives are mentioned. EWG sent this proposal to the Library Evolution Working Group to design a concrete alternative API, with the understanding that the resulting proposal will come back to EWG for review as well.
  • Efficient sized deletion for variable-sized classes. This proposal builds on the sized deletion feature added to C++14 to enable this optimization for “variable-sized classes” (that is, classes that take control of their own allocation and allocate a variable amount of extra space before or after the object itself to store extra information). EWG found the use cases motivating, but encouraged the development of a syntax that less prone to accidental misuse, as well as additional consultation with implementers to ensure that ABI is not broken.
  • Return type deduction and SFINAE. This paper proposes a special syntax for single-expression lambdas, which would also come with the semantic change that the return expression be subject to SFINAE (this is a desirable property that often leads authors to repeat the return expression, wrapped in decltype, in an explicit return type declaration (and then to devise macros to avoid the repetition)). EWG liked the goal but had parsing-related concerns about the syntax; the author was encouraged to continue exploring the syntax space to find something that’s both parseable and readable. Continued exploration of terser lambdas, whether as part of the same proposal or a separate proposal, was also encouraged. It’s worth noting that there was another proposal in the mailing (which wasn’t discussed since the author wasn’t present) that had significant overlap with this proposal; EWG observed that it might make sense to collaborate on a revised proposal in this space.
  • Default-constructible stateless lambdas. Lambdas are currently not default-constructible, but for stateless lambdas (that is, lambdas that do not capture any variables) there is no justification for this restriction, so this paper proposed removing it. EWG agreed, but suggested that they should also be made assignable. (An example of a situation where one currently runs into these restrictions is transform iterators: such iterators often aggregate the transformation function, and algorithms often default-construct or assign iterators.)
  • Product type access. Recall that structured bindings work with both tuple-like types, and structures with public members. The former expose get<>() functions to access the tuple elements by index; for the latter, structured bindings achieve such index-based access by “language magic”. This paper proposed exposing such language magic via a new syntax or library interface, so that things other than structured bindings (for example, code that wants to iterate over the public members of a structure) can take advantage of it. EWG agreed this was desirable, expressed a preference for a library interface, and sent the proposal onward to LEWG to design said interface. (Compilers will likely end up exposing intrinsics to allow library implementers to implement such an interface. I personally don’t see the advantage of doing things this way over just introducing standard language syntax, but I’m happy to get the functionality one way or the other.)
  • Changing the attack vector of constexpr_vector. At the previous meeting, implementers reported that supporting full-blown dynamic memory allocation in a constexpr context was not feasible to implement efficiently, and suggested a more limited facility, such as a special constexpr_vector container. This proposal argues that such a container would be too limiting, and suggests supporting a constexpr allocator (which can then be used with regular containers) instead. Discussion at this meeting suggested that (a) on the one hand, a constexpr allocator is no less general (and thus no easier to support) than new itself; but (b) on the other hand, more recent implementer experiments suggest that supporting new itself, with some limitations, might be feasible after all. Continued exploration of this topic was warmly encouraged.
  • Implicit evaluation of auto variables. This is a resurrection of an earlier proposal to allow a class to opt into having a conversion function of some sort called when an instance of it is assigned to an auto-typed variable. The canonical use case is an intermediate type is an expression template system, for which it’s generally desirable to trigger evaluation when initializing an auto-typed variable. EWG wasn’t fond of widening the gap between the deduction rules for auto and the deduction rules for template parameters (which is what auto is modelled on), and suggested approaching the problem form a different angle; one idea that was thrown around was the possibility of extending the notion of deduction guides (currently used for class template argument deduction) to apply to situations like this.
  • Allowing class template specializations in unrelated namespaces. The motivation here is to avoid having to reopen the namespace in which a class template was defined, to provide a specialization of that template. EWG liked the idea, but suggested that it might be prudent to still restrict such specializations to occur within associated namespaces of the specialization (e.g. the namespaces of the specialization’s template arguments) – kind of like how Rust doesn’t allow you to implement a trait unless you’re either the author of the trait, or the author of the type you’re implementing the trait for.
  • Precise semantics for contract assertions. This paper explores the design space of contract assertions, enumerating the various (sometimes contradictory) objectives we may want to achieve by using them, and proposes a set of primitive operations that facilitate implementing assertions in ways that meet useful subsets of these objectives. EWG expressed an interest in standardizing some of the proposed primitives, specifically a mechanism to deliberately introduce unspecified (nondeterministic) behaviour into a program, and a “prevent continuation handler” that an assertion check can invoke if an assertion fails and execution should not continue as a result. (A third primitive, for deliberately invoking undefined behaviour, is already handled by the independently proposed std::unreachable() function that EWG approved at this meeting.)

Rejected proposals:

  • Attributes for structured bindings. This proposal would have allowed applying attributes to individual bindings, such as auto [a, b [[maybe_unused]], c] = f();. EWG found this change insufficiently motivated; some people also thought it was inappropriate to give individual bindings attributes when we can’t even give them types.
  • Making pointers-to-members callable. This would have allowed p(s) to be valid and equivalent to s.*p when p is a pointer to a member of a type S, and s is an object of that type. It had been proposed before, and was rejected for largely the same reason: some people argued that it was a half-baked unified call syntax proposal. (I personally thought this was a very sensible proposal – not at all like unified call syntax, which was controversial for changing name lookup rules, which this proposal didn’t do.)
  • Explicit structs. The proposal here was to allow marking a struct as explicit, which meant that all its fields had to be initialized, either by a default member initializer, by a constructor intializer, or explicitly by the caller (not relying on the fallback to default initialization) during aggregate initialization. EWG didn’t find this well enough motivated, observing that either your structure has an invariant, in which case it’s likely to be more involved than “not the default values”, or it doesn’t, in which case the default values should be fine. (Uninitialized values, which can arise for primitive types, are another matter, and can be addressed by different means, such as via the [[uninitialized]] attribute proposal.)
  • Changing the way feature-test macros are standardized. Feature test macros (like __cpp_constexpr, intended to be defined by an implementation if it supports constexpr) are currently standardized in the form of a standing document published by the committee, which is not quite a standard (for example, it does not undergo balloting by national bodies). As they have become rather popular, Microsoft proposed that they be standardized more formally; they claimed that it’s something they’d like to support, but can’t unless it’s a formal standard, because they’re trying to distance themselves from their previous habit of supporting non-standard extensions. (I didn’t quite follow the logic behind this, but I guess large companies sometimes operate in strange ways.) However, there was no consensus to change how feature test macros are standardized; some on the committee dislike them, in part because of their potential for fragmentation, and because they don’t completely remove the need for compiler version checks and such (due to bugs etc.)
  • Removing other language features deprecated in C++17. In addition to throw() (whose removal passed, as mentioned above), two other removals were proposed.
    • Out-of-line declarations of static constexpr data members. By making static constexpr data members implicitly inline, C++17 made it so that the in-line declaration which provides the value is also a definition, making an out-of-line declaration superfluous. Accordingly, the ability to write such an out-of-line declaration at all was deprecated, and was now proposed for removal in C++20.
    • Implicit generation of a copy constructor or copy assignment operator in a class with a user-defined copy assignment operator, copy constructor, or destructor. This has long been known to be a potential footgun (since generally, if you need to user-define one of these functions, you probably need to user-define all three), and C++11 already broke with this pattern by having a user-defined move operation disable implicit generation of the copy operations. The committee has long been eyeing the possibility extending this treatment to user-defined copy operations, and the paper suggested that perhaps C++20 would be the time to do so. However, the reality is that there still is a lot of code out there that relies on this implicit generation, and much of it isn’t actually buggy (though much of it is).

    Neither removal gained consensus. In each case, undeprecating them was also proposed, but that was rejected too, suggesting that the hope that these features can be removed in a future standard remains alive.

  • Capturing *this with initializer. C++17 added the ability to have a lambda capture the entire *this object by value. However, it’s still not possible to capture it by move (which may be reasonable if e.g. constructing the lambda is the last thing you do with the current object). To rectify this, this paper proposed allowing the capture of *this with the init-capture syntax. Unfortunately, this effectively allows rebinding this to refer to a completely unrelated object inside the lambda, which EWG believed would be far too confusing, and there didn’t appear to be a way to restrict the feature to only work for the intended use case of moving the current object.
  • bit_sizeof and bit_offsetof. These are similar to sizeof and offsetof, but count the number of bits, and work on bitfield members. EWG preferred to hold off on these until they are implementable with a library interface on top of reflection.
  • Parametric functions. This oddly-named proposal is really a fresh approach to named arguments. In contrast with the named arguments proposal that I co-authored a few years back, which proposed to allow using named arguments with existing functions and existing parameter names (and garnered significant opposition over concerns that it would make parameter names part of a function’s interface when the function hadn’t been written with that in mind), this paper proposed introducing a new kind of function, which can have named arguments, declared with a new syntax, and for which the parameter names are effectively part of the function’s type. While this approach does address the concerns with my proposal, EWG felt the new syntax and new language machinery it would require was disproportionate to the value of the feature. In spite of the idea’s repeated rejection, no one was under any illusion that this would be the last named arguments proposal to come in front of EWG.

There were a handful of proposals that were not discussed due to their authors not being present. They included the other terse lambdas proposal and its offshoot idea of making forwarding less verbose, and a proposal for an [[uninitialized]] attribute.

Concepts

A major focus of this meeting was to achieve consensus to get Concepts into C++20. To this end, EWG spent half a day plus an evening session discussing several papers on the topic.

Two of the proposals – a unified concept definition syntax and semantic constraint matching – were write-ups of design directions that had already been discussed and approved in Kona; their discussion at this meeting was more of a rubber-stamp. (The second paper contained a provision to require that re-declarations of a constrained template use the same syntax (e.g. you can’t have one using requires-clauses and the other using a shorthand form); this provision had some objections, just as it did in Kona, but was passed anyways.)

EWG next looked at a small proposal to address certain syntax ambiguities; the affected scenarios involve constrained function templates with a requires-clause, where it can be ambiguous where the require-clause after the template parameter list ends, and where the function declaration itself begins. The proposed solution was to restrict the grammar for the expression allowed in a top-level requires-clauses so as to remove the ambiguity; expressions that don’t fit in the restricted grammar can still be used if they are parenthesized (as in requires (expr)). This allows common forms of constraints (like trait<T>::value or trait_v<T>) to be used without parentheses, while allowing any expression with parentheses. This was also approved.

That brings us to the controversial part of the discussion: abbreviated function templates (henceforth, “AFTs”), also called “terse templates”. To recap, AFTs are function templates declared without a template parameter list, where the parameter types use concept names (or auto), which the compiler turns into invented template parameters. A canonical example is void sort(Sortable& s);, which is a shorthand for template <Sortable S> void sort(S& s); (which is itself a shorthand for template <typename S> requires Sortable<S> void sort(S& s);).

AFTs have been controversial since their introduction, due to their ability to make template code look like non-template code. Many have argued that this is a bad idea, beacuse template code is fundamentally different from non-template code (e.g. consider different name lookup rules, the need for syntactic disambiguators like typename, and the ability to define a function out of line). Others have argued that making generic programming (programming with templates) look more like regular programming is a good thing.

(A related feature that shared some of the controversy around AFTs was concept introductions, which were yet another shorthand, making Mergeable{In1, In2, Out} void merge(In1, In1, In2, Out); short for template <typename In1, typename In2, typename Out> requires Mergeable<In1, In2, Out> void merge(In1, In1, In2, Out);. Concept introductions don’t make a template look like a non-template the way AFTs do, but were still controversial as many felt they were an odd syntax and offered yet another way of defining constrained function templates with relatively little gain in brevity.)

The controversy around AFTs and concept introductions was one of the reasons the proposed merger of the Concepts TS into C++17 failed to gain consensus. Eager not to repeat this turn of events for C++20, AFTs and concept introductions were proposed for removal from Concepts, at least for the time being, with the hope that this would allow the merger of Concepts into C++20 to gain consensus. After a long and at times heated discussion, EWG approved this removal, and approved the merger of Concepts, as modified by this removal (and by the other proposals mentioned above), into C++20. As mentioned above, this merger was subsequently passed by the full committee at the end of the week, resulting in Concepts now being in the C++20 working draft!

It’s important to note that the removal of AFTs was not a rejection of having a terse syntax for defining constrained function templates in general. There is general agreement that such a terse syntax is desirable; people just want such a syntax to come with some kind of syntactic marker that makes it clear that a function template (as opposed to a non-template function) is being declared. I fully expect that proposals for an alternative terse syntax that comes with such a syntactic marker will forthcome (in fact, I’ve already been asked for feedback on one such draft proposal), and may even be approved in the C++20 timeframe; after all, we’re still relatively early in the C++20 cycle.

There was one snag about the removal of AFTs that happened at this meeting. In the Concepts wording, AFTs are specified using a piece of underlying language machinery called constrained type specifiers. Besides AFTs, this machinery powers some other features of Concepts, such as the ability to write ConceptName var = expr;, or even vector<auto> var = expr;. While these other features weren’t nearly as controversial as AFTs were, from a specification point of view, removing AFTs while keeping these in would have required a significant refactor of the wording that would have been difficult to accomplish by the end of the week. Since the committee wanted to “strike the iron while it’s hot” (meaning, get Concepts into C++20 while there is consensus for doing so), it was decided that for the time being, constrained type specifiers would be removed altogether. As a result, in the current C++20 working draft, things like vector<auto> var = expr; are ill-formed. However, it’s widely expected that this feature will make it back into C++20 Concepts at future meetings.

Lastly, I’ll note that there were two proposals (one I co-authored, and a second one that didn’t make the pre-meeting mailing) concerning the semantics of constrained type specifiers. The removal of constrained type specifiers made these proposals moot, at least for the time being, so they were not discussed at this meeting. However, as people propose re-adding some of the uses of constrained type specifiers, and/or terse templates in some form, these papers will become relevant again, and I expect they will be discussed at that time.

Modules

Another major goal of the meeting was to send out the Modules TS for its PDTS ballot. I gave an overview of the current state of Modules above. Here, I’ll mention the Modules-related proposals that came before EWG this week:

  • Distinguishing the declaration of a module interface unit from the declaration of a module implementation unit. The current syntax is module M; for both. In Kona, a desire was expressed for interface units to have a separate syntax, and accordingly, one was proposed: export module M;. (The re-use of the export keyword here is due to the committee’s reluctance to introduce new keywords, even context-sensitive ones. module interface M; would probably have worked with interface being a keyword in this context only.) This was approved for the Modules TS.
  • Module partitions (first part of the paper only). These are module units that form part of a module interface, rather than being a complete module interface; the proposed syntax in this paper is module M [[partition]];. This proposal failed to gain consensus, not over syntax concerns, but over semantic concerns: unlike the previous module partitions proposal (which was not presented at this meeting, ostensibly for lack of time), this proposal did not provide for a way for partitions to declare dependencies on each other; rather, each partition was allowed to depend on entities declared in all other partitions, but only on their forward-declarations, which many felt was too limiting. (The corresponding implementation model was to do a quick “heuristic parse” of all partitions to gather such forward-declarations, and then do full processing of the partitions in parallel; this itself resulted in some raised eyebrows, as past experience doing “heuristic parsing” of C++ hasn’t been very promising.) Due to the controversy surrounding this topic, and not wanting to hold up the Modules TS, EWG decided to defer module partitions to Modules v2.
  • Exporting using-declarations. This wasn’t so much a proposal, as a request for clarification of the semantics. The affected scenario was discussed, and the requested clarification given; no changes to the Modules TS were deemed necessary.
  • Name clashes between private (non-exported) entities declared in different modules. Such a name clash is ill-formed (an ODR violation) according to the current spec; several people found that surprising, since one of the supposed advantages of Modules is to shield non-exported entities like private helpers from the outside world. This matter was discussed briefly, but a resolution was postponed to after the PDTS ballot (note: that’s not the same as being postponed to Modules v2; changes can be made to the Modules TS between the PDTS ballot and final publication).
  • A paper describing some requirements that a Modules proposal would need to have to be useful in evolving a particular large codebase (Bloomberg’s). Discussion revealed that the current spec meets some but not all of these requirements; the gaps mainly concern the ability to take a legacy (non-modular) code component, and non-intrusively (“additively”) provide a modular “view” of that component. No changes were proposed at this time, but some changes to fill these gaps are likely to appear as comments on the PDTS ballot.
  • Identifying module source code. Currently, the module M; or export module M; declaration that introduces a module unit is not required to be the first declaration in the file. Preceding declarations are interpreted as being part of the global module (and this is often used to e.g. include legacy headers). The author of this proposal would nonetheless like something that’s required to be the first declaration in the file, that announces “this file is a module unit”, and proposed module ; as being such a marker. EWG was favourable to the idea, but postponed discussion of a concrete syntax until after the PDTS.

With these discussions having taken place, the committee was successful in getting Modules sent out for its PDTS ballot. This is very exciting – it’s a major milestone for Modules! At the same time, I think it’s clear from the nature of the some of the proposals being submitted on the topic (including the feedback from implementation and deployment experience at Google, some of which is yet to be fully discussed by EWG) that this is a feature where there’s still a fair amount of room for implementation convergence and user feedback to gain confidence that the feature as specified will be useful and achieve its intended objectives for a broad spectrum of C++ users. The PDTS ballot formally begins the process of collecting that feedback, which is great! I am very curious about the kinds of comments it will garner.

If you’re interested in Modules, I encourage you to give the prototype implementations in Clang and MSVC a try, play around with them, and share your thoughts and experiences. (Formal PDTS comments can be submitted via your national standards body, but you can also provide informal feedback on the committee’s public mailing lists.)

Other Working Groups

The Library Working Group had a busy week, completing its wording review of the Ranges TS, Networking TS, and library components of the Coroutines TS, and allowing all three of these to be published at the end of the week. They are also in the process of reviewing papers targeting C++20 (including span, which provides an often-requested “array view” facility), Parallelism TS v2, and Library Fundamentals TS v3.

The Library Evolution Working Group was, as usual, working through its large backlog of proposed new library features. As much as I’d love to follow this group in as much detail as I follow EWG, I can’t be in two places at once, so I can’t give a complete listing of the proposals discussed and their outcomes, but I’ll mention a few highlights:

SG 7 (Reflection and Metaprogramming)

SG 7 met for an evening session and discussed three topics.

The first was an extension to the existing static reflection proposal (which is headed towards publication as the initial Reflection TS) to allow reflection of functions. Most of the discussion concerned a particular detail: whether you should be allowed to reflect over the names of a function’s parameters. It was decided that you should, but that in the case of a function with multiple declarations, the implementation is free to choose any one of them as the source of the reflected parameter names.

The second topic was what we want metaprogramming to look like in the longer term. There was a paper exploring the design space that identified three major paradigms: type-based metaprogramming (examples: Boost.MPL, the current static reflection proposal), heterogenerous value-based (example: Boost.Hana), and homogeneous value-based (this would be based on constexpr metaprogramming, and would require some language extensions). Another paper then argued that the third one, homogeneous value-based metaprogramming, is the best choice, both from a compile-time performance perspective (the other two involve a lot of template instantiations which are compile-time expensive), and because it makes metaprogramming look more like regular programming, making it more accessible. SG 7 agreed that this is the long-term direction we should aim for. Note that this implies that, while the Reflection TS will likely be published in its current form (with its type-based interface), prior to merger into the C++ IS it would likely be revised to have a homogenerous value-based interface.

The third topic was a proposal for a more advanced reflection/metaprogramming feature, metaclasses. Metaclasses combine reflection facilities with proposed code injection facilities to allow class definitions to undergo arbitrary user-defined compile-time transformations. A metaclass can be thought of as a “kind” or category of class; a class definition can be annotated (exact syntax TBD) to declare the class as belonging to that metaclass, and such a class definition will undergo the transformations specified in the metaclass definition. Examples of metaclasses might be “interfaces”, where the transformation includes making every method pure virtual, and “value types”, where the transformation includes generating memberwise comparison operators; widely used metaclasses could eventually become part of the standard library. Obviously, this is a very powerful feature, and the proposal is at a very early stage; many aspects, including the proposed code injection primitives (which are likely to be pursued as a separate proposal), need further development. Early feedback was generally positive, with some concerns raised about the feature allowing codebases to grow their own “dialects” of C++.

The Velocity of C++ Evolution

C++ standardization has accelerated since the release of C++11, with the adoption of a three-year standardization cycle, the use of Technical Specifications to gather early feedback on major new features, and an increase in overall participation and the breadth of domain areas and user communities represented in the committee.

All the same, sometimes it feels like the C++ language is still slow to evolve, and a big part of that is the significant constraint of needing to remain backwards-compatible, as much as possible, with older versions of the language. (Minor breakages have occurred, of course, like C++11 user-defined literals changing the meaning of "literal"MACRO. But by and large, the committee has held backwards-compatibility as one of its cornerstone principles.)

A paper, discussed at an evening session this week, explores the question of whether, in today’s age of modern tools, the committee still need to observe this constraint as strictly as it has in the past. The paper observes that it’s already the case that upgrading to a newer compiler version typically entails making some changes / fixes to a codebase. Moreover, in cases where a language change does break or change the meaning of code, compilers have gotten pretty good at warning users about it so they can fix their code accordingly (e.g. consider clang’s -Wc++11-compat warnings). The paper argues that, perhaps, the tooling landscape has matured to the point where we should feel free to make larger breaking changes, as long as they’re of the sort that compilers can detect and warn about statically, and rely on (or even require) compilers to warn users about affected code, allowing them to make safe upgrades (tooling could potentially help with the upgrades, too, in the form of automated refactorings). This would involve more work for compiler implementers, and more work for maintainers of code when upgrading compilers, but the reward of allowing the language to shed some of its cumbersome legacy features may be worth it.

The committee found this idea intriguing. No change in policy was made at this time, but further exploration of the topic was very much encouraged.

If the committee does end up going down this path, one particularly interesting implication would be about the future of the standard library. The Concepts-enabled algorithms in the Ranges TS are not fully backwards-compatible with the algorithms in the current standard library. As a result, when the topic of how to merge the Ranges TS into the C++ standard came up, the best idea people had was to start an “STLv2”, a new version of the standard library that makes a clean break from the current version, while being standardized alongside it. However, in a world where we are not bound to strict backwards-compatibility, that may not be necessary – we may just be able to merge the Ranges TS into the current standard library, and live with the resulting (relatively small) amount of breakage to existing code.

Conclusion

With C++17 effectively done, the committee had a productive meeting working on C++20 and advancing Technical Specifications like Modules and Coroutines.

The merger of Concepts into C++20 was a definite highlight of this meeting – this feature has been long in the making, having almost made C++11, and its final arrival is worth celebrating. Sending out Modules for its PDTS ballot was a close second, as it allows us to start collecting formal feedback on this very important C++ feature. And there are many other goodies in the pipeline: Ranges, Networking, Coroutines, contracts, reflection, graphics, and many more.

The next meeting of the Committee will be in Albuquerque, New Mexico, the week of November 6th, 2017. Stay tuned for my report!

Other Trip Reports

Others have written reports about this meeting as well. Some that I’ve come across include Herb Sutter’s and Guy Davidson’s. Michael Wong also has a report written just before the meeting, that covers concurrency-related topics in more depth than my reports. I encourage you to check them out!


Categorieën: Mozilla-nl planet

Andrew Halberstadt: Try Fuzzy: A Try Syntax Alternative

Mozilla planet - wo, 02/08/2017 - 15:50

It's no secret that I'm not a fan of try syntax, it's a topic I've blogged about on several occasions before. Today, I'm pleased to announce that there's a real alternative now landed on mozilla-central. It works on all platforms with mercurial and git. For those who just like to dive in:

bash $ mach mercurial-setup --update # only if using hg $ mach try fuzzy

This will prompt you to install fzf. After bootstrapping is finished, you'll enter an interface populated with a list of all possible taskcluster tasks. Start typing and the list will be filtered down using a fuzzy matching algorithm. I won't go into details on how to use this tool in this blog post, for that see:

bash $ mach try fuzzy --help # or $ man fzf

For who prefer to look before you leap, I've recorded a demo:

<video width="100%" height="100%" controls> <source src="/static/vid/blog/2017/mach-fuzzy.mp4" type="video/mp4"> </video>

Like the existing mach try command, this should work with mercurial via the push-to-try extension or git via git-cinnabar. If you encounter any problems or bad UX, please file a bug under Testing :: General.

Try Task Config

The following section is all about the implementation details, so if you're curious or want to write your own tools for selecting tasks on try, read on!

This new try selector is not based on try syntax. Instead it's using a brand new scheduling mechanism called try task config. Instead of encoding scheduling information in the commit message, mach try fuzzy encodes it in a JSON file at the root of the tree called try_task_config.json. Very simply (for now), the decision task knows to look for that file on try. If found, it will read the JSON object and schedule every task label it finds. There are also hooks to prevent this file from accidentally being landed on non-try branches.

What this means is that anything that can generate a list (or dict) of task labels can be a try selector. This new JSON format is much easier for tools to write, and for taskgraph to read.

Creating a Try Selector

There are currently two ways to schedule tasks on try (syntax and fuzzy). But I envision 4-5 different methods in the future. For example, we might implement a TestResolver based try selector which given a path can determine all affected jobs. Or there could be one that uses globbing/regex to filter down the task list which would be useful for saving "presets". Or there could be one that uses a curses UI like the hg trychooser extension.

To manage all this, each try selector is implemented as an @SubCommand of mach try. The regular syntax selector, is implemented under mach try syntax now (though mach try without any subcommand will dispatch to syntax to maintain backwards compatibility). All this lives in a newly created tryselect module.

If you have want to create a new try selector, you'll need two things:

  1. A list of task labels as input.
  2. The ability to write those labels to try_task_config.json and push it to try.

Luckily tryselect provides both those things. The first, can be obtained using the tasks.py module. It basically does the equivalent of running mach taskgraph target, but will also automatically cache the resulting task list so future invocations run much quicker.

The second can be achieved using the vcs.py module. This uses the same approach that the old syntax selector has been using all along. It will commit try_task_config.json temporarily and then remove all traces of the commit (and of try_task_config.json).

So to recap, creating a new try selector involves:

  1. Add an @SubCommand to the mach_commands.py, which dispatches to a file under the selectors directory.
  2. Generate a list of tasks using tasks.py.
  3. Somehow filter down that list (this part is up to you)
  4. Push the filtered list using vcs.py

You can inspect the fuzzy implementation to see how all this ties together.

Future Considerations

Right now, the try_task_config.json method only allows specifying a list of task labels. This is good enough to say what is running, but not how it should run. In the future, we could expand this to be a dict where task labels make up the keys. The values would be extra task metadata that the taskgraph module would know how to apply to the relevant tasks.

With this scheme, we could do all sorts of crazy things like set prefs/env/arguments directly from a try selector specialized to deal with those things. There are no current plans to implement any of this, but it would definitely be a cool ability to have!

Categorieën: Mozilla-nl planet

Andrew Halberstadt: Try Fuzzy: A Try Syntax Alternative

Mozilla planet - wo, 02/08/2017 - 15:50

It's no secret that I'm not a fan of try syntax, it's a topic I've blogged about on several occasions before. Today, I'm pleased to announce that there's a real alternative now landed on mozilla-central. It works on all platforms with mercurial and git. For those who just like to dive in:

bash $ mach mercurial-setup --update # only if using hg $ mach try fuzzy

This will prompt you to install fzf. After bootstrapping is finished, you'll enter an interface populated with a list of all possible taskcluster tasks. Start typing and the list will be filtered down using a fuzzy matching algorithm. I won't go into details on how to use this tool in this blog post, for that see:

bash $ mach try fuzzy --help # or $ man fzf

For who prefer to look before you leap, I've recorded a demo:

<video width="100%" height="100%" controls> <source src="/static/vid/blog/2017/mach-fuzzy.mp4" type="video/mp4"> </video>

Like the existing mach try command, this should work with mercurial via the push-to-try extension or git via git-cinnabar. If you encounter any problems or bad UX, please file a bug under Testing :: General.

Try Task Config

The following section is all about the implementation details, so if you're curious or want to write your own tools for selecting tasks on try, read on!

This new try selector is not based on try syntax. Instead it's using a brand new scheduling mechanism called try task config. Instead of encoding scheduling information in the commit message, mach try fuzzy encodes it in a JSON file at the root of the tree called try_task_config.json. Very simply (for now), the decision task knows to look for that file on try. If found, it will read the JSON object and schedule every task label it finds. There are also hooks to prevent this file from accidentally being landed on non-try branches.

What this means is that anything that can generate a list (or dict) of task labels can be a try selector. This new JSON format is much easier for tools to write, and for taskgraph to read.

Creating a Try Selector

There are currently two ways to schedule tasks on try (syntax and fuzzy). But I envision 4-5 different methods in the future. For example, we might implement a TestResolver based try selector which given a path can determine all affected jobs. Or there could be one that uses globbing/regex to filter down the task list which would be useful for saving "presets". Or there could be one that uses a curses UI like the hg trychooser extension.

To manage all this, each try selector is implemented as an @SubCommand of mach try. The regular syntax selector, is implemented under mach try syntax now (though mach try without any subcommand will dispatch to syntax to maintain backwards compatibility). All this lives in a newly created tryselect module.

If you have want to create a new try selector, you'll need two things:

  1. A list of task labels as input.
  2. The ability to write those labels to try_task_config.json and push it to try.

Luckily tryselect provides both those things. The first, can be obtained using the tasks.py module. It basically does the equivalent of running mach taskgraph target, but will also automatically cache the resulting task list so future invocations run much quicker.

The second can be achieved using the vcs.py module. This uses the same approach that the old syntax selector has been using all along. It will commit try_task_config.json temporarily and then remove all traces of the commit (and of try_task_config.json).

So to recap, creating a new try selector involves:

  1. Add an @SubCommand to the mach_commands.py, which dispatches to a file under the selectors directory.
  2. Generate a list of tasks using tasks.py.
  3. Somehow filter down that list (this part is up to you)
  4. Push the filtered list using vcs.py

You can inspect the fuzzy implementation to see how all this ties together.

Future Considerations

Right now, the try_task_config.json method only allows specifying a list of task labels. This is good enough to say what is running, but not how it should run. In the future, we could expand this to be a dict where task labels make up the keys. The values would be extra task metadata that the taskgraph module would know how to apply to the relevant tasks.

With this scheme, we could do all sorts of crazy things like set prefs/env/arguments directly from a try selector specialized to deal with those things. There are no current plans to implement any of this, but it would definitely be a cool ability to have!

Categorieën: Mozilla-nl planet

Mozilla Addons Blog: August’s Featured Extensions

Mozilla planet - ti, 01/08/2017 - 23:49

Firefox Logo on blue background

Pick of the Month: Grammarly

by Grammarly
It’s like having an expert proofreader with you at all times. Grammarly offers contextual spell checks (it understands the distinction between “there” and “their” unlike 98% of English-speaking humans) as well as grammar edits.

“As a student at a university, this is the perfect tool to correct all of my writing mistakes.”

Featured: FoxyTab

by erosman
Enjoy a suite of tab related actions, like copy all URLs, close multiple tabs, tab duplication, and more.

“This literally saved my day. Especially someone like me who works as an administrator and moderator on multiple forums, this is a great tool.”

Featured: Zoom Page WE

by DW-dev
Use full-page zoom, text-only zoom, fit-to-width feature, and other ways to focus in on Web pages.

“This is the best text zooming add-on you can find.”

Featured: Save All Images

by belav
Detects all images on any given page and presents a simple way to instantly download them.

“Very useful.”

Featured: YouTube Dark Mode

by HTCom
Turn YouTube completely dark to enhance your viewing experience.

“Simple, effective, no unintentional side effects.”

Featured: EPUBReader

by EPUBReader
Read ebook files right in your browser.

“Works flawlessly.”

Featured: Tab Auto Refresh

by Alex
Automatically refresh tabs based on custom time intervals.

“This is the first auto reload/refresh Firefox extension that I’ve found that can be customized for each tab that I have.”

Featured: TinEye Reverse Image Search

by TinEye
A new kind of reverse image searcher that uses image identification technology rather than keywords, metadata, or watermarks.

“If the image in question can’t be found, this finds it.”

Featured: Awesome Screenshot Plus

by Diigo Inc.
Take full page or partial screen grabs. Annotate with text and graphics. Store and share files. This is a full-service screenshot tool.

“It gives you a lot of options to edit, email, print. You will not be sorry!”

Nominate your favorite add-ons

Featured add-ons are selected by a community board made up of add-on developers, users, and fans. Board members change every six months. Here’s further information on AMO’s featured content policies.

If you’d like to nominate an add-on for featuring, please send it to amo-featured [at] mozilla [dot] org for the board’s consideration. We welcome you to submit your own add-on!

The post August’s Featured Extensions appeared first on Mozilla Add-ons Blog.

Categorieën: Mozilla-nl planet

Air Mozilla: Intern Presentations: Round 2: Tuesday, August 1st

Mozilla planet - ti, 01/08/2017 - 22:00

 Tuesday, August 1st Intern Presentations 11 presenters Time: 1:00PM - 3:45PM (PDT) - each presenter will start every 15 minutes 7 MTV, 2 TOR, 1 Paris, 1 London

Categorieën: Mozilla-nl planet

Air Mozilla: Intern Presentations: Round 2: Tuesday, August 1st

Mozilla planet - ti, 01/08/2017 - 22:00

 Tuesday, August 1st Intern Presentations 11 presenters Time: 1:00PM - 3:45PM (PDT) - each presenter will start every 15 minutes 7 MTV, 2 TOR, 1 Paris, 1 London

Categorieën: Mozilla-nl planet

Sebastin Santy: GSoC Week 9: Attaching Files

Mozilla planet - ti, 01/08/2017 - 20:22

Hi everyone,

This is a very short update, and there wasn’t much of an issue regarding this patch. It was merely adding the attachments with less options available to the user. We eliminated the review flags, as a bug filing process usually includes adding an image or a log as an attachment, and not a patch (for which a review might be required).

We also planned that we’ll avoid putting the tracking flags feature in the new-bug, as flagging is often done, after the bug is filed, and having them, would unnecessarily clutter the interface. The development is going a bit slow again, as it is getting difficult for me to juggle between timezones, but I hope, it will be completed before the deadline - Batteries Included! :-)

Categorieën: Mozilla-nl planet

Mark Banner: Moving Blog

Mozilla planet - ti, 01/08/2017 - 20:05

Just a short note to say that I’m moving my blogging to https://www.thebanners.uk/standard8/

Categorieën: Mozilla-nl planet

Air Mozilla: Webdev Extravaganza: August 2017

Mozilla planet - ti, 01/08/2017 - 19:00

 August 2017 Once a month web developers across the Mozilla community get together (in person and virtually) to share what cool stuff we've been working on. This...

Categorieën: Mozilla-nl planet

Air Mozilla: Webdev Extravaganza: August 2017

Mozilla planet - ti, 01/08/2017 - 19:00

 August 2017 Once a month web developers across the Mozilla community get together (in person and virtually) to share what cool stuff we've been working on. This...

Categorieën: Mozilla-nl planet

Mozilla Addons Blog: NoScript’s Migration to WebExtensions APIs

Mozilla planet - ti, 01/08/2017 - 18:00

We asked Giorgio Maone, developer of the popular security extension NoScript, to share his experience about migrating to WebExtension APIs. Originally released in 2005, NoScript was developed to address security vulnerabilities in browsers by pre-emptively blocking scripts from untrusted websites. Over the time it grew into a security suite including many additional and often unique countermeasures against various web-based threats, such as Cross-Site Scripting (XSS), Cross-Site Request Forgery (CSRF) and Clickjacking.

Why did you decide to transition NoScript to WebExtension APIs?

The so-called “legacy” add-on technology which NoScript has been built with is going to be banned very soon; therefore, like too often in real life, it’s either migrate or die. Many people rely on NoScript for being safer on the Web and in some cases for their physical security too, making this transition, although quite painful, an ethical obligation not to leave them in the cold.

For a long time, I strove to maintain as much backwards compatibility as possible, in order to offer some protection to those users stuck for various reasons with older, inherently less safe, versions of Firefox. For this reason, the legacy version of NoScript contains a lot of code for working around bugs that Firefox has since fixed: this cruft can safely go away during the migration. The plan is to have a lean and mean version of NoScript available as soon as Firefox 57 is released. Some of the APIs required for full parity with the legacy version won’t land until Firefox 57. Until then, I can selectively delegate and prioritize some features to WebExtension APIs that already work by packaging NoScript as an Embedded WebExtension, which is also the best way to migrate user preferences.

On the other hand, people who need NoScript most are those who use the anonymity and security specialized Tor Browser, which is based on Firefox ESR (Gecko 54 until June 2018) where NoScript is not viable yet as a WebExtension. Therefore I’m forced to maintain two very different code bases for almost one year after the release of Firefox 57, in order to support the vast majority of my users.

Can you tell us about where you are with the migration?

NoScript already ships as a hybrid add-on, and I am in the process of moving all the code to WebExtensions APIs. Some features are even more performant on the new platform: it’s the case of the XSS filter, which takes advantage of the more asynchronous architecture of WebExtensions. The legacy XSS filter may stall the browser for a few seconds while checking very large (fortunately unusual) payloads; the WebExtensions-based version allows the browser to stay responsive no matter the load.

Have you had to use WebExtension APIs in creative ways?

NoScript 10 as a WebExtension is built mainly around the WebRequest API, which in its Firefox incarnation features some tweaks differentiating it from its Chromium counterpart. Last year I’ve been working together with the WebExtensions team to develop this enhanced API: a very pleasant experience and a welcome chance for me to contribute code on Mozilla Central again, after quite awhile. Dynamic permissions for embedded JavaScript are not natively supported by WebExtensions. Rather than requesting a new API, I am using Content Security Policies (CSP), a Web Application Security standard, to control scripting execution and other security properties of the webpage. Likewise I’m leveraging other Web Platform (HTML5) features, which were not available yet in the early NoScript days, to rebuild functionality originally based on “legacy” XUL and XPCOM technology.

What advice would you give to other legacy add-on developers?

Try to find workarounds for any missing pieces and be creative when using available APIs, not limited to just within the WebExtensions APIs boundaries but also exploring the whole Web Platform. If workarounds are impossible, ask the add-ons team for additions or enhancements.

Also, try to join and lobby the Browser Extensions Community Group hosted by the W3C. I feel that Mozilla has the most flexible and dynamically growing browser extensions platform, but it would be nice to make sure that the good ideas landing in Firefox also be available in Chrome and other browsers.

Thank you, Giorgio! Best of luck with the migration.

The post NoScript’s Migration to WebExtensions APIs appeared first on Mozilla Add-ons Blog.

Categorieën: Mozilla-nl planet

Justin Dolske: Photon Engineering Newsletter #10

Mozilla planet - ti, 01/08/2017 - 17:55

Woo yeah! Time for Photon newsletter #10!

Nightly-57 this week

Way back in newsletter #2, I talked about the Photon program schedule. Briefly, to save you a click: Photon is shipping with Firefox 57, and to allow time for bugfixes, quality, and polish we’ve been targeting August 7th as the date when we’ll be done with “major work.” That gives us 6 weeks of Nightly-57 to do that bugfixing (and another 6 weeks of Beta-57 for any further critical or low-risk improvements).

I’m pleased to report that we’re still solidly on track. Most of the big-ticket features for Photon have already landed, and the last few (notably: rectangular tabs, pinning Page Action items to the URL bar) are in good shape to land soon. That’s not to say Photon is “done” – just that the biggest and riskiest work will largely be behind us, and upcoming work will start to be more about finishing off rough edges.

Recent Changes

Menus/structure:

Animation:

  • The Stop/Reload animation has been tweaked to run faster.
  • Animations have been fixed to be positioned correctly regardless of display font size. [1] [2]
  • The Save to Bookmarks animation has landed in Nightly. (Add the Library button to the toolbar for the full effect!)
    star
    bookmark-animation
  • The Save to Pocket animation has also landed  (Again, you’ll want to ensure the Library button is in the toolbar to see all of the animation.)
    pocket-animation

Preferences:

  • Fixed Performance section regression around number of processes and uplifted it to Beta-55.
  • Started working on visual refresh but are holding off landing until after the uplift. This allows QA to finish verifying the changes (in Nightly) that will ship with Firefox 56, without these 57-only changes getting in the way.

Visual redesign:

Onboarding:

  • The first uncompleted tour is now shown by default (instead of just the first tour).
  • Updated the stub installer tagline to “Built for people, not for profit.
  • Made the “Learn More” button not wrap.
  • The Sync tour will be automatically marked as completed when you sign in with a Firefox account.
  • When refreshing a profile, don’t migrate a user’s session (tabs) unless the refresh was invoked by the user. This allows the reset triggered by the stub installer (e.g. for users coming back to Firefox after a long absence) to have a fresh experience, instead of seeing old tabs from months ago.
  • Made the onboarding UI look better in high-contrast mode.

Performance:

 

That’s it for now!


Categorieën: Mozilla-nl planet

Air Mozilla: Martes Mozilleros, 01 Aug 2017

Mozilla planet - ti, 01/08/2017 - 17:00

Martes Mozilleros Reunión bi-semanal para hablar sobre el estado de Mozilla, la comunidad y sus proyectos. Bi-weekly meeting to talk (in Spanish) about Mozilla status, community and...

Categorieën: Mozilla-nl planet

Air Mozilla: Martes Mozilleros, 01 Aug 2017

Mozilla planet - ti, 01/08/2017 - 17:00

Martes Mozilleros Reunión bi-semanal para hablar sobre el estado de Mozilla, la comunidad y sus proyectos. Bi-weekly meeting to talk (in Spanish) about Mozilla status, community and...

Categorieën: Mozilla-nl planet

The Mozilla Blog: New Test Pilot Experiments Available Today

Mozilla planet - ti, 01/08/2017 - 16:00

It’s been a busy summer for Firefox!                     Last month, we delivered the first in a series of groundbreaking updates to the browser. This week, the Test Pilot team is continuing to evolve Firefox features with three new experiences that will make for a simpler, faster and safer experience.

Send

Sending files over the internet is something many of us do everyday. Mozilla makes it easy to keep your files safe. With Send, your files self-destruct after download, so they can’t be accessed by anyone else. Your files are encrypted during transmission. Plus, Send encrypts files on the client side, so that not even Mozilla can read them.

Voice Fill

Mozilla is a champion of making the web open and accessible to everyone. With Voice Fill, we’re experimenting with support for Speech to Text (STT) functionality in Firefox, making it possible for users to input text in Firefox by voice. Your contributions to this experiment will help us optimize speech to text input so that we can expand support throughout Firefox.

Notes

Whether it’s a sticky note, an app or the back of an envelope, so many of us rely on jotting down quick notes to keep track of our busy lives. Notes is a simple, convenient place to take, store and retrieve notes – all within Firefox. We’re also working to build in support for Firefox Accounts, so you can sync your notes wherever you use Firefox

The Test Pilot program is open to all Firefox users and helps us test and evaluate a variety of potential Firefox features. To activate Test Pilot and help us build the future of Firefox, visit testpilot.firefox.com.

If you’ve experimented with Test Pilot features before, you know that you might run into some bugs or lose some of the polish in Firefox, so you can easily enable or disable features at any time.

We want your feedback! Try out these and other Test Pilot experiments and help us decide which new features to build into future versions of Firefox.

The post New Test Pilot Experiments Available Today appeared first on The Mozilla Blog.

Categorieën: Mozilla-nl planet

The Mozilla Blog: Mozilla releases research results: Zero rating is not serving as an on-ramp to the internet

Mozilla planet - ti, 01/08/2017 - 07:00

Can digital literacy and Equal Rating solutions help connect the unconnected?

Today, 4 billion people live without the internet. There’s a global debate about how to connect the unconnected, but it’s often dominated by assumptions and not a lot of data or talking to actual users on the ground.

To better inform this issue, Mozilla recently supported a series of focus groups to investigate how and why people use subsidized services in India, Myanmar, Peru, Kenya, Nigeria, Rwanda and South Africa. Today, we’re releasing the results of this research carried out by Research ICT Africa, LIRNEasia and IEP.

Credit: Peter Cihon

Why do we care?
Many companies and organizations are working to connect the unconnected. For us at Mozilla, it is our mission to ensure the internet is a global public resource that’s open and accessible to all.

We’ve focused our work in this space on a concept we call Equal Rating. Building on Mozilla’s strong commitment to net neutrality, Equal Rating models are free of discrimination, gatekeepers, and pay-to-play schemes. Equal Rating stands in contrast to zero rating business models, which reduce the cost to zero only for some sites and services. We’ve pursued this through policy engagement with governments, an innovation challenge to catalyze new thinking in providing affordable access, and this research.

What did we ask?

  • What barriers are keeping people offline?
  • Is zero rating serving as an on-ramp to the internet?
  • Why and how do people use subsidized services?
  • Do people move beyond subsidized services, or do they just stay in the subsidized walled garden?
  • How does use of subsidized services affect future internet usage?

What did we find?

Zero rating is not serving as an on-ramp to the internet
In all countries surveyed — excluding India where zero rating has been banned by the regulator — focus groups revealed that users are not coming online through zero rated services. While more research is needed, if zero rating is not actually serving as an on-ramp to bring people online, the benefits seem low, while the resulting risk of these offerings creating an anti-competitive environment is extremely high.

People use zero rating as one of many cost saving strategies
This research revealed that people who use zero rated services usually also have full access to the internet, and make use of zero rated and subsidized data services as one of many money-saving strategies, including:

  • Use of multiple SIM cards to take advantage of promotions, better reception quality, or better prices for a given service.
  • Use of public Wi-Fi. For example, many buses in Kenya now provide Wi-Fi access, and participants reported being willing to wait for a bus that was Wi-Fi-enabled.
  • Tethering to mobile hotspots. In South Africa and India, users not only share data but also promotions and subsidized offers from one phone to another.
  • Earned reward applications (where users download, use, or share a promoted application in return for mobile data/credit). The research indicates that most users tend to play the system to get the most credit possible and then abandon the earned reward application.
  • While users, especially in the African studies, report skepticism about whether zero rated promotions are truly free, partially subsidized bundles are popular. Notably, many of these offerings are Equal Rating compliant.

Some, particularly rural and low income users, are trapped in walled gardens
While zero rated services tend to be only part of internet usage for most users studied, some users are getting trapped in the walled gardens of these subsidized offerings.

  • In particular, low income respondents in Peru and Rwanda use zero rated content for much of their browsing activity, as do rural respondents in Myanmar.
  • Awareness matters: in Myanmar, respondents who know they are in a zero rated walled garden (e.g., due to lack of photos and video) are more likely to access the full internet beyond the walled garden.
  • But, when Facebook is subsidized without impacting user experience, users tend to concentrate their usage on that single site, demonstrating concerns around the anti-competitive effects of zero rating.

Digital illiteracy limits access for connected and unconnected alike
Infrastructure and affordability are commonly cited barriers to internet access around the world; yet, this research also points to a third important barrier: digital literacy.

  • Users and non-users alike do not understand all that the internet can offer.
  • Users generally restrict their internet use to a few large websites and services.
  • A lack of understanding about the internet and internet-connected devices exacerbates misconceptions and spreads fear around privacy, security, and health, which in turn undermines use of the internet. One Kenyan respondent said of non-users: “there are some assumptions that they can get diseases transmitted to them like skin cancer through the use of the internet.”

Many companies and NGOs are already doing great work to advance digital literacy, but we need to scale up these efforts.

Competition, literacy, language, and gender are also barriers to internet access

This research highlighted a series of consistent and persistent barriers to access.

  • While 95% of the world has access to an internet signal, far too often, users have access to only one, low quality provider, usually the most expensive option in their country.
  • Without basic literacy, some respondents cannot access the internet. As one respondent in rural South Africa said, “if you cannot read or write you cannot use internet, many people in this community are not educated and I believe most of them want to be able to use internet because it makes life easier.”
  • Others in Myanmar, Peru, and Rwanda cite the lack of local language content and tools as keeping them from coming online.
  • Evidence of a gendered digital divide is seen throughout all of the countries studied, with some women afraid of “breaking the machine” while others say social stigma, domestic abuse, negative impressions, and housework obligations limit their use of the internet.

These are just some of the highlights and interesting findings. We have results from nearly 80 focus groups in these seven countries. For more detailed information, the country summaries and full reports are available here.

Next steps to bring the next 4 billion online

Mozilla supported this research to help better inform what we believe is a global imperative to bring the world’s 4 billion unconnected people online to access the full and open internet.

Based on these findings, we believe the internet needs:

  • The development of more Equal Rating compliant models, many of which seemed to be quite popular with research respondents and provide access to the full diversity of the open internet, not just some parts of it.
  • Further investment in digital literacy training, especially in schools, on devices, and in retail outlets. For more information about Mozilla’s digital literacy efforts, see our recent Digital Skills Observatory study.
  • Work on all barriers to access to address infrastructure investment especially in rural areas, affordability, local content and local language tools, and gender equality.

Bringing the full internet to all people is one of the great challenges of our time. While we know there is more research needed, this research better informs the global debate on how to connect the unconnected, and makes clear the challenges ahead. We are committed to tackling these challenges but we know it will take all of us — tech companies, telecom companies, governments, civil society groups and philanthropists — working together to get everyone online.


We’d like to thank the researchers at Research ICT Africa, LIRNEasia, and IEP, as well as Jochai Ben-Avie (who manages all our Equal Rating work) and Peter Cihon (our awesome summer intern) who helped analyze this research
.

The post Mozilla releases research results: Zero rating is not serving as an on-ramp to the internet appeared first on The Mozilla Blog.

Categorieën: Mozilla-nl planet

This Week In Rust: This Week in Rust 193

Mozilla planet - ti, 01/08/2017 - 06:00

Hello and welcome to another issue of This Week in Rust! Rust is a systems language pursuing the trifecta: safety, concurrency, and speed. This is a weekly summary of its progress and community. Want something mentioned? Tweet us at @ThisWeekInRust or send us a pull request. Want to get involved? We love contributions.

This Week in Rust is openly developed on GitHub. If you find any errors in this week's issue, please submit a PR.

Updates from Rust Community News & Blog Posts Crate of the Week

This week's crate is tarpaulin, a crate to collect test coverage of your Rust code. Thanks to Colin Kiegel for the suggestion.

Submit your suggestions and votes for next week!

Call for Participation

Always wanted to contribute to open-source projects but didn't know where to start? Every week we highlight some tasks from the Rust community for you to pick and get started!

Some of these tasks may also have mentors available, visit the task page for more information.

If you are a Rust project owner and are looking for contributors, please submit tasks here.

Updates from Rust Core

146 pull requests were merged in the last week

New Contributors
  • Daiki Mizukami
  • Danek Duvall
  • Isaac van Bakel
  • Joshua Liebow-Feeser
  • MaulingMonkey
  • Richard Dodd
Approved RFCs

Changes to Rust follow the Rust RFC (request for comments) process. These are the RFCs that were approved for implementation this week:

Final Comment Period

Every week the team announces the 'final comment period' for RFCs and key PRs which are reaching a decision. Express your opinions now. This week's FCPs are:

New RFCs Style RFCs

Style RFCs are part of the process for deciding on style guidelines for the Rust community and defaults for Rustfmt. The process is similar to the RFC process, but we try to reach rough consensus on issues (including a final comment period) before progressing to PRs. Just like the RFC process, all users are welcome to comment and submit RFCs. If you want to help decide what Rust code should look like, come get involved!

The RFC style is now the default style in Rustfmt - try it out and let us know what you think!

Currently being discussed:

Upcoming Events

If you are running a Rust event please add it to the calendar to get it mentioned here. Email the Rust Community Team for access.

Rust Jobs

Tweet us at @ThisWeekInRust to get your job offers listed here!

Quote of the Week

No quote was selected for QotW.

Submit your quotes for next week!

This Week in Rust is edited by: nasa42, llogiq, and brson.

Categorieën: Mozilla-nl planet

Pages