mozilla

Mozilla Nederland LogoDe Nederlandse
Mozilla-gemeenschap

Ryan Harter: Managing Someday-Maybe Projects with a CLI

Mozilla planet - wo, 03/01/2018 - 09:00

I have a problem managing projects I'm interested in but don't have time for. For example, the CLI for generating slack alerts I posted about last year. Not really a priority, but helpful and not that complicated. I sat on that project for about a year before I could finally …

Categorieën: Mozilla-nl planet

The Rust Programming Language Blog: New Year's Rust: A Call for Community Blogposts

Mozilla planet - wo, 03/01/2018 - 01:00

‘Tis the season for people and communities to reflect and set goals- and the Rust team is no different. Last month, we published a blogpost about our accomplishments in 2017, and the teams have already begun brainstorming goals for next year.

Last year, the Rust team started a new tradition: defining a roadmap of goals for the upcoming year. We leveraged our RFC process to solicit community feedback. While we got a lot of awesome feedback on that RFC, we’d like to try something new in addition to the RFC process: a call for community blog posts for ideas of what the goals should be.

As open source software becomes more and more ubiquitous and popular, the Rust team is interested in exploring new and innovative ways to solicit community feedback and participation. We’re commited to extending and improving our community organization and outreach- and this effort is just the first of what we hope to be many iterations of new kinds of community feedback mechanisms.

#Rust2018

Starting today and running until the end of January we’d like to ask the community to write blogposts reflecting on Rust in 2017 and proposing goals and directions for Rust in 2018. These can take many forms:

  • A post on your personal or company blog
  • A Medium post
  • A GitHub gist
  • Or any other online writing platform you prefer.

We’re looking for posts on many topics:

  • Ideas for community programs
  • Language features
  • Documentation improvements
  • Ecosystem needs
  • Tooling enhancements
  • Or anything else Rust related you hope for in 2018 :D

A great example of what we’re looking for is this post, “Rust and the case for WebAssembly in 2018” by @mgattozzi or this post, “Rust in 2017” by Anonyfox.

You can write up these posts and email them to community@rust-lang.org or tweet them with the hashtag #Rust2018. We’ll aggregate any blog posts sent via email or with the hashtag in one big blog post here.

The Core team will be reading all of the submitted posts and using them to inform the initial roadmap RFC for 2018. Once the RFC is submitted, we’ll open up the normal RFC process, though if you want, you are welcome to write a post and link to it on the GitHub discussion.

Preliminary Timeline

We hope to get a draft roadmap RFC posted in mid January, so blog posts written before then would be the most useful. We expect discussion and final comment period of that RFC to last through at least the end of January, though, so blog posts until then will also be considered for ideas.

Dates are likely to change, but this is a general overview of the upcoming process:

  • Jan 3: call for posts!
  • throughout Jan: read current posts, draft roadmap RFC
  • mid Jan: post roadmap RFC on GitHub
  • late Jan: evaluate roadmap based on RFC comments
  • late Jan - early Feb: final RFC comment period
  • Feb: assuming discussion has reached a steady state, and we’ve found consensus, accept final roadmap

So, as we kick off 2018, if you find yourself in a reflective mood and your mind drifts to the future of Rust, please write up your thoughts and share them! We’re excited to hear where you want Rust to go this coming year!

Categorieën: Mozilla-nl planet

Shing Lyu: Taking notes with MkDocs

Mozilla planet - ti, 02/01/2018 - 21:14

I’ve been using TiddlyWiki for note-taking for a few years. I use them to keep track of my technical notes and checklists. TiddlyWiki is a brilliant piece of software. It is a single HTML file with a note-taking interface, where the notes you take are stored directly in the HTML file itself, so you can easily carry (copy) the file around and easily deploy it online for sharing. However, most modern browsers don’t allow web pages to access the filesystem, so in order to let TiddlyWiki save the notes, you need to rely on browser extensions or Dropbox integration service like TiddlyWiki in the Sky. But they still have some frictions.

So recently I started to look for other alternatives for note-taking. Here are some of the requirements I desire:

  1. Notes are stored in non-proprietary format. In case the service/software is discontinued, I can easily migrate them to other tool.
  2. Has some form of formatting, e.g. write in Markdown and render to HTML.
  3. Auto-generated table of contents and search functionality.
  4. Can be used offline and data is stored locally.
  5. Can be easily shared with other people.
  6. Can be version-controlled, and you won’t lose data if there is a conflict.

TiddlyWiki excels at 1 to 5, and I can easily sync it with Dropbox. However, I often forgot to click “save” in TiddlyWiki, and in some case when I accidentally create some conflict while syncing, Dropbox simply creates two copies and I have to manually merge them. There is also no version history so it’s hard to merge and look into the history.

Then I suddenly had an epiphany during shower: all I need is a bunch of Markdown files, version controlled by git, then I can use a static site generator to render them as HTML, with table of contents and client-side search. After some quick search I found MkDocs, which is a Python-based static-site generator for project documentations. It also have a live-reloading development server, which I really love.

Using MkDocs

MkDocs is really straight-forward to setup and use (under Linux). To install, simply use pip (assuming you have Python and pip installed):

sudo pip install mkdocs

Then you can create your document folder using

mkdocs new <project name>

The generated folder will have the following structure:

<project name> ├── docs │   └── index.md └── mkdocs.yml

You can then start to write documents in the docs folder. You can create subfolders to organize the Markdown files. To view the generated HTML, cd into the project folder, then run mkdocs serve, the development server will start to serve on port 8000. Opening 127.0.0.1:8000 in your browser then you can see the document. You can also run mkdocs build to generate the static HTML into the sites folder. The folder can then be hosted using any server.

MkDocs

The mkdocs.yml file contains some configuration. For example, you can use the ReadTheDocs theme by adding the line:

theme: readthedocs

readthedocs

If you wish to open the generated site locally using the file:// protocol, you can add this line:

use_directory_urls: false

When you have a note named foo.md, the generated file will be /foo/index.html. If use_directory_urls is set to true, the URL for linking to the foo.md page will be /foo/, which is a more modern URL naming convention. But for this routing to work, you must host the files in a web server. If you want to open it locally, you need to set the config to false and all the link will be /foo/index.html instead.

Migrating from TiddlyWiki

Moving all the notes from TiddlyWiki to MkDocs is very easy. If you are using TiddlyWiki 5.x+, you can go to “Tools” right under the search box, there is a “export all” button. Export the tiddlers (notes) to CSV. Then you can use the tiddly2md script to convert the CSV to individual .md files. If your tiddler has UTF-8 titles, you need to add a parameter encoding='utf-8' to the pd.read_csv() call in the script for it to work.

export_all

The exported Markdown files will lose the tag information, so if you are using tag as folder structure like me, you’ll have to manually create folders to arrange them. Some tiddlers using the old WikiText format will also be empty (I use Markdown in my TiddlyWiki, but there are some old ones from the old versions). You can use ls -lS to see which file has no content and manually fix them. After the .md files are in place, run mkdocs as usual.

Conclusion

MkDocs is a pretty simple but powerful tool for note-taking. You get all the benefit of editing the notes using plain text editor, and have them version controlled by git. But you also get a nicely rendered HTML version with search functionality. One thing I miss from TiddlyWiki is the ability to generate a single HTML file containing all the notes. (MkDocs generates a folder of separate HTML, CSS and JS files.) There are some scripts like mkdocs-combine that claims to do this using Pandoc, but I haven’t tested them.

Categorieën: Mozilla-nl planet

Ryan Harter: Removing Disqus

Mozilla planet - ti, 02/01/2018 - 09:00

I'm removing Disqus from this blog. Disqus allowed readers to post comments on articles. I added it because it was easy to do, but I no longer think it's worth keeping.

If you'd like to share your thoughts, feel free to shoot me an email at harterrt on gmail. I …

Categorieën: Mozilla-nl planet

This Week In Rust: This Week in Rust 215

Mozilla planet - ti, 02/01/2018 - 06:00

Hello and welcome to another issue of This Week in Rust! Rust is a systems language pursuing the trifecta: safety, concurrency, and speed. This is a weekly summary of its progress and community. Want something mentioned? Tweet us at @ThisWeekInRust or send us a pull request. Want to get involved? We love contributions.

This Week in Rust is openly developed on GitHub. If you find any errors in this week's issue, please submit a PR.

Updates from Rust Community News & Blog Posts Crate of the Week

This week's crate is YEW, a framework for making Elm/React/Angular-like client web-apps with Rust. Thanks to Willi Kappler for the suggestion!

Submit your suggestions and votes for next week!

Call for Participation

Always wanted to contribute to open-source projects but didn't know where to start? Every week we highlight some tasks from the Rust community for you to pick and get started!

Some of these tasks may also have mentors available, visit the task page for more information.

If you are a Rust project owner and are looking for contributors, please submit tasks here.

Updates from Rust Core

79 pull requests were merged in the last week

New Contributors
  • Bastian Köcher
  • Josh Holmer
  • Matt Peterson
  • Rafael Fernández López
Approved RFCs

Changes to Rust follow the Rust RFC (request for comments) process. These are the RFCs that were approved for implementation this week:

No RFCs were approved this week.

Final Comment Period

Every week the team announces the 'final comment period' for RFCs and key PRs which are reaching a decision. Express your opinions now. This week's FCPs are:

New RFCs Upcoming Events

If you are running a Rust event please add it to the calendar to get it mentioned here. Email the Rust Community Team for access.

Rust Jobs

Tweet us at @ThisWeekInRust to get your job offers listed here!

Quote of the Week

If C is like playing with knives and C++ is like juggling chainsaws then Rust is like parkour wearing protective gear while suspended from strings. It may look ridiculous at times, but you can do all sorts of awesome moves that would be damn scary or outright impossible without it.

u/llogiq on r/rust.

Thanks to Christopher Durham for the suggestion!

Submit your quotes for next week!

This Week in Rust is edited by: nasa42 and llogiq.

Categorieën: Mozilla-nl planet

Robert O'Callahan: Mixed Blessings Of Greenfield Software Development

Mozilla planet - sn, 30/12/2017 - 01:17

The biggest software project I have ever worked on, and hopefully ever will work on, was Gecko. I was not one of its original architects, so my work on Gecko was initially very tightly constrained by design decisions made by others. Over the years some of those decisions were rescinded, and some of the new decisions were mine, but I always felt frustrated at being locked into designs that I didn't choose and (with the benefit of hindsight, at least) would not have chosen. "Wouldn't it be great", I often thought, "to build something entirely new from scratch, hopefully getting things right, or at least having no-one other than myself to blame for my mistakes!" I guess a lot of programmers feel this, and that's why we see more project duplication than the world really needs.

I was lucky enough at Mozilla to work on a project that gave me a bit of an outlet for this — rr. I didn't actually do much coding in rr at the beginning — Albert Noll, Nimrod Partush and Chris Jones got it underway — but I did participate in the design decisions.

Over the last two years Kyle Huey and I have been working on a new project which is broader in scope and more complicated than rr. The two of us have designed everything (with some feedback from some wonderful people, who know who they are) and implemented everything. We've been able to make our design choices quite freely — still constrained by external realities (e.g. the vagaries of the x86 instruction set), but not by legacy code. It has been exhilarating.

However, with freedom comes responsibility. My decision-making has constantly been haunted by the fear of being "That Person" — the one whom, years from now, developers will curse as they work around and within That Person's mistakes. I've also come to realize that being unencumbered by legacy design decisions can make design more difficult because the space is so unconstrained. This is exacerbated by the kind of project we're undertaking: it's a first-of-a-kind, so there aren't patterns to follow, and our system has novel, powerful and flexible technical capabilities, so we can dream up all sorts of features and implement them in different ways. It's scary. We have to accept that mistakes will be made and just hope that none of them are crippling. After all, if developers are cursing our design decisions years from now, that means we succeeded!

One of our biggest challenges is constantly trying to strike the right balance between forward planning and expediency. It's always tempting to respond to uncertainty by planning for all possibilities, by building abstractions that make it easier to change decisions later. Another temptation is to obsessively study issues to reduce the uncertainty. All that represents delay for our minimum viable product, and therefore must be minimized. My general approach here is to try to think about the future, but not code for it; to tell stories about how we would handle possible future scenarios, but not invest in them yet.

I think we're doing OK. Being nearly two years in is long enough to have regrets, and we don't have many. It will be very interesting, and humbling, to review our scorecard over the longer term.

Categorieën: Mozilla-nl planet

Cameron Kaiser: TenFourFox FPR5b1 available

Mozilla planet - fr, 29/12/2017 - 05:51
TenFourFox Feature Parity Release 5 beta 1 is now available (downloads, hashes, release notes).

The biggest changes are lots and lots of AltiVec: additional AltiVec decoding acceleration for VP9 (you need MSE enabled to make the most of this, or things like YouTube will default to VP8), more usage of our VMX-accelerated string search package, and (new in this version) AltiVec-accelerated PNG image decoding. There are also a number of minor but notable improvements to layout compatibility, DOM and HTML5 support (most notably true passive event listeners, which should improve scroll and general event performance on some sites), as well as some additional efficiency improvements to JavaScript. FPR5 also marks mostly the end of all of the Quantum-related platform changes we can safely backport to our fork of 45; while there will be some additional optimization work, I will primarily be concentrating on actual new support rather than speedups since most of the low-hanging fruit and some of the stuff I have to jump a little for has already been plucked.

There are two somewhat more aggressive things in this version. The first is to even more heavily throttle inactive or background tabs to reduce their impact; the normal Firefox runs animation frames on background tabs at 1Hz but this version reduces that to a third (i.e., a tick every three seconds instead of one). It's possible to make this even more aggressive, including just not ticking any timers in background tabs at at all, but I'm a little nervous about halting it entirely. Neither approach affects audio playing in inactive or background tabs, which I tested thoroughly in Amazon Music and YouTube. This should make having multiple tabs open and loaded a bit less heavyweight, particularly on single processor Macs.

The second has to do with session store. Currently, every 25 seconds (Firefox has a default of 15), the browser serializes its state and writes it out to disk so you can pick up where you left off in the event of a crash. I'm loathe to completely halt this or make the interval much more than 60 seconds, but I also know that this does drag on the browser and can also spin SSD write cycles. This version increases it to 30 seconds, but also reduces the number of forward pages written (up to 5 instead of unlimited), and also purges old closed tabs more aggressively -- Firefox purges these after about two weeks, but we now purge old tabs every 24 hours, which I thought was a good compromise. This means much less data is written and much less cruft accumulates as you browse, reducing the browser's overhead and memory usage over longer uptimes. However, I know this may upset those of you who have tab closure regret, so advise if this drives you nuts even though I am unlikely to reverse this change completely.

This version is timed to come out with Firefox ESR 52.6.0, which is not scheduled for release until January 23, so don't panic if you don't see much commit activity in Github for awhile. 52.8.0, scheduled for May 7, will be the transition point to 60ESR. More on that when we get there.

For FPR6, I'm looking at a couple features to get us more HTML5 support points, and possibly something like date-time input controls or <details> and <summary> support. This may also be the first release with built-in adblock, though the adblock support will only be basic, will not be comprehensive, and may include blocking certain tracking scripts as well as image ads. It won't be enabled by default.

Party safe on New Years'.

Categorieën: Mozilla-nl planet

Oops! Mozilla's partnership with acclaimed TV series Mr Robot didn't go as ... - Techly

Nieuws verzameld via Google - fr, 29/12/2017 - 03:39

Techly

Oops! Mozilla's partnership with acclaimed TV series Mr Robot didn't go as ...
Techly
A recent partnership between Mozilla and the acclaimed television series Mr Robot intended to give users a fun Easter egg, but instead, it just freaked out the internet. Mozilla rolled out the mysterious extension called “Looking Glass 1.0.3 ...

Categorieën: Mozilla-nl planet

Mozilla Issues Critical Security Patch for Thunderbird Flaw - Dark Reading

Nieuws verzameld via Google - to, 28/12/2017 - 19:44

Mozilla Issues Critical Security Patch for Thunderbird Flaw
Dark Reading
The critical patch was one of five security bugs Mozilla fixed this month. Others include two vulnerabilities rated high, one moderate, and one low. Both of the highly rated security flaws affected the RSS feed. The moderate and low bugs affected RSS ...

Categorieën: Mozilla-nl planet

Don Marti: Predictions for 2018

Mozilla planet - to, 28/12/2017 - 09:00

Bitcoin to the moooon: The futures market is starting up, so here comes a bunch more day trader action. More important, think about all the bucket shops (I even saw an "invest in Bitcoin without owning Bitcoin" ad on public transit in London), legit financial firms, Libertarian true believers, and coins lost forever because of human error. Central bankers had better keep an eye on Bitcoin, though. Last recession we saw that printing money doesn't work as well as it used to, because it ends up in the hands of rich people who, instead of priming economic pumps with it, just drive up the prices of assets. I would predict "Entire Round of Quantitative Easing Gets Invested in Bitcoin Without Creating a Single New Job" but I'm saving that one for 2019. Central banks will need to innovate. Federal Reserve car crushers? Relieve medical deby by letting the UK operate NHS clinics at their consulates in the USA, and we trade them US green cards for visas that allow US citizens to get treated there? And—this is a brilliant quality of Bitcoin that I recognized too late—there is no bad news that could credibly hurt the value of a purely speculative asset.

The lesson for regular people here is not so much what to do with Bitcoin, but remember to keep putting some well-considered time into actions that you predict have unlikely but large and favorable outcomes. Must remember to do more of this.

High-profile Bitcoin kidnapping in the USA ends in tragedy: Kidnappers underestimate the amount of Bitcoin actually available to change hands, ask for more than the victim's family (or fans? a crowdsourced kidnapping of a celebrity is now a possibility) can raise in time. Huge news but not big enough to slow down something that the finance scene has already committed to.

Tech industry reputation problems hit open source. California Internet douchebags talk like a positive social movement but act like East Coast vampire squid—and people are finally not so much letting them define the terms of the conversation. The real Internet economy is moving to a three-class system: plutocrats, well-paid brogrammers with Aeron chairs, free snacks and good health insurance, and everyone else in the algorithmically-managed precariat. So far, people are more concerned about the big social and surveillance marketing companies, but open source has some of the same issues. Just as it was widely considered silly for people to call Facebook users "the Facebook community" in 2017, some of the "community" talk about open source will be questioned in 2018. Who's working for who, and who's vulnerable to the risks of doing work that someone else extracts the value of? College athletes are ahead of the open source scene on this one.

Adfraud becomes a significant problem for end users: Powerful botnets in data centers drove the pivot to video. Now that video adfraud is well-known, more of the fraud hackers will move to attribution fraud. This ties in to adtech consolidation, too. Google is better at beating simple to midrange fraud than the rest of the Lumascape, so the steady progress towards a two-logo Lumascape means fewer opportunities for bots in data centers.

Attribution fraud is nastier than servers-talking-to-servers fraud, since it usually depends on having fraudulent and legit client software on the same system—legit to be used for a human purchase, fraudulent to "serve the ad" that takes credit for it. Unlike botnets that can run in data centers, attribution fraud comes home with you. Yeech. Browsers and privacy tools will need to level up from blocking relatively simple Lumascape trackers to blocking cleverer, more aggressive attribution fraud scripts.

Wannabe fascists keep control of the US Congress, because your Marketing budget: "Dark" social campaigns (both ads and fake "organic" activity) are still a thing. In the USA, voter suppression and gerrymandering have been cleverly enough done that social manipulation can still make a difference, and it will.

In the long run, dark social will get filtered out by habits, technology, norms, and regulation—like junk fax and email spam before it—but we don't have a "long run" between now and November 2018. The only people who could make an impact on dark social now are the legit advertisers who don't want their brands associated with this stuff. And right now the expectations to advertise on the major social sites are stronger than anybody's ability to get an edgy, controversial "let's not SPONSOR ACTUAL F-----G NAZIS" plan through the 2018 marketing budget process.

Yes, the idea of not spending marketing money on supporting nationalist extremist forums is new and different now. What a year.

Bonus links

These Publishers Bought Millions Of Website Visits They Later Found Out Were Fraudulent

No boundaries for user identities: Web trackers exploit browser login managers

Best of 2017 #8: The World's Most Expensive Clown Show

My Internet Mea Culpa

2017 Was the Year I Learned About My White Privilege

With the people, not just of the people

When Will Facebook Take Hate Seriously?

Using Headless Mode in Firefox – Mozilla Hacks : the Web developer blog

Why Chuck E. Cheese’s Has a Corporate Policy About Destroying Its Mascot’s Head

Dozens of Companies Are Using Facebook to Exclude Older Workers From Job Ads

How Facebook’s Political Unit Enables the Dark Art of Digital Propaganda

Categorieën: Mozilla-nl planet

The Firefox Frontier: Firefox Extensions for New Year’s Resolutions

Mozilla planet - wo, 27/12/2017 - 15:00

It’s that time of year again where we endeavor to improve ourselves, to wash away poor habits of the past and improve our lot in life. Yet most of us … Read more

The post Firefox Extensions for New Year’s Resolutions appeared first on The Firefox Frontier.

Categorieën: Mozilla-nl planet

Thunderbird-gaten maken mailvervalsing mogelijk - AG Connect - AG Connect

Nieuws verzameld via Google - wo, 27/12/2017 - 13:58

Thunderbird-gaten maken mailvervalsing mogelijk - AG Connect
AG Connect
De open source stichting Mozilla heeft net voor de kerst zijn e-mailprogramma Thunderbird ge-update. De nieuwe versie 52.5.2 dicht diverse gaten, waaronder enkele kritieke. Naast een bug die JavaScript ongecontroleerd liet draaien, is er een bug die ...

en meer »
Categorieën: Mozilla-nl planet

Mozilla Patches Critical Bug in Thunderbird | Threatpost | The first ... - Threatpost

Nieuws verzameld via Google - ti, 26/12/2017 - 20:12

Threatpost

Mozilla Patches Critical Bug in Thunderbird | Threatpost | The first ...
Threatpost
Mozilla issued a critical security update to its popular open-source Thunderbird email client. The patch was part of a December release of five fixes that included two bugs rated high and one rated moderate and another low. Mozilla said Thunderbird ...
Mozilla patches one critical, two high flaws in ThunderbirdSC Magazine

alle 2 nieuwsartikelen »
Categorieën: Mozilla-nl planet

Mozilla dicht spoofinglek in e-mailclient Thunderbird - Security.nl

Nieuws verzameld via Google - ti, 26/12/2017 - 09:04

Mozilla dicht spoofinglek in e-mailclient Thunderbird
Security.nl
Mozilla heeft een beveiligingsupdate uitgebracht die meerdere kwetsbaarheden in de e-mailclient Thunderbird verhelpt, waaronder een lek dat het mogelijk maakt om de afzender te spoofen. In Thunderbird 52.5.2 zijn in totaal vijf kwetsbaarheden gepatcht ...

Categorieën: Mozilla-nl planet

This Week In Rust: This Week in Rust 214

Mozilla planet - ti, 26/12/2017 - 06:00

Hello and welcome to another issue of This Week in Rust! Rust is a systems language pursuing the trifecta: safety, concurrency, and speed. This is a weekly summary of its progress and community. Want something mentioned? Tweet us at @ThisWeekInRust or send us a pull request. Want to get involved? We love contributions.

This Week in Rust is openly developed on GitHub. If you find any errors in this week's issue, please submit a PR.

Updates from Rust Community News & Blog Posts Crate of the Week

This week's crate is crossbeam-channel, a crate that improves multi-producer multi-consumer channels compared to what the standard library offers. Thanks to leodasvacas for the suggestion!

Submit your suggestions and votes for next week!

Call for Participation

Always wanted to contribute to open-source projects but didn't know where to start? Every week we highlight some tasks from the Rust community for you to pick and get started!

Some of these tasks may also have mentors available, visit the task page for more information.

If you are a Rust project owner and are looking for contributors, please submit tasks here.

Updates from Rust Core

118 pull requests were merged in the last week

New Contributors
  • Antal Szabó
  • Christopher Durham
  • Ed Schouten
  • Florian Keller
  • Jonas Platte
  • Matti Niemenmaa
  • Sam Green
  • Scott Abbey
  • Wilco Kusee
Approved RFCs

Changes to Rust follow the Rust RFC (request for comments) process. These are the RFCs that were approved for implementation this week:

No RFCs were approved this week.

Final Comment Period

Every week the team announces the 'final comment period' for RFCs and key PRs which are reaching a decision. Express your opinions now. This week's FCPs are:

New RFCs Upcoming Events

If you are running a Rust event please add it to the calendar to get it mentioned here. Email the Rust Community Team for access.

Rust Jobs

Tweet us at @ThisWeekInRust to get your job offers listed here!

Quote of the Week

Every great language needs a Steve.

aaron-lebo on Hacker News about @steveklabnik.

Thanks to Aleksey Kladov for the suggestion!

Submit your quotes for next week!

This Week in Rust is edited by: nasa42 and llogiq.

Categorieën: Mozilla-nl planet

Dustin J. Mitchell: FRustrations 1

Mozilla planet - mo, 25/12/2017 - 16:00
Foreword

I’ve been hacking about learing Rust for a bit more than a year now, building a Hawk crate and hacking on a distributed lock service named Rubbish (which will never amount to anything but gives me a purpose).

In the process, I’ve run into some limits of the language. I’m going to describe some of those in a series of posts starting with this one.

One of the general themes I’ve noticed is lots of things work great in demos, where everything is in a single function (thus allowing lots of type inference) and most variables are 'static. Try to elaborate these demos out into a working application, and the borrow checker immediately blocks your path.

Today’s frustration is a good example.

Ownership As Exclusion and Async Rust

A common pattern in Rust is to take ownership of a resource as a way of excluding other uses until the operation is finished. In particular, the send method of Sinks in the futures crate takes self, not &self. Because this is an async function, its result is a future that will yield the Sink on success.

The safety guarantee here is that only one async send can be performed at any given time. The language guarantees that nothing else can access the Sink until the send is complete.

Building a Chat Application

As part of learning about Tokio, Futures, and so on, I elected to build a chat application, starting with the simple pipelined server in the Tokio docs. This example uses Stream::and_then to map requests to responses, which makes sense for a strict request/response protocol, but does not make sense for a chat protocol. It should be possible to send or receive a message at any time in a chat protocol, so I modified the example to use send to send one message at a time:

let server = connections.for_each(move |(socket, _peer_addr)| { let (writer, _reader) = socket.framed(LineCodec).split(); let server = writer.send("Hello, World!".to_string()) .and_then(|writer2| writer2.send("Welcome to Chat.".to_string()) .then(|_| Ok(())); handle.spawn(server); Ok(()) });

This bit works fine: the resulting server greets each user, then drops the socket and disconnects them, as expected. Note the threading of the writer: the first writer.send is using the writer returned from split, while the second is using the result of the Future from the first (I have unecessarily called it writer2 here for clarity). In fact, send moves self, so writer.send("Welcome to Chat".to_string()) would not be permitted as that value has been moved.

Based on how I would design a chat app in Python, JavaScript, or any other language, I chose to make a struct to represent a connected user in the chat:

pub struct ChatConnection { reader: SplitStream<Framed<TcpStream, LineCodec>>, writer: SplitSink<Framed<TcpStream, LineCodec>>, peer: SocketAddr, } impl ChatConnection { fn new(socket: TcpStream, peer: SocketAddr) -> ChatConnection { let (writer, reader) = socket.framed(LineCodec).split(); ChatConnection { writer: writer, reader: reader, peer: peer, } } fn run(&self) -> Box<Future<Item = (), Error = ()>> { Box::new(self.writer .send("Welcome to Chat!".to_string()) .then(|_| Ok(()))) } }

When a new connection arrives, other code allocates a new ChatConnection and calls its run method, spawning a task into the event loop with the resulting future.

This doesn’t work, though:

error[E0507]: cannot move out of borrowed content --> src/main.rs:82:18 | | Box::new(self.writer | ^^^^ cannot move out of borrowed content

There’s sense in this: the language is preventing multiple simultaneous sends. If it allowed self.writer to be accessible to other code while the Future was not complete, then that other code could potentially, unsafely, call send again.

But it makes it difficult to store the writer in a struct – something any reasonably complex application is going to need to do. The two Rust-approved solutions here are to always move writer around as a local variable (as done in the demo), or to move self in the run method (fn run(self) ..). The first “hides” writer in a thicket of closures, making it difficult or impossible to find when, for example, another user sends this one a private message. The second just moves the problem: now we have a ChatConnection object to which nothing but the run method is allowed to refer, meaning that nothing can communicate with it.

The Fix

The most obvious fix is to wrap the writer in another layer of abstraction with runtime safety guarantees. This means something like a Mutex, although the Mutex class will block a thread on conflict, which will result in deadlock in a single-threaded, asynchronous situation such as this one. I assume there is some Futures equivalent to Mutex that will return a Future<Item = Guard> which resolves when the underlying resource is available.

Looking at some of the existing chat examples, I see that they use futures::sync::mpsc channels to communciate between connections. This is in keeping with the stream/sink model (channels are just another form of a stream), but replacing the Future-yielding send with the non-blocking (but memory-unbounded) unbounded_send method.

Frustration

I feel like this solution is “cheating”: the language makes it difficult to send messages on the channel it provides, so wrap it in anohter channel with better semantics. Even the code to connect those two channels is, to my eye, obfuscating this issue:

let socket_writer = rx.fold(writer, |writer, msg| { let amt = io::write_all(writer, msg.into_bytes()); let amt = amt.map(|(writer, _)| writer); amt.map_err(|_| ()) });

That rx.fold function is doing a lot of work, but there is nary a comment to draw attention to this fact. Those accustomed to functional programming, and familiar with Rust’s use of the term “fold” for what most languages call “reduce”, might figure out what’s going on more quickly. The writer is moved into the accumulator for the fold (reduce) operation, then moved into the closure argument, and when the future is finished it is moved back into the accumulator for the next iteration. This is a clever application of the first Rust-approved solution above: move the writer around in local variables without ever landing it in a stable storage location.

So, this is a key characteristic of asynchronous Rust, without which programs will not compile. Yet these “examples”, which are meant to be instructive, bury the approach behind some clever stream combinators as if they are ashamed of it. The result is almost immediate frustration and confusion for the newcomer to asynchronous Rust trying to learn from these examples.

Categorieën: Mozilla-nl planet

Manish Goregaokar: Undefined vs Unsafe in Rust

Mozilla planet - snein, 24/12/2017 - 01:00

Recently Julia Evans wrote an excellent post about debugging a segfault in Rust. (Go read it, it’s good)

One thing it mentioned was

I think “undefined” and “unsafe” are considered to be synonyms.

This is … incorrect. However, we in the Rust community have never really explicitly outlined the distinction, so that confusion is on us! This blog post is an attempt to clarify the difference of terminology as used within the Rust community. It’s a very useful but subtle distinction and I feel we’d be able to talk about safety more expressively if this was well known.

Unsafe means two things in Rust, yay

So, first off, the waters are a bit muddied by the fact that Rust uses unsafe to both mean “within an unsafe {} block” block and “something Bad is happening here”. It’s possible to have safe code within an unsafe block; indeed this is the primary function of an unsafe block. Somewhat counterintutively, the unsafe block’s purpose is to actually tell the compiler “I know you don’t like this code but trust me, it’s safe!” (where “safe” is the negation of the second meaning of “unsafe”, i.e. “something Bad is not happening here”).

Similarly, we use “safe code” to mean “code not using unsafe{} blocks” but also “code that is not unsafe”, i.e. “code where nothing bad happens”.

This blog post is primarily about the “something bad is happening here” meaning of “unsafe”. When referring to the other kind I’ll specifically say “code within unsafe blocks” or something like that.

Undefined behavior

In languages like C, C++, and Rust, undefined behavior is when you reach a point where the compiler is allowed to do anything with your code. This is distinct from implementation-defined behavior, where usually a given compiler/library will do a deterministic thing, however they have some freedom from the spec in deciding what that thing is.

Undefined behavior can be pretty scary. This is usually because in practice it causes problems when the compiler assumes “X won’t happen because it is undefined behavior”, and X ends up happening, breaking the assumptions. In some cases this does nothing dangerous, but often the compiler will end up doing wacky things to your code. Dereferencing a null pointer will sometimes cause segfaults (which is the compiler generating code that actually dereferences the pointer, making the kernel complain), but sometimes it will be optimized in a way that assumes it won’t and moves around code such that you have major problems.

Undefined behavior is a global property, based on how your code is used. The following function in C++ or Rust may or may not exhibit undefined behavior, based on how it gets used:

int deref(int* x) { return *x; } // do not try this at home fn deref(x: *mut u32) -> u32 { unsafe { *x } }

As long as you always call it with a valid pointer to an integer, there is no undefined behavior involved.

But in either language, if you use it with some pointer conjured out of thin air (or, like 0x01), that’s probably undefined behavior.

As it stands, UB is a property of the entire program and its execution. Sometimes you may have snippets of code that will always exhibit undefined behavior regardless of how they are called, but in general UB is a global property.

Unsafe behavior

Rust’s concept of “unsafe behavior” (I’m coining this term because “unsafety” and “unsafe code” can be a bit confusing) is far more scoped. Here, fn deref is “unsafe”1, even if you always call it with a valid pointer. The reason it is still unsafe is because it’s possible to trigger UB by only changing the “safe” caller code. I.e. “changes to code outside unsafe blocks can trigger UB if they include calls to this function”.

Basically, in Rust a bit of code is “safe” if it cannot exhibit undefined behavior under all circumstances of that code being used. The following code exhibits “safe behavior”:

unsafe { let x = 1; let raw = &x as *const u32; println!("{}", *raw); }

We dereferenced a raw pointer, but we knew it was valid. Of course, actual unsafe blocks will usually be “actually totally safe” for less obvious reasons, and part of this is because unsafe blocks sometimes can pollute the entire module.

Basically, “safe” in Rust is a more local property. Code isn’t safe just because you only use it in a way that doesn’t trigger UB, it is safe because there is literally no way to use it such that it will do so. No way to do so without using unsafe blocks, that is2.

This is a distinction that’s possible to draw in Rust because it gives us the ability to compartmentalize safety. Trying to apply this definition to C++ is problematic; you can ask “is std::unique_ptr<T> safe?”, but you can always use it within code in a way that you trigger undefined behavior, because C++ does not have the tools for compartmentalizing safety. The distinction between “code which doesn’t need to worry about safety” and “code which does need to worry about safety” exists in Rust in the form of “code outside of unsafe {}” and “code within unsafe {}”, whereas in C++ it’s a lot fuzzier and based on expectations (and documentation/the spec).

So C++’s std::unique_ptr<T> is “safe” in the sense that it does what you expect but if you use it in a way counter to how it’s supposed to be used (constructing one from an invalid pointer, for example) it can blow up. This is still a useful sense of safety, and is how one regularly reasons about safety in C++. However it’s not the same sense of the term as used in Rust, which can be a bit more formal about what the expectations actually are.

So unsafe in Rust is a strictly more general concept – all code exhibiting undefined behavior in Rust is also “unsafe”, however not all “unsafe” code in Rust exhibits undefined behavior as written in the current program.

Rust furthermore attempts to guarantee that you will not trigger undefined behavior if you do not use unsafe {} blocks. This of course depends on the correctness of the compiler (it has bugs) and of the libraries you use (they may also have bugs) but this compartmentalization gets you most of the way there in having UB-free programs.

  1. Once again in we have a slight difference between an “unsafe fn”, i.e. a function that needs an unsafe block to call and probably is unsafe, and an “unsafe function”, a function that exhibits unsafe behavior.

  2. This caveat and the confusing dual-usage of the term “safe” lead to the rather tautological-sounding sentence “Safe Rust code is Rust code that cannot cause undefined behavior when used in safe Rust code”

Categorieën: Mozilla-nl planet

Botond Ballo: Control Flow Visualizer (CFViz): an rr / gdb plugin

Mozilla planet - fr, 22/12/2017 - 19:59

rr (short for “record and replay”) is a very powerful debugging tool for C++ programs, or programs written in other compiled languages like Rust1. It’s essentially a reverse debugger, which allows you to record the execution of a program, and then replay it in the debugger, moving forwards or backwards in the replay.

I’ve been using rr for Firefox development at Mozilla, and have found it to be enormously useful.

One task that comes up very often while debugging is figuring out why a function produced a particular value. In rr, this is often done by going back to the beginning of the function, and then stepping through it line by line.

This can be tedious, particularly for long functions. To help automate this task, I wrote – in collaboration with my friend Derek Berger, who is learning Rust – a small rr plugin called Control Flow Visualizer, or CFViz for short.

To illustrate CFViz, consider this example function foo() and a call site for it:

example code

With the CFViz plugin loaded into rr, if you invoke the command cfviz while broken anywhere in the call to foo() during a replay, you get the following output:

example output

Basically, the plugin illustrates what path control flow took through the function, by coloring each line of code based on whether and how often it was executed. This way, you can tell at a glance things like:

  • which of several return statements produced the function’s return value
  • which conditional branches were taken during the execution
  • which loops inside the function are hot (were executed many times)

saving you the trouble of having to step through the function to determine this information.

CFViz’s implementation strategy is simple: it uses gdb’s Python API to step through the function of interest and see which lines were executed in what order. In then passes that information to a small Rust program which handles the formatting and colorization of the output.

While designed with rr in mind, CFViz also works with vanilla gdb, with the limitation that it will only visualize the rest of the function’s execution from the point where it was invoked (since, without rr, it cannot go backwards to the function’s starting point).

I’ve found CFViz to be quite useful for debugging Firefox’s C++ code. Hope you find it useful too!

CFViz is open source. Bug reports, patches, and other contributions are welcome!

Footnotes

1. rr also has a few important limitations: it only runs on Intel CPUs, and only on Linux (although there is a similar tool called Time-Travel Debugging for Windows)


Categorieën: Mozilla-nl planet

Firefox Test Pilot: Graduation Report: Activity Stream

Mozilla planet - fr, 22/12/2017 - 18:01

Activity Stream launched as one of the first Test Pilot experiments. Our goal with Activity Stream from the beginning has been to create new ways for Firefox users to interact with and benefit from their history and bookmarks. Web browsers have historically kept this valuable information tucked away, limiting its usefulness. We were inspired by some smart features in Firefox, like the Awesome Bar, and wanted to bring that kind of intelligence to the rest of the browser.

<figcaption>SVP of Firefox Mark Mayo at the Mozilla All Hands in December 2016</figcaption>

We believed that if people could easily get back to the pages they had recently viewed and saved, they would be happier and more productive. We wanted to help people rediscover where they had been and help them decide where to go next.

Here’s what we learned

Our first attempt at this included two new features in Firefox: a new New Tab page and a Library view to see all of your bookmarks and history ordered from newest to oldest.

<figcaption>First version of Activity Stream on New Tab and the Library view</figcaption>

While we we were equally excited about the possibilities of both of these features, we found very quickly that people spent much more time interacting with New Tab. We decided that splitting our efforts on these two features wasn’t the best way to positively impact most people in Firefox and made the decision to retire the Library.

The good news is that this gave us more time to focus on New Tab. The first version included 4 major sections: Search, Top Sites, Spotlight (later renamed Highlights), and Top Activity. Each of these sections changed and morphed as we collected feedback through surveys, A/B tests, and user interviews.

<figcaption>Snapshot of the many experiments that we ran for each version of Activity Stream</figcaption>Search

Up first was Search, which might be the most obvious. Or maybe it wasn’t. When we asked people what this search box did, many answered that it would search their history and bookmarks. That’s a pretty good guess considering the other items that are on the page. The problem is that it actually searches the web using your default search engine (Google or Bing for example). Because of this feedback, we changed the label of the search box to say “Search the Web”. This seemed to clear things up for most people.

<figcaption>Awesome Bar search box, toolbar search box, New Tab search box, oh my!</figcaption>

One of the most surprising things that we learned while running this experiment is that around thirty percent of New Tab interactions were with that search box. You might wonder why that’s so surprising, but if you look at Firefox closely, you’ll notice that there are actually two other search boxes above this one: the Awesome Bar in the top left and the Search box in the top right. We believe that the New Tab search box is this popular because it’s in the content of the page and reminds people of the familiar search box from their favorite search engine.

Top Sites

After the search box, we have the ever popular Top Sites which are, well, your top… sites. To be more specific the sites (or pages) that show up here are ones that you have visited both frequently and fairly recently. This is the same technology that powers the Awesome Bar in Firefox, and it’s called frecency. Basically it’s good at guessing which sites you might want to visit based on your recent browsing. We made some minor changes to the algorithm that powers Top Sites but the bigger changes that we made were visual.

The Top Sites tiles in previous versions of Firefox used large screenshots of the sites you visited. We wanted something that was both more compact and easier to recognize, given the other items that were on the page, and decided to use icons to represent each site.

<figcaption>Top Sites in previous versions of Firefox were a large grid of screenshots</figcaption>

This seemed like a pretty obvious solution that would mirror the app launchers that people were familiar with on both their phones and laptops. The problem was that it wasn’t all that obvious in the end. Many sites had poor quality icons that were very small. This made it difficult for people to recognize their favorite sites.

<figcaption>The first version of Top Sites had smaller icons with a matching background color</figcaption>

We addressed this by creating our own collection of high-quality icons. Unfortunately for our icon artists, there are an endless number of sites on the web and therefore too many icons for us to hand curate. The other problem with icons is that they’re great for home pages but not so good for specific sections or pages on a site. So you might see a Reddit or CNN icon that looks like the home page when it was actually a specific page on the site.

This made it difficult to guess where an icon might take you. In the end, we settled on the best of both worlds. For home pages with a nice icon, we give you that in all its glory. For sections of a site or where a large icon isn’t available, we combine the small icon with a screenshot to give you some extra hints about which page you’ll land on.

<figcaption>The final version has a large icon when available and otherwise a small icon with screenshot</figcaption>Highlights… or was it Spotlight?

Next up on New Tab was the ever changing Spotlight section. The name Spotlight didn’t last for too long thanks to another feature with that same name in a certain popular (mac)OS. We settled on the name Highlights as a replacement even though to this day we worry that it isn’t quite right. We’ve debated the name several times since but always end up back at Highlights. The original idea for this section is that it would be the “highlights” of your recent activity in the browser that you would see in the more expansive Library view.

<figcaption>Earlier version of Highlights with different kinds of content mixed together</figcaption>

We actually spent a lot of time iterating on this section. Our goal was to provide a similar feature to Top Sites but in reverse. Rather than showing you the things you visited most, we wanted to show you the things you had just discovered and might want to get back to again. Ideally these would be things you might have bookmarked had you thought of it.

We ended up with a fairly sophisticated system where Firefox would assign each of your recently visited pages and bookmarks a score, and it would show you the items with the highest score each time you opened a new tab. We gave bookmarks more points since you had told us they were important and that way they would hang around and be available to you for a little bit longer.

<figcaption>We ran many experiments on Highlights. Not all of them were as conclusive as we hoped.</figcaption>

For many of us on the team, this was a really great feature that we loved using. Unfortunately, when interviewing users, especially those using New Tab for the first time, they found Highlights to be confusing. They didn’t understand why items weren’t in chronological order (thanks to the scoring system) and when the section was empty, they didn’t know what to expect.

We made a number of changes to address these concerns. We went back to a simpler version of Highlights that is mostly chronological with bookmarks showing up first. We also added little ? bubbles to explain the different sections and give users quick access to customization. Finally, we added message boxes to explain the sections when they were empty.

<figcaption>We added message boxes to explain sections that were empty</figcaption>Top Activity

Last (and maybe least) we had Top Activity at the bottom of the page. Somewhat like Highlights, Top Activity was meant to be some of the most interesting things from your recent history. In reality, it was just the first few items from the Library view.

<figcaption>Top Activity showed the most recent entries from history</figcaption>

This actually turned out to be a more effective feature than we had anticipated. We had a lot of positive feedback about easy access to the most recently visited pages. We did soon realize though that Top Activity and Highlights were remarkably similar features and decided to combine them. Through a few different iterations we ended up with the 9 cards you are familiar with in Highlights today.

Customization

Something that became clear through much of our testing is that people wanted to customize their New Tab. We found ourselves wanting the same thing in different ways. Some people wanted two rows of Top Sites. Others wanted to remove the search box and still others wanted to choose between just history or bookmarks in Highlights. So we added a whole slew of customization options to a nice side panel where it’s easy to see what your New Tab will look like as you make changes.

<figcaption>Preferences let you customize the sections on New Tab</figcaption>Recommended by Pocket

So those are all the sections right? Well almost! Last year we tested some content recommendations with our good friends at Pocket. We had some mixed results back then and some technical challenges that kept us from doing additional tests. Since that time though, Mozilla acquired Pocket, and we’re now part of the same company! This made it even easier to run experiments together and so we did. Recently we shipped the latest version of this feature called Recommended by Pocket, which helps you find the most interesting articles from around the web.

<figcaption>Pocket recommendations help you find the most interesting articles from around the web</figcaption>

We rolled this out as a test so that not everyone received this feature to begin with. We compared how much people used New Tab with and without this feature, and we were excited to find that people used New Tab more when this was enabled.

<figcaption>Percentage of New Tab page views where a user clicked on something on the page</figcaption>

These results gave us the confidence to ship Pocket recommendations in a number of key countries including the United States, Canada, and Germany.

Conclusion

All of these lessons and iterations came together into the really great New Tab experience that we have today:

Many of the details are different but most of the big ideas are very much the same. We have stayed focused on helping people connect to the places they’ve been and hope to help them find where they might want to go next. We could not have done any of this without the amazing help, feedback, and patience of you, our loyal test pilots. Thank you so much for joining us on this journey!

Here’s what happens next

The exciting news is that we shipped this feature as part of Firefox Quantum! We continue to learn and iterate and look forward to making these features even better for all of our Firefox users.

Thank you again for your help, and we encourage you to participate in helping other Test Pilot experiments learn and grow the same way that we’ve done with this one.

Graduation Report: Activity Stream was originally published in Firefox Test Pilot on Medium, where people are continuing the conversation by highlighting and responding to this story.

Categorieën: Mozilla-nl planet

Mozilla Open Innovation Team: Open Innovation for Inclusion

Mozilla planet - fr, 22/12/2017 - 03:07

In the second half of 2017 Mozilla’s Open Innovation team worked on initiatives aimed at exploring new ways to advance Web accessibility. Through a decentralized design sprint we arrived at a number of early concepts which could help enhance accessibility of Web content in Firefox.

Designing with the Crowd

We partnered with Stanford University for a user-centric open design sprint. Technology is permeating most human interactions, but we still have very centralized design processes, that only include few people. We wanted to experiment with an open innovation approach that would allow users with accessibility needs to take an active part in the design process. Our chosen path to tackle this challenge allowed for a collaborative form of crowdsourcing. Instead of relying on individual work, we got our participants to work in teams across countries, time zones and professional expertise.
The design sprint ran for one week and 113 participants that signed up online joined the slack channel we used to coordinate interaction. We had a very diverse group of people in terms of their background, their geography, gender and age.

In fact, 42% of our participants either have disabilities themselves or take care for someone in need. Participants from this group were essential for the sprint outcomes as they brought direct experiences to the design process, which inspired other participants with expertise in design and coding.

We narrowed down the problem space by focusing on three specific user groups: elderly people, people with severely limited gestures and people with cognitive impairments.

Winning Ideas from our Decentralized Design Sprint

The sprint resulted in over 60 early stage ideas for how to make browsing with Firefox more accessible. From those ideas Mozilla’s Test Pilot and our Accessibility team chose the 5 that best fulfilled the overarching criteria of the sprint: ideas that showed an understanding of the user needs, demonstrated empathy, were unique, addressed a real problem, had real-world applicability and applicability beyond accessibility needs.

The winning ideas are:

  • Verbose Mode: A voice that guides users in the browsing experience. From: Casey Rigby, Brian Hochhalter, Chandan Baba, Theresa Anderson, Daniel Alexander
  • Onboarding for all abilities: Including new Firefox users from the first interaction. From: Jason Przewoznik, Rahul Sawn, Sohan Subhash, Drashti Kaushik
  • Numbered commands: Helping navigate voice commands. From: Angela, Mary Brennan, Sherry Feng, smcharg
  • Browser based breadcrumbs: Help users understand where they are in the web. From: Phil Daquila, Sherry Stanley, Anne Zbitnew, Ilene E
  • Color the web for readability: Control the colors of websites to match your readability preferences. From: Bri Norton, Jessica Fung, Kristing Maughan, Parisa Nikzap, Neil McCaffrey, Tiffany Chen

Our Test Pilot and Accessibility teams have drawn a lot of inspiration from the ideas that came from the participants and will include some of the ideas in their product development explorations. Of particular interest were ideas designed around voice as user interface. As our Machine Learning Group at Mozilla invests research and development resources in the still young field of speech recognition we want to encourage further iterations on how to make the technology applicable to address accessibility needs. If you want to join the slack channel and continue iterating on these ideas please contact us at firefoxaccessibility@cs.stanford.edu. We are committed to keep this process open for everyone.

We’d like to thank all participants which have contributed to this initiative and we invite you to continue following us on our next steps.

If you’d like to learn more about a second accessibility design initiative we ran together with the Test Pilot team and students of the Design and Media School Ravensbourne in London, check out the Test Pilot blog.

Open Innovation for Inclusion was originally published in Mozilla Open Innovation on Medium, where people are continuing the conversation by highlighting and responding to this story.

Categorieën: Mozilla-nl planet

Pages