mozilla

Mozilla Nederland LogoDe Nederlandse
Mozilla-gemeenschap

Ανεβαίνουν κατακόρυφα οι επιδόσεις του Mozilla στα games (vid) - gazzetta.gr

Nieuws verzameld via Google - di, 14/03/2017 - 03:11

gazzetta.gr

Ανεβαίνουν κατακόρυφα οι επιδόσεις του Mozilla στα games (vid)
gazzetta.gr
Σε αυτό το θέμα έρχεται να δώσει τέλος o Mozilla καθώς πλέον είναι διαθέσιμη μια έκδοσή του για PC, Mac και Linux που προσφέρει “μοντέρνα” γραφικά στον browser σας χρησιμοποιώντας μια νέα τεχνολογία με την ονομασία WebAssembly. Μπορείτε να πάρετε ...

Categorieën: Mozilla-nl planet

Mitchell Baker: The “Worldview” of Mozilla

Mozilla planet - ma, 13/03/2017 - 21:59

There are a set of topics that are important to Mozilla and to what we stand for in the world — healthy communities, global communities, multiculturalism, diversity, tolerance, inclusion, empathy, collaboration, technology for shared good and social benefit.  I spoke about them at the Mozilla All Hands in December, if you want to (re)listen to the talk you can find it here.  The sections where I talk about these things are at the beginning, and also starting at about the 14:30 minute mark.

These topics are a key aspect of Mozilla’s worldview.  However, we have not set them out officially as part of who we are, what we stand for and how we describe ourselves publicly.   I’m feeling a deep need to do so.

My goal is to develop a small set of principles about these aspects of Mozilla’s worldview. We have clear principles that Mozilla stands for topics such as security and free and open source software (principles 4 and 7 of the Manifesto).  Similarly clear principles about topic such as global communities and multiculturalism will serve us well as we go forward.  They will also give us guidance as to the scope and public voice of Mozilla, spanning official communications from Mozilla, to the unofficial ways each of us describes Mozilla.

Currently, I’m working on a first draft of the principles.  We are working quickly, as quickly as we can have rich discussions and community-wide participation. If you would like to be involved and can potentially spend some hours reviewing and providing input please sign up here. Jascha and Jane are supporting me in managing this important project.  
I’ll provide updates as we go forward.  

Categorieën: Mozilla-nl planet

Chris Cooper: Shameless self (release) promotion: Firefox 53.0b1 from TaskCluster

Mozilla planet - ma, 13/03/2017 - 20:17

You may recall two short months ago when we moved Linux and Android nightlies from buildbot to TaskCluster. Due to the train model, this put us (release engineering) on a clock: either we’d be ready to release a beta version of Firefox 53 for Linux and Android using release promotion in TaskCluster, or we’d need to hold back our work for at least the next cycle, causing uplift headaches galore.

I’m happy to report that we were able to successfully release Firefox 53.0b1 for Linux and Android from TaskCluster last week. This is impressive for 3 reasons:

  1. Mac and Windows builds were still promoted from buildbot, so we were able to seamlessly integrate the artifacts of two different continuous integration (CI) platforms.
  2. The process whereby nightly builds are generated has always been different from how we generate release builds. Firefox 53.0b1 represents the first time a beta build was generated using the same taskgraph we use for a nightly, thereby reducing the delta between CI builds and release builds. More work to be done here, for sure.
  3. Nobody noticed. With all the changes under the hood, this may be the most impressive achievement of all.

A round of thanks to Aki, Johan, Kim, and Mihai who worked hard to get the pieces in place for Android, and a special shout-out to Rail who handled the Linux beta while also dealing with the uplift requirements for ESR52. Of course, thanks to everyone else who has helped with the migration thus far. All of that foundational work is starting to pay off.

Much more to do, but I look forward to updating you about Mac and Windows progress soon.

Categorieën: Mozilla-nl planet

The “Worldview” of Mozilla

Mitchell Baker - ma, 13/03/2017 - 19:28

There are a set of topics that are important to Mozilla and to what we stand for in the world — healthy communities, global communities, multiculturalism, diversity, tolerance, inclusion, empathy, collaboration, technology for shared good and social benefit.  I spoke about them at the Mozilla All Hands in December, if you want to (re)listen to the talk you can find it here.  The sections where I talk about these things are at the beginning, and also starting at about the 14:30 minute mark.

These topics are a key aspect of Mozilla’s worldview.  However, we have not set them out officially as part of who we are, what we stand for and how we describe ourselves publicly.   I’m feeling a deep need to do so.

My goal is to develop a small set of principles about these aspects of Mozilla’s worldview. We have clear principles that Mozilla stands for topics such as security and free and open source software (principles 4 and 7 of the Manifesto).  Similarly clear principles about topic such as global communities and multiculturalism will serve us well as we go forward.  They will also give us guidance as to the scope and public voice of Mozilla, spanning official communications from Mozilla, to the unofficial ways each of us describes Mozilla.

Currently, I’m working on a first draft of the principles.  We are working quickly, as quickly as we can have rich discussions and community-wide participation. If you would like to be involved and can potentially spend some hours reviewing and providing input please sign up here. Jascha and Jane are supporting me in managing this important project.  
I’ll provide updates as we go forward.  

Categorieën: Mozilla-nl planet

Advertiser tracking concerns second only to hacking for web users ... - The Drum

Nieuws verzameld via Google - ma, 13/03/2017 - 19:19

The Drum

Advertiser tracking concerns second only to hacking for web users ...
The Drum
Hacking is a lead concern for web users, but it is closely followed by advertiser tracking, an issue the marketing industry may have to address if it wants to ...
Mozilla: people have no idea how to protect their privacy and ...BetaNews

alle 2 nieuwsartikelen »
Categorieën: Mozilla-nl planet

Air Mozilla: Mozilla Weekly Project Meeting, 13 Mar 2017

Mozilla planet - ma, 13/03/2017 - 19:00

Mozilla Weekly Project Meeting The Monday Project Meeting

Categorieën: Mozilla-nl planet

Doar un român din zece mai folosește Mozilla Firefox! - Capital

Nieuws verzameld via Google - ma, 13/03/2017 - 18:07

Capital

Doar un român din zece mai folosește Mozilla Firefox!
Capital
Dacă ani de zile toată lumea folosea Internet Explorer(IE), în ultimii ani Chrome, lansat de Google, domină autoritar piaţa browserelor. Pe plan local, 64,9% dintre români folosesc Chrome, în timp ce 11,55% Firefox, 7,34% Safari, 3.29% Samsung Internet ...

Categorieën: Mozilla-nl planet

Mozilla Addons Blog: WebExtensions in Firefox 54

Mozilla planet - ma, 13/03/2017 - 18:00

Firefox 54 landed in Developer Edition this week, so we have another update on WebExtensions for you. In addition to new APIs to help more developers port over to WebExtensions, we also announced a new Office Hours Support schedule where developers can get more personalized help with the transition.

New APIs

A new API for creating sidebars was implemented. This allows you to place a local HTML file inside the sidebar. The API is similar to the one in Opera. If you specify the sidebar_action manifest key, Firefox will create a sidebar:

To allow keyboard commands to be sent to the sidebar, a new _execute_sidebar_action was added to the commands API which allows you trigger the showing of the sidebar.

The ability to override the about:newtab with pages inside your extension was added to the chrome_url_overrides field in the manifest. Check out the example that uses the topSites API to show the top sites you visit .

The privacy API gives you the ability to flip certain Firefox preferences related to privacy. Although the preferences in Chrome and Firefox aren’t a direct mapping, we’ve mapped the Firefox preferences that makes sense to the APIs. Currently implemented are: networkPredictionEnabled, webRTCIPHandlingPolicy and hyperlinkAuditingEnabled.

The protocol_handler API lets you easily map protocols to actions in your extension. For example: we use irccloud at Mozilla, so we can map ircs:// links to irccloud by adding this into an extension:

"protocol_handlers": [ { "protocol": "ircs", "name": "IRC Mozilla Extension", "uriTemplate": "https://irccloud.mozilla.com/#!/%s" } ]

When a user clicks on an IRC link, it shows the application selector with the IRC Mozilla Extension visible:

This release also marks the landing of the first sets of devtools APIs. Quite a few APIs landed including: inspectedWindow.reload(), inspectedWindow.eval(), inspectedWindow.tabId, network.onNavigated, and panels.create().

Here’s an example of the Redux DevTools extension running on Firefox:

Backwards incompatible changes

The webRequest API will now require that you’ve requested the appropriate hosts’ permissions before allowing you to perform webRequest operations on a URL. This will be a backwards-incompatible change for any extension which used webRequest but did not request the host permission.

Deletes in storage.sync are now encrypted. This would be a breaking change for any extensions using storage.sync on Developer Edition.

API Changes

Some key events were completed in some popular APIs:

  • webRequest.onBeforeRequest is initiated before a server side redirect is about occur and webRequest.onAuthRequired is fired when an authentication failure occurs. These allow you to catch authentication requests from servers, such as proxy authentication.
  • webNavigation.onCreatedNavigationTarget event has been completed. This is fired when a new window or tab is created to be navigated to.
  • runtime.onMessageExternal event has been implemented. This is fired when a message is sent from another extension.

Other notable bugs include:

Android

Notably in Firefox 54, basic tabs API support was landed for Android. The API support focuses on the parts of the API that make sense for Android, so some tab methods and events are deliberately not implemented.

This is an important API in its own right, but other key APIs did not have good support without this. By landing this, Android WebExtensions got much better webNavigation and webRequest support. This gives us a clear path to getting ad blockers, the most common extension type on Android.

Contributors

A big thank you to our contributors Rob Wu, Tomislav Jovanovic and Tushar Saini who helped out with this release.

The post WebExtensions in Firefox 54 appeared first on Mozilla Add-ons Blog.

Categorieën: Mozilla-nl planet

Mozilla: people have no idea how to protect their privacy and security online - BetaNews

Nieuws verzameld via Google - ma, 13/03/2017 - 17:33

Mozilla: people have no idea how to protect their privacy and security online
BetaNews
Privacy and security are major concerns when it comes to life online, but a survey by Mozilla reveals that a worrying number of people do not know how to stay in control of them. The company also found that a third of people feel they have no control ...

Categorieën: Mozilla-nl planet

Waterfox, un fork de Firefox pour les déçus de Mozilla - MacGeneration

Nieuws verzameld via Google - ma, 13/03/2017 - 17:06

MacGeneration

Waterfox, un fork de Firefox pour les déçus de Mozilla
MacGeneration
Waterfox est en quelque sorte la version de Firefox pensée pour les déçus de Mozilla. Ce fork n'intègre ni Pocket ni le module DRM d'Adobe (qui va bientôt être remplacé dans Firefox par un module similaire de Google) et ne recueille aucune donnée ...

Categorieën: Mozilla-nl planet

Tim Guan-tin Chien: Ben’s Story of Firefox OS

Mozilla planet - ma, 13/03/2017 - 15:50

Like my good old colleague Ben Francis, I too have a lot to say about Firefox OS.

It’s been little over a year since my team and I moved away from Firefox OS and the ill-fated Connected Devices group. Over the course of last year, each time I think about the Firefox OS experience, I arrived at different conclusions and complicated, sometimes emotional, belives. I didn’t feel I am ready to take a snapshot of these thoughts and publish it permanently, so I didn’t write about it here. Frankly, I don’t even think I am ready now.

In his post, Ben has pointed out many of the turning points of the project. I agree with many of his portrayals, most importantly, lost of direction between being a product (which must ship fast and deliver whatever partners/consumers wanted and used to) and a research project (which involves engineering endeavors that answer questions asked in the original announcement). I, however, have not figured out what can be done instead (which Ben proposed in his post nicely).

Lastly, a final point: to you, this might as well be another story in the volatile tech industry, but to me, I felt the cost of people enormously whenever a change was announced during the “slow death” of Firefox OS.

People moves on and recovers, including me (which fortunately wasn’t nearly being hit the hardest). I can only extend my best wishes to those who had fought the good fight with.

Categorieën: Mozilla-nl planet

Gregory Szorc: from __past__ import bytes_literals

Mozilla planet - ma, 13/03/2017 - 10:55

Last year, I simultaneously committed one of the ugliest and impressive hacks of my programming career. I haven't had time to write about it. Until now.

In summary, the hack is a source-transforming module loader for Python. It can be used by Python 3 to import a Python 2 source file while translating certain primitives to their Python 3 equivalents. It is kind of like 2to3 except it executes at run-time during import. The main goal of the hack was to facilitate porting Mercurial to Python 3 while deferring having to make the most invasive - and therefore most annoying - elements of the port in the canonical source code representation.

For the technically curious, it works as follows.

The hg Python executable registers a custom meta path finder instance. This entity is invoked during import statements to try to find the module being imported. It tells a later phase of the import mechanism how to load that module from wherever it is (usually a .py or .pyc file on disk) to a Python module object. The custom finder only responds to requests for modules known to be managed by the Mercurial project. For these modules, it tells the next stage of the import mechanism to invoke a custom SourceLoader instance. Here's where the real magic is: when the custom loader is invoked, it tokenizes the Python source code using the tokenize module, iterates over the token stream, finds specific patterns, and rewrites them to something more appropriate. It then untokenizes back to Python source code then falls back to the built-in loader which does the heavy lifting of compiling the source to Python code objects. So, we have Python 2 source files on disk that magically get transformed to be Python compatible when they are loaded by Python 3. Oh, and there is no performance penalty for the token transformation on subsequence loads because the transformed bytecode is cached in the .pyc file (using a custom header so we know it was transformed and can be invalidated when the transformation logic changes).

At the time I wrote it, the token stream manipulation converted most string literals ('') to bytes literals (b''). In other words, it restored the Python 2 behavior of string literals being bytes and not unicode. We jokingly call it from __past__ import bytes_literals (a play on Python 2's from __future__ import unicode_literals special syntax which changes string literals from Python 2's str/bytes type to unicode to match Python 3's behavior).

Since I implemented the first version, others have implemented:

As one can expect, when I tweeted a link to this commit, many Python developers (including a few CPython core developers) expressed a mix of intrigue and horror. But mostly horror.

I fully concede that what I did here is a gross hack. And, it is the intention of the Mercurial project to undo this hack and perform a proper port once Python 3 support in Mercurial is more mature. But, I want to lay out my defense on why I did this and why the Mercurial project is tolerant of this ugly hack.

Individuals within the Mercurial project have wanted to port to Python 3 for years. Until recently, it hasn't been a project priority because a port was too much work for too little end-user gain. And, on the technical front, a port was just not practical until Python 3.5. (Two main blockers were no u'' literals - restored in Python 3.3 - and no % formatting for b'' literals - restored in 3.5. And as I understand it, senior members of the Mercurial project had to lobby Python maintainers pretty hard to get features like % formatting of b'' literals restored to Python 3.)

Anyway, after a number of failed attempts to initiate the Python 3 port over the years, the Mercurial project started making some positive steps towards Python 3 compatibility, such as switching to absolute imports and addressing syntax issues that allowed modules to be parsed into an AST and even compiled and loadable. These may seem like small steps, but for a larger project, it was a lot of work.

The porting effort hit a large wall when it came time to actually make the AST-valid Python code run on Python 3. Specifically, we had a strings problem.

When you write software that exchanges data between machines - sometimes machines running different operating systems or having different encodings - and there is an expectation that things work the same and data roundtrips accordingly, trying to force text encodings is essentially impossible and inevitably breaks something or someone. It is much easier for Mercurial to operate bytes first and only take text encoding into consideration when absolutely necessary (such as when emitting bytes to the terminal in the wanted encoding or when emitting JSON). That's not to say Mercurial ignores the existence of encodings. Far from it: Mercurial does attempt to normalize some data to Unicode. But it often does so with a special Python type that internally stores the raw byte sequence of the source so that a consumer can choose to operate at the bytes or Unicode level.

Anyway, this means that practically every string variable in Mercurial is a bytes type (or something that acts like a bytes type). And since string literals in Python 3 are the str type (which represents Unicode), that would mean having to prefix almost every '' string literal in Mercurial with b'' in order to placate Python 3. Having to update every occurrence of simple primitives that could be statically transformed automatically felt like busy work. We wanted to spend time on the meaningful parts of the Python 3 port so we could find interesting problems and challenges, not toil with mechanical conversions that add little to no short-term value while simultaneously increasing cognitive dissonance and quite possibly increasing the odds of introducing a bug in Python 2. In other words, why should humans do the work that machines can do for us? Thus, the source-transforming module importer was born.

While I concede what Mercurial did is a giant hack, I maintain it was the correct thing to do. It has allowed the Python 3 port to move forward without being blocked on the more tedious and invasive transformations that could introduce subtle bugs (including performance regressions) in Python 2. Perfect is the enemy of good. People time is valuable. The source-transforming module importer allowed us to unblock an important project without sinking a lot of people time into it. I'd make that trade-off again.

While I won't encourage others to take this approach to porting to Python 3, if you want to, Mercurial's source is available under a GPL license and the custom module importer could be adapted to any project with minimal modifications. If someone does extract it as reusable code, please leave a comment and I'll update the post to link to it.

Categorieën: Mozilla-nl planet

Chrome en Firefox bestempelen miljoenen HTTP-websites als onveilig - Knack.be

Nieuws verzameld via Google - ma, 13/03/2017 - 09:01

Chrome en Firefox bestempelen miljoenen HTTP-websites als onveilig
Knack.be
Miljoenen websites wereldwijd worden getroffen door een beslissing van Google en Mozilla om in de webbrowsers Chrome en Firefox om http-websites als onveilig te bestempelen. Het gaat onder andere om websites van populaire media, apothekers en ...

Categorieën: Mozilla-nl planet

The Servo Blog: These Weeks In Servo 94

Mozilla planet - ma, 13/03/2017 - 01:30

In the last two weeks, we landed 185 PRs in the Servo organization’s repositories.

Planning and Status

Our overall roadmap is available online, including the overall plans for 2017 and Q1. Please check it out and provide feedback!

This week’s status updates are here.

Notable Additions
  • samgiles and rabisg added the Origin header to fetch requests.
  • hiikezoe made CSS animations be processed by Stylo.
  • Manishearth supported SVG presentation attributes in Stylo.
  • ferjm allowed redirects to occur after a CORS preflight fetch request.
  • sendilkumarn corrected the behaviour of loading user scripts in iframes.
  • pcwalton improved the performance of layout queries and requestAnimationFrame.
  • emilio removed unnecessary heap allocation for some CSS parsers.
  • mephisto41 implemented gradient border support in WebRender.
  • jdm avoided some panics triggered by image elements initiating multiple requests.
  • MortimerGoro improved the Android integration and lifecycle hooks.
  • nox removed the last uses of serde_codegen.
  • ferjm avoided a deadlock triggered by the Document.elementsFromPoint API.
  • fitzgen improved the rust-bindgen support for complex template parameter usages.
  • gw implemented page zoom support in WebRender.
  • dpyro added support for the nosniff algorithm in the Fetch implementation.
  • KiChjang implemented the :lang pseudoclass.
New Contributors

Interested in helping build a web browser? Take a look at our curated list of issues that are good for new contributors!

Categorieën: Mozilla-nl planet

Mike Hommey: When the memory allocator works against you

Mozilla planet - zo, 12/03/2017 - 02:47

Cloning mozilla-central with git-cinnabar requires a lot of memory. Actually too much memory to fit in a 32-bits address space.

I hadn’t optimized for memory use in the first place. For instance, git-cinnabar keeps sha-1s in memory as hex values (40 bytes) rather than raw values (20 bytes). When I wrote the initial prototype, it didn’t matter that much, and while close(ish) to the tipping point, it didn’t require more than 2GB of memory at the time.

Time passed, and mozilla-central grew. I suspect the recent addition of several thousands of commits and files has made things worse.

In order to come up with a plan to make things better (short or longer term), I needed data. So I added some basic memory resource tracking, and collected data while cloning mozilla-central.

I must admit, I was not ready for what I witnessed. Follow me for a tale of frustrations (plural).

I was expecting things to have gotten worse on the master branch (which I used for the data collection) because I am in the middle of some refactoring and did many changes that I was suspecting might have affected memory usage. I wasn’t, however, expecting to see the clone command using 10GB(!) memory at peak usage across all processes.

(Note, those memory sizes are RSS, minus “shared”)

It also was taking an unexpected long time, but then, I hadn’t cloned a large repository like mozilla-central from scratch in a while, so I wasn’t sure if it was just related to its recent growth in size or otherwise. So I collected data on 0.4.0 as well.

Less time spent, less memory usage… ok. There’s definitely something wrong on master. But wait a minute, that slope from ~2GB to ~4GB on the git-remote-hg process doesn’t actually make any kind of sense. I mean, I’d understand it if it were starting and finishing with the “Import manifest” phase, but it starts in the middle of it, and ends long before it finishes. WTH?

First things first, since RSS can be a variety of things, I checked /proc/$pid/smaps and confirmed that most of it was, indeed, the heap.

That’s the point where you reach for Google, type something like “python memory profile” and find various tools. One from the results that I remembered having used in the past is guppy’s heapy.

Armed with pdb, I broke execution in the middle of the slope, and tried to get memory stats with heapy. SIGSEGV. Ouch.

Let’s try something else. I reached out to objgraph and pympler. SIGSEGV. Ouch again.

Tried working around the crashes for a while (too long while, retrospectively, hindsight is 20/20), and was somehow successful at avoiding them by peaking at a smaller set of objects. But whatever I did, despite being attached to a process that had 2.6GB RSS, I wasn’t able to find more than 1.3GB of data. This wasn’t adding up.

It surely didn’t help that getting to that point took close to an hour each time. Retrospectively, I wish I had investigated using something like Checkpoint/Restore in Userspace.

Anyways, after a while, I decided that I really wanted to try to see the whole picture, not smaller peaks here and there that might be missing something. So I resolved myself to look at the SIGSEGV I was getting when using pympler, collecting a core dump when it happened.

Guess what? The Debian python-dbg package does not contain the debug symbols for the python package. The core dump was useless.

Since I was expecting I’d have to fix something in python, I just downloaded its source and built it. Ran the command again, waited, and finally got a backtrace. First Google hit for the crashing function? The exact (unfixed) crash reported on the python bug tracker. No patch.

Crashing code is doing:

((f)->f_builtins != (f)->f_tstate->interp->builtins)

And (f)->f_tstate is NULL. Classic NULL deref.

Added a guard (assessing it wouldn’t break anything). Ran the command again. Waited. Again. SIGSEGV.

Facedesk. Another crash on the same line. Did I really use the patched python? Yes. But this time (f)->f_tstate->interp is NULL. Sigh.

Same player, shoot again.

Finally, no crash… but still stuck on only 1.3GB accounted for. Ok, I know not all python memory profiling tools are entirely reliable, let’s try heapy again. SIGSEGV. Sigh. No debug info on the heapy module, where the crash happens. Sigh. Rebuild the module with debug info, try again. The backtrace looks like heapy is recursing a lot. Look at %rsp, compare with the address space from /proc/$pid/maps. Confirmed. A stack overflow. Let’s do ugly things and increase the stack size in brutal ways.

Woohoo! Now heapy tells me there’s even less memory used than the 1.3GB I found so far. Like, half less. Yeah, right.

I’m not clear on how I got there, but that’s when I found gdb-heap, a tool from Red Hat’s David Malcolm, and the associated talk “Dude, where’s my RAM?” A deep dive into how Python uses memory (slides).

With a gdb attached, I would finally be able to rip python’s guts out and find where all the memory went. Or so I thought. The gdb-heap tool only found about 600MB. About as much as heapy did, for that matter, but it could be coincidental. Oh. Kay.

I don’t remember exactly what went through my mind then, but, since I was attached to a running process with gdb, I typed the following on the gdb prompt:

gdb> call malloc_stats()

And that’s when the truth was finally unvealed: the memory allocator was just acting up the whole time. The ouput was something like:

Arena 0: system bytes = some number above (but close to) 2GB in use bytes = some number above (but close to) 600MB

Yes, the glibc allocator was just telling it had allocated 600MB of memory, but was holding onto 2GB. I must have found a really bad allocation pattern that causes massive fragmentation.

One thing that David Malcolm’s talk taught me, though, is that python uses its own allocator for small sizes, so the glibc allocator doesn’t know about them. And, roughly, adding the difference between RSS and what glibc said it was holding to to the use bytes it reported somehow matches the 1.3GB I had found so far.

So it was time to see how those things evolved in time, during the entire clone process. I grabbed some new data, tracking the evolution of “system bytes” and “in use bytes”.

There are two things of note on this data:

  • There is a relatively large gap between what the glibc allocator says it has gotten from the system, and the RSS (minus “shared”) size, that I’m expecting corresponds to the small allocations that python handles itself.
  • Actual memory use is going down during the “Import manifests” phase, contrary to what the evolution of RSS suggests.

In fact, the latter is exactly how git-cinnabar is supposed to work: It reads changesets and manifests chunks, and holds onto them while importing files. Then it throws away those manifests and changesets chunks one by one while it imports them. There is, however, some extra bookkeeping that requires some additional memory, but it’s expected to be less memory consuming than keeping all the changesets and manifests chunks in memory.

At this point, I thought a possible explanation is that since both python and glibc are mmap()ing their own arenas, they might be intertwined in a way that makes things not go well with the allocation pattern happening during the “Import manifest” phase (which, in fact, allocates and frees increasingly large buffers for each manifest, as manifests grow in size in the mozilla-central history).

To put the theory at work, I patched the python interpreter again, making it use malloc() instead of mmap() for its arenas.

“Aha!” I thought. That definitely looks much better. Less gap between what glibc says it requested from the system and the RSS size. And, more importantly, no runaway increase of memory usage in the middle of nowhere.

I was preparing myself to write a post about how mixing allocators could have unintended consequences. As a comparison point, I went ahead and ran another test, with the python allocator entirely disabled, this time.

Heh. It turns out glibc was acting up all alone. So much for my (plausible) theory. (I still think mixing allocators can have unintended consequences.)

(Note, however, that the reason why the python allocator exists is valid: without it, the overall clone took almost 10 more minutes)

And since I had been getting all this data with 0.4.0, I gathered new data without the python allocator with the master branch.

This paints a rather different picture than the original data on that branch, with much less memory use regression than one would think. In fact, there isn’t much difference, except for the spike at the end, which got worse, and some of the noise during the “Import manifests” phase that got bigger, implying larger amounts of temporary memory used. The latter may contribute to the allocation patterns that throw glibc’s memory allocator off.

It turns out tracking memory usage in python 2.7 is rather painful, and not all the tools paint a complete picture of it. I hear python 3.x is somewhat better in that regard, and I hope it’s true, but at the moment, I’m stuck with 2.7. The most reliable tool I’ve used here, it turns out, is pympler. Or rebuilding the python interpreter without its allocator, and asking the system allocator what is allocated.

With all this data, I now have some defined problems to tackle, some easy (the spike at the end of the clone), and some less easy (working around glibc allocator’s behavior). I have a few hunches as to what kind of allocations are causing the runaway increase of RSS. Coincidentally, I’m half-way through a refactor of the code dealing with manifests, and it should help dealing with the issue.

But that will be the subject of a subsequent post.

Categorieën: Mozilla-nl planet

Your connection is not secure – Mozilla Firefox Browser - TWCN Tech News (blog)

Nieuws verzameld via Google - za, 11/03/2017 - 18:51

TWCN Tech News (blog)

Your connection is not secure – Mozilla Firefox Browser
TWCN Tech News (blog)
Firefox had been my preferred browser earlier, and even now I keep on balancing between Chrome and Firefox. That said be it any browser one is expected to run into peculiar problems, and when this happens, troubleshooting is pretty tough. We need to ...

Categorieën: Mozilla-nl planet

Mozilla et BrowserStack : un partenariat pour résoudre les problèmes d ... - Ere Numérique

Nieuws verzameld via Google - za, 11/03/2017 - 15:03

Mozilla et BrowserStack : un partenariat pour résoudre les problèmes d ...
Ere Numérique
Mozilla BrowserStack Mozilla vient de s'unir avec BrowserStack afin de vérifier la compatibilité des sites dans les différents navigateurs et sur toutes les plateformes. C'est par l'intermédiaire d'un billet de blog paru la semaine dernière que Mozilla ...

Categorieën: Mozilla-nl planet

Mozilla veut réduire sa dépendance à Firefox - Le Monde

Nieuws verzameld via Google - za, 11/03/2017 - 10:21

Le Monde

Mozilla veut réduire sa dépendance à Firefox
Le Monde
Mozilla doit diversifier ses revenus. » C'est le message à retenir du passage à Paris de Denelle Dixon-Thayer, directrice juridique et commerciale de la fondation qui a développé le navigateur Firefox. Première manifestation de cette volonté : l ...
Firefox 53, Plus de prise en charge des PC Linux sous des CPU inférieurs aux ...GinjFo

alle 2 nieuwsartikelen »
Categorieën: Mozilla-nl planet

Mike Conley: Things I’ve Learned This Week (May 25 – May 29, 2015)

Thunderbird - ma, 01/06/2015 - 07:49
MozReview will now create individual attachments for child commits

Up until recently, anytime you pushed a patch series to MozReview, a single attachment would be created on the bug associated with the push.

That single attachment would link to the “parent” or “root” review request, which contains the folded diff of all commits.

We noticed a lot of MozReview users were (rightfully) confused about this mapping from Bugzilla to MozReview. It was not at all obvious that Ship It on the parent review request would cause the attachment on Bugzilla to be r+’d. Consequently, reviewers used a number of workarounds, including, but not limited to:

  1. Manually setting the r+ or r- flags in Bugzilla for the MozReview attachments
  2. Marking Ship It on the child review requests, and letting the reviewee take care of setting the reviewer flags in the commit message
  3. Just writing “r+” in a MozReview comment

Anyhow, this model wasn’t great, and caused a lot of confusion.

So it’s changed! Now, when you push to MozReview, there’s one attachment created for every commit in the push. That means that when different reviewers are set for different commits, that’s reflected in the Bugzilla attachments, and when those reviewers mark “Ship It” on a child commit, that’s also reflected in an r+ on the associated Bugzilla attachment!

I think this makes quite a bit more sense. Hopefully you do too!

See gps’s blog post for the nitty gritty details, and some other cool MozReview announcements!

Categorieën: Mozilla-nl planet

Mike Conley: The Joy of Coding (Ep. 16): Wacky Morning DJ

Thunderbird - do, 28/05/2015 - 04:12

I’m on vacation this week, but the show must go on! So I pre-recorded a shorter episode of The Joy of Coding last Friday.

In this episode1, I focused on a tool I wrote that I alluded to in the last episode, which is a soundboard to use during Joy of Coding episodes.

I demo the tool, and then I explain how it works. After I finished the episode, I pushed to repository to GitHub, and you can check that out right here.

So I’ll see you next week with a full length episode! Take care!

  1. Which, several times, I mistakenly refer to as the 15th episode, and not the 16th. Whoops. 

Categorieën: Mozilla-nl planet

Pagina's