Over the last many months, flash has been crashing for me on any page with flash video when I browse away. This means flash crashes when clicking a link or typing in the location bar. Or probably worst when a site that has a flash intro it crashes when loading the main content of the site.

This seems to only affect the Flash Player 10 plugin on OS X, but it was still a bit tricky finding out what crash reports are related to this particular issue.

– There are many versions of the flash plugin

If a bug is reported with a crash signature like [@ FlashPlayer-10.6@0x4b4b39 ], watching the number of crashes for that signature go down may initially seem like something was fixed, but actually users are upgrading their version of flash and crashing at a different address. Additionally, it’s harder to see the actual number of crashes for this issue at a glance as the reports are spread across many entries.

– FlashPlayer-10.6 plugin version is missing

Adding to the issue of there being many versions, for some reason the version information for FlashPlayer-10.6 shows up in reports as 0.0.0.0. Fortunately there’s an additional debug identifier that is consistently reported for the same version of the plugin.

Below is a listing of some recent versions and debug ids:

10.2.151.49
nov 30
3CDC6F1339C791580EC31A451724F0110
10.1.102.64
nov 4
7F846A865E18B6E54DA2E5B46E7CF5B20
10.1.85.3
sep 20
392E898E2D7BCE8B02FFCE2287F7DF570
10.1.82.76
aug 10
5295F1D9A734E9FDD5350646BF2615A40
10.1.53.64
jun 10
28AEE5D0CE3E55DB5AE5787143EC0F960

I gathered these by looking through Adobe’s security bulletins to find the date and version of the new release. With the release date, I could then search for crash reports from before the release and see which reports disappeared. Then from the reports that did disappear, I can record the debug identifier to be for the new version.

– Crash reports not being submitted

Another issue that has recently been fixed was that most people running into these plugin crashes had no option to send a crash report. This was because Firefox by default runs in 64-bit mode and was unable to create a report for 32-bit plugin crashes. This means the frequency of the crashes were massively under-reported and the sparse crash data made it harder to find relevant reports.

– Improving crash-stats triaging

For this particular case, I can think of a couple features that could have made tracking down flash crashes more efficient.

Because the crash signature kept changing from version to version, I had to inspect many crashes to see if the whole stack is similar to other stacks. For one set of crashes, I was looking for AnswerNPP_Destroy in frame #8 to identify the family of crashes. Being able to search in part of the stack could also make it easier for various component teams find relevant reports.

For another set of crashes, the crash reason was always EXC_BAD_ACCESS at address 0x44c. So in addition to being able to filter by crash signature, being able to search by reason and address could have avoided needing to click through many crash signature reports.

There’s a number of bugs filed for these flash crashes like bug 590955, bug 572134, and bug 617469. I’m still not sure if it’s an issue with OOPP or a bug in the plugin, so it’s hard to say if flash will stop crashing by a new Firefox beta release or a plugin update. But just as a heads up, getting the latest Flash Player 10.2(.151.49) beta or Firefox 4 beta8 won’t fix these crashes.

Being able to run chrome-privileged code from the Error Console has been very convenient for me to help users debug issues with Firefox or an add-on when they get stuck in some strange state. I can quickly test an idea or a snippet of code and have the user run it to track down where things are going wrong. However, there’s some issues that make it tricky to use:

• Users get confused doing Tools -> Error Console -> Paste -> Evaluate
• Users might not copy/paste correctly or perhaps the code got wrapped oddly
• Javascript version seems to be 1.6, so let needs to be replaced with var, etc.
• Scripts are heavily GCed, so modules are imported to attach objects to keep them alive
• Everything needs to be on one line, so beware of any lines with // comments!
• Console output is html-like and strips tags and empty output leading to confusion

One additional drawback in Firefox 4 is the Error Console is now disabled by default, so having users first go to about:config and toggling devtools.errorconsole.enabled to true is an extra step where users can get lost. (Don’t forget about the click-through warning page! ;)) Additionally, I believe enabling the Error Console requires a restart, so debugging a live instance is troublesome unless the problem is easily reproduced.

New preference to enable the Error Console (under the Tools menu)

Fortunately, Firefox 4 comes with restartless add-ons! You can just package up an add-on with two files: bootstrap.js and install.rdf, with the relevant code running from the startup() function. It’s simpler for users to try out as it looks like any other add-on install, so just point to the .xpi and click Install.

Just make sure that if you want the code to only run once, have the add-on uninstall itself after running. Otherwise it’ll be like any other add-on that runs when Firefox starts.

Code snippets to for one-time scripts and easy reloading of scripts

Another neat trick for reloading or running an updated script is having the add-on auto-enable itself when it’s disabled. I can now just make changes to the files and click Disable once from about:addons, and it’ll automatically reload itself instead of needing to click Enable. Also because now I can just edit the file and not need to copy/paste into Error Console, I can easily track my changes with revision control software. 🙂

I’ve provided the bootstrap.js and install.rdf files needed for this simple restartless add-on. From there you just need to:

1. Delete one of the auto-uninstall or auto-enable snippets from bootstrap.js
2. Add in the code you want to run inside startup() of bootstrap.js
3. Package up the files into an xpi: zip restartless.xpi bootstrap.js install.rdf
4. Distribute/install restartless.xpi

As Atul mentioned in “The Social Constraints of Bettering The Web” [toolness.com], Account Manager will likely not make its way into Firefox 4. He points out one of the biggest bottlenecks as getting approval from Firefox product drivers:

Within Mozilla, I see my coworkers vie for the attention of this tiny handful of gatekeepers. People in charge need convincing; the clever social engineer has a lot of power when it comes to navigating this landscape.

This last step of getting approval comes at the very end of a long line of work. While Dan and I have been busy implementing the core feature and making sure it doesn’t regress performance and tests, a good number of people have been involved both inside and outside of Mozilla to design the user interface, to flesh-out the spec and to integrate the feature into sites or as plug-ins.

Alex Faaborg's mockup of letting users pick different types of accounts

But this approval process affects people outside of Mozilla as well. New developers in the community hear about neat features of the upcoming Firefox and want to help by hacking on patches of related features. For example, the status bar removal in Firefox 4 has a number of side-effects including the removal of the download statusbar. Alex Limi has suggested ways for “Improving download behaviors in web browsers” [limi.net], which would add a new toolbar interface, and while there are initial patches from multiple community members, it seems unlikely to make it to Firefox 4.

I even ran into this same roadblock a couple years ago when I was trying to get the AwesomeBar into Firefox 3. Back then I was a random community member that had a good idea and was able to hack on stuff in my “free” (ha! ;)) time. It was only after a lot of prodding and persistence that got just a bit of what I worked on into Firefox 3, but that was all very stressful as I looked back on “Why I Worked On Firefox.”

But fear not community members! There’s always another release. The product drivers are not approving patches during this beta-crunch time because there’s always some risk involved with changes (especially those “it’s just a one line change” fixes :D). New changes typically are followed up by a number of new issues and patches that then need to be additionally hacked on, tested, reviewed, and approved. So just because it doesn’t get approval now doesn’t mean it won’t be accepted when the tree is more open.

Additionally, patches usually come with a number of dependent fixes that might be able to land first. And in the case of Account Manager, I’ve already gotten in some changes into Firefox 4 that improve the new PopupNotifications (used by Geolocation and Add-ons) and testing infrastructure. Some other useful changes that could land independently of Account Manager are some upgrades to the Password Manager and networking APIs. So while the core feature might not be in yet, the platform is made better and ready for it.

So keep hacking away and perhaps your feature will be ready to land on the open tree after Firefox 4 branches. And then it’ll have many months to bake and get tested by other community members and eventually be seen by millions of Firefox users. 🙂

In designing the Firefox-facing APIs of Account Manager, thunder [sandmill.org] and I decided to make most of the interfaces take a callback/continuation. Two main reasons were 1) to allow add-ons to add extra account types like OpenID and 2) to find out if you’re logged-in to a site over multiple network requests.

For an add-on to provide a new account type, it needs to be able to tell Firefox what saved accounts are available and how to connect to them. These might need to read data from disk and/or network; or perhaps request more information from the user like a 1-time code sent over SMS. For Firefox to support these more-complex interactions, it can’t assume that a method implemented by the add-on can return a value immediately.

Asking for additional information to connect to a banking site

In this particular mockup, the site has told Firefox not to use the basic username-password account type but instead to use an account type that asks for more information. Here, an add-on has already saved the username/password and doesn’t to ask for them, but it can’t immediately connect without asking for more information, so it tells Firefox to show this popup. With the async. API, the add-on can later tell Firefox that it has finished the connect process.

The second reason for going async. overlaps with the first of supporting add-ons; Account Manager and account types need to talk over the network to other machines. These requests can take more than just a few milliseconds, so blocking the rest of Firefox when you click on the “Sign in” button using a synchronous request would be bad. In the case of finding the account status, Firefox might need to make requests for any or all of host-meta, AMCD and the session status.

The Password Manager API in Firefox happens to be synchronous—as are many other interfaces in Firefox. Converting it to be asynchronous for use in the username/password account type was fairly simple, but there’s a couple things to keep in mind: when the work is done and when the result is given.

Implementing the async. interface with the synchronous Password Manager

Here, we’ve made the call to savedAccounts immediately do the work of finding the logins, so this means the caller might have to wait before the asynchronous savedAccounts call returns. We could have just as easily delayed this work to run after the function returns by moving the logic into the async function call, but one needs to be aware that if the arguments are live objects, the contents might change by the time async triggers the callback.

The second point of “when the result is given” is less flexible. The function above could have been written to just call onComplete(accounts) immediately without the async wrapper, but that could break the caller as the implementation is no longer truly asynchronous. The caller would stop working if code is supposed to execute after the call to savedAccounts but before the call to continuation, onComplete.

Trivial example of how non-async. implementations could break

For web developers, implementing async is pretty simple as the global window object has setTimeout. So one implementation could look like function async(callback) setTimeout(callback, 0);

Making the choice of having the APIs be asynchronous does have some drawbacks in terms of code structure. Any function that eventually calls an async. function will be forced to take a callback as well, and if it needs to call multiple async. functions, the logic needs to be broken up into multiple callbacks. If you want to share variables across these callbacks, you’ll end up creating many nested anonymous functions (closures).

Additionally, some simple things like doing Array.reduce (fold), which visits each array item and applies a function to produce a single result, can get pretty complicated if you want to pass in an async. function. In a later post, I’ll describe a way to have asynchronous functions look like they’re synchronous so that you can avoid some of this callback craziness. 😉