failure is not an option
There’s a great writeup over on Matasano (home of many a great writeup) about how a supergenius hacker was able to exploit a NULL pointer coming out of malloc failure to run arbitrary code in Flash. This is interesting to Mozilla in part because a lot of our users have Flash installed, but also specially interesting to me because we’re working with Adobe to converge on a common, high-performance scripting engine for both JavaScript-as-she-is-written-today and ECMAScript future. I’m actually in Boston tomorrow to work with some Mozillians to map out the next interim milestone on our way to JS2 and Tamarin.
As part of the same general effort, known as “Mozilla 2″, we’re also going to be changing how we do memory allocation, so that — just as Thomas recommends — out of memory is a hard-stop failure, rather than an opportunity for a clever (or, as in this case, hyper-clever) exploit to take hold.
Of course, in a system as large as ours, you don’t want to do it all by hand, so we’ll be using static analysis tools to identify and rewrite our code mechanically. This will give us better performance from less computer time spent checking allocation results, reduced code complexity from less human time spent reading through tedious failure-handling code, and protection against a large class of potential attacks. That’s a pretty nice set of things to get in one package.
