<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
		>
<channel>
	<title>Comments on: fsyncers and curveballs</title>
	<atom:link href="http://shaver.off.net/diary/2008/05/25/fsyncers-and-curveballs/feed/" rel="self" type="application/rss+xml" />
	<link>http://shaver.off.net/diary/2008/05/25/fsyncers-and-curveballs/</link>
	<description>noise from signal</description>
	<lastBuildDate>Mon, 11 Jan 2010 00:40:58 +0000</lastBuildDate>
	<generator>http://wordpress.org/?v=2.9.2</generator>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
		<item>
		<title>By: Shawn Wilsher</title>
		<link>http://shaver.off.net/diary/2008/05/25/fsyncers-and-curveballs/comment-page-2/#comment-140390</link>
		<dc:creator>Shawn Wilsher</dc:creator>
		<pubDate>Wed, 31 Dec 2008 02:52:40 +0000</pubDate>
		<guid isPermaLink="false">http://shaver.off.net/diary/?p=3895#comment-140390</guid>
		<description>&lt;p&gt;Do we have any idea if ext4 has this fixed finally?  I haven&#039;t found any information indicating either way after a bit of digging tonight (admittedly, I haven&#039;t looked at source code).  It&#039;s no longer considered as under development, and is included in Kernel 2.6.28 (recently released).&lt;/p&gt;
</description>
		<content:encoded><![CDATA[<p>Do we have any idea if ext4 has this fixed finally?  I haven&#8217;t found any information indicating either way after a bit of digging tonight (admittedly, I haven&#8217;t looked at source code).  It&#8217;s no longer considered as under development, and is included in Kernel 2.6.28 (recently released).</p>]]></content:encoded>
	</item>
	<item>
		<title>By: Sean</title>
		<link>http://shaver.off.net/diary/2008/05/25/fsyncers-and-curveballs/comment-page-2/#comment-139363</link>
		<dc:creator>Sean</dc:creator>
		<pubDate>Fri, 20 Jun 2008 02:40:38 +0000</pubDate>
		<guid isPermaLink="false">http://shaver.off.net/diary/?p=3895#comment-139363</guid>
		<description>&lt;p&gt;I&#039;d like to point out that this is not just a Linux, or Linux Filesystem, or even EXT3 only problem.  PortableApps.com packages FireFox up for users who run it from their flash drives.  FF2 was a good performer on this media.  FF3 is a spectacular failure on flash drives.  The heavy IO on every page load crushes the flash drives (with typically slow write times) and makes FF3 and any other app running off the drive unusable.  Waiting for Linux to speed up the fsync() or for EXT3 to fix their ordered mode is not going to make the problem go away.  I&#039;m running off a 4Gig flash drive formatted Fat32, and the IO LED is on constantly while browsing.  Every page browsed freezes for 2-4 seconds.  Unusable.&lt;/p&gt;
</description>
		<content:encoded><![CDATA[<p>I&#8217;d like to point out that this is not just a Linux, or Linux Filesystem, or even EXT3 only problem.  PortableApps.com packages FireFox up for users who run it from their flash drives.  FF2 was a good performer on this media.  FF3 is a spectacular failure on flash drives.  The heavy IO on every page load crushes the flash drives (with typically slow write times) and makes FF3 and any other app running off the drive unusable.  Waiting for Linux to speed up the fsync() or for EXT3 to fix their ordered mode is not going to make the problem go away.  I&#8217;m running off a 4Gig flash drive formatted Fat32, and the IO LED is on constantly while browsing.  Every page browsed freezes for 2-4 seconds.  Unusable.</p>]]></content:encoded>
	</item>
	<item>
		<title>By: Richard Lloyd</title>
		<link>http://shaver.off.net/diary/2008/05/25/fsyncers-and-curveballs/comment-page-2/#comment-139351</link>
		<dc:creator>Richard Lloyd</dc:creator>
		<pubDate>Sun, 15 Jun 2008 10:23:37 +0000</pubDate>
		<guid isPermaLink="false">http://shaver.off.net/diary/?p=3895#comment-139351</guid>
		<description>&lt;p&gt;Just thought I&#039;d mention that I &lt;em&gt;do&lt;/em&gt; use the release version Linux .tar.gz of Firefox (and Thunderbird) from mozilla.com, but only because I have a script to create an RPM from it (using a spec file, icon and the .tar.gz) and I&#039;m using CentOS 5 (which is slow-moving and doesn&#039;t even have Firefox 2 yet!).&lt;/p&gt;

&lt;p&gt;I suspect the reason the Linux release binaries don&#039;t get downloaded a lot is because they&#039;re not shipped in any distro packaging format (RPM and deb being the obvious ones). Let&#039;s also not forget that there&#039;s no 64-bit builds of Firefox/Thunderbird on any platform either (another reason Linux distros have to step in and provide them...).&lt;/p&gt;

&lt;p&gt;In an ideal world, there should be Mozilla repos for RPM/deb and 32-bit/64-bit, but the impression I get is that either there isn&#039;t enough resources/willpower to get this done or the &quot;easy&quot; option of leaving the distros to do the builds/packaging takes the load off the Mozilla developers (but isn&#039;t this like saying &quot;Microsoft should build and ship Windows Firefox for us&quot;?!).&lt;/p&gt;
</description>
		<content:encoded><![CDATA[<p>Just thought I&#8217;d mention that I <em>do</em> use the release version Linux .tar.gz of Firefox (and Thunderbird) from mozilla.com, but only because I have a script to create an RPM from it (using a spec file, icon and the .tar.gz) and I&#8217;m using CentOS 5 (which is slow-moving and doesn&#8217;t even have Firefox 2 yet!).</p>

<p>I suspect the reason the Linux release binaries don&#8217;t get downloaded a lot is because they&#8217;re not shipped in any distro packaging format (RPM and deb being the obvious ones). Let&#8217;s also not forget that there&#8217;s no 64-bit builds of Firefox/Thunderbird on any platform either (another reason Linux distros have to step in and provide them&#8230;).</p>

<p>In an ideal world, there should be Mozilla repos for RPM/deb and 32-bit/64-bit, but the impression I get is that either there isn&#8217;t enough resources/willpower to get this done or the &#8220;easy&#8221; option of leaving the distros to do the builds/packaging takes the load off the Mozilla developers (but isn&#8217;t this like saying &#8220;Microsoft should build and ship Windows Firefox for us&#8221;?!).</p>]]></content:encoded>
	</item>
	<item>
		<title>By: Michel S.</title>
		<link>http://shaver.off.net/diary/2008/05/25/fsyncers-and-curveballs/comment-page-2/#comment-139330</link>
		<dc:creator>Michel S.</dc:creator>
		<pubDate>Thu, 05 Jun 2008 02:32:58 +0000</pubDate>
		<guid isPermaLink="false">http://shaver.off.net/diary/?p=3895#comment-139330</guid>
		<description>&lt;p&gt;The 3.0 RC2 announcement does not seem to mention the fsync() issue at all. Was it simply not communicated to the release team? The tarballed Linux build refuses to detect my network connection on my Fedora 9 / x86_64 laptop, so I guess I&#039;ll have to rebuild from source. Would be nice to know if the patch has indeed been applied.&lt;/p&gt;
</description>
		<content:encoded><![CDATA[<p>The 3.0 RC2 announcement does not seem to mention the fsync() issue at all. Was it simply not communicated to the release team? The tarballed Linux build refuses to detect my network connection on my Fedora 9 / x86_64 laptop, so I guess I&#8217;ll have to rebuild from source. Would be nice to know if the patch has indeed been applied.</p>]]></content:encoded>
	</item>
	<item>
		<title>By: anon</title>
		<link>http://shaver.off.net/diary/2008/05/25/fsyncers-and-curveballs/comment-page-2/#comment-139319</link>
		<dc:creator>anon</dc:creator>
		<pubDate>Sat, 31 May 2008 21:57:46 +0000</pubDate>
		<guid isPermaLink="false">http://shaver.off.net/diary/?p=3895#comment-139319</guid>
		<description>&lt;p&gt;Why is firefox the only SQLITE application that does this? Amarok for example, with an absurdly large collection, has no problem whatsoever with it&#039;s sqlite database pulling my system down into the mud. And it has never lost my data either (and I&#039;ve mishandled it quite a lot, killing it, pulling the power plug while updating the collection, etc.).
So, it seems to me the firefox developers are again blaming he outside world for their lack of skill and ingenuity (&quot;oooh, we don&#039;t leak memory! it&#039;s the os! and, oh, opera doesn&#039;t? let&#039;s ignore them, noone use them anyways!&quot;).&lt;/p&gt;
</description>
		<content:encoded><![CDATA[<p>Why is firefox the only SQLITE application that does this? Amarok for example, with an absurdly large collection, has no problem whatsoever with it&#8217;s sqlite database pulling my system down into the mud. And it has never lost my data either (and I&#8217;ve mishandled it quite a lot, killing it, pulling the power plug while updating the collection, etc.).
So, it seems to me the firefox developers are again blaming he outside world for their lack of skill and ingenuity (&#8220;oooh, we don&#8217;t leak memory! it&#8217;s the os! and, oh, opera doesn&#8217;t? let&#8217;s ignore them, noone use them anyways!&#8221;).</p>]]></content:encoded>
	</item>
	<item>
		<title>By: shaver</title>
		<link>http://shaver.off.net/diary/2008/05/25/fsyncers-and-curveballs/comment-page-2/#comment-139315</link>
		<dc:creator>shaver</dc:creator>
		<pubDate>Fri, 30 May 2008 10:19:05 +0000</pubDate>
		<guid isPermaLink="false">http://shaver.off.net/diary/?p=3895#comment-139315</guid>
		<description>&lt;p&gt;Mark: you can quote me on that, if you want.  On what more robust citation do you base your current belief?  You&#039;re making much more specific claims.&lt;/p&gt;
</description>
		<content:encoded><![CDATA[<p>Mark: you can quote me on that, if you want.  On what more robust citation do you base your current belief?  You&#8217;re making much more specific claims.</p>]]></content:encoded>
	</item>
	<item>
		<title>By: Mike Jeeves</title>
		<link>http://shaver.off.net/diary/2008/05/25/fsyncers-and-curveballs/comment-page-2/#comment-139314</link>
		<dc:creator>Mike Jeeves</dc:creator>
		<pubDate>Fri, 30 May 2008 03:00:48 +0000</pubDate>
		<guid isPermaLink="false">http://shaver.off.net/diary/?p=3895#comment-139314</guid>
		<description>&lt;p&gt;Hey (Don&#039;t know your name...)&lt;/p&gt;

&lt;p&gt;I read up on this from the bug, and this story, a few days ago. I re-read today and something struck me. You&#039;re writting this off as an OS bug and &#039;we didn&#039;t do this&#039;.&lt;/p&gt;

&lt;p&gt;Well... I also ran through your proposed fix for &#039;lowering fsync temporarily&#039;.&lt;/p&gt;

&lt;p&gt;So, in the final fix of this: You will fsync the same amount as you do now. Correct?&lt;/p&gt;

&lt;p&gt;Therefore, do you anticipate that Firefox will cause disk activity continually? You&#039;ve just said here that your fsyncing isn&#039;t a problem. You say that when it doesn&#039;t actually write, that is a problem.&lt;/p&gt;

&lt;p&gt;So you think Firefox should write out to disk that often?&lt;/p&gt;

&lt;p&gt;FF 2.0.10 + and FF 3.0.* have been the most unstable releases for me, both on Windows and Linux. I moved to 3.0 when 2.0.10 and above stopped being usable, and had to keep up with each 3.0 release because the number of &#039;click link, firefox dies&#039; bugs was unbelievable.&lt;/p&gt;

&lt;p&gt;I installed Opera. I just want to know - what is causing all these regressions? Adding pingback support? I bet it bloody well is, some stupid feature that nobody wants, and it is causing crashing issues. Anyway.&lt;/p&gt;
</description>
		<content:encoded><![CDATA[<p>Hey (Don&#8217;t know your name&#8230;)</p>

<p>I read up on this from the bug, and this story, a few days ago. I re-read today and something struck me. You&#8217;re writting this off as an OS bug and &#8216;we didn&#8217;t do this&#8217;.</p>

<p>Well&#8230; I also ran through your proposed fix for &#8216;lowering fsync temporarily&#8217;.</p>

<p>So, in the final fix of this: You will fsync the same amount as you do now. Correct?</p>

<p>Therefore, do you anticipate that Firefox will cause disk activity continually? You&#8217;ve just said here that your fsyncing isn&#8217;t a problem. You say that when it doesn&#8217;t actually write, that is a problem.</p>

<p>So you think Firefox should write out to disk that often?</p>

<p>FF 2.0.10 + and FF 3.0.* have been the most unstable releases for me, both on Windows and Linux. I moved to 3.0 when 2.0.10 and above stopped being usable, and had to keep up with each 3.0 release because the number of &#8216;click link, firefox dies&#8217; bugs was unbelievable.</p>

<p>I installed Opera. I just want to know &#8211; what is causing all these regressions? Adding pingback support? I bet it bloody well is, some stupid feature that nobody wants, and it is causing crashing issues. Anyway.</p>]]></content:encoded>
	</item>
	<item>
		<title>By: Ian McKellar</title>
		<link>http://shaver.off.net/diary/2008/05/25/fsyncers-and-curveballs/comment-page-2/#comment-139310</link>
		<dc:creator>Ian McKellar</dc:creator>
		<pubDate>Thu, 29 May 2008 21:01:53 +0000</pubDate>
		<guid isPermaLink="false">http://shaver.off.net/diary/?p=3895#comment-139310</guid>
		<description>&lt;p&gt;Oh man, why do I get myself into blog flamewars late at night. FWIW, I don&#039;t think SQLite&#039;s a great choice for a lot of things, I mostly dislike the apparent knee-jerk reaction from all quarters to use SQLite. That (bad) attitude sometimes causes me to fly off the handle when I should be sleeping instead.&lt;/p&gt;

&lt;p&gt;In Songbird SQLite has given us a fairly quick, fairly reliable datastore, but we&#039;ve come across a ton of limitations, bugs, and unexpected performance problems. We&#039;ve also had to make compromises when modeling our application&#039;s data as a set of tables, columns and relations that hurt us a little now and will probably hurt us more in the long-term.&lt;/p&gt;

&lt;p&gt;In my experience here I&#039;ve found SQLite&#039;s best feature is the development community. Once we&#039;re able to produce reduced test cases for the problems we&#039;ve had there&#039;s a really quick turn around with fixes, unit tests and releases.&lt;/p&gt;
</description>
		<content:encoded><![CDATA[<p>Oh man, why do I get myself into blog flamewars late at night. FWIW, I don&#8217;t think SQLite&#8217;s a great choice for a lot of things, I mostly dislike the apparent knee-jerk reaction from all quarters to use SQLite. That (bad) attitude sometimes causes me to fly off the handle when I should be sleeping instead.</p>

<p>In Songbird SQLite has given us a fairly quick, fairly reliable datastore, but we&#8217;ve come across a ton of limitations, bugs, and unexpected performance problems. We&#8217;ve also had to make compromises when modeling our application&#8217;s data as a set of tables, columns and relations that hurt us a little now and will probably hurt us more in the long-term.</p>

<p>In my experience here I&#8217;ve found SQLite&#8217;s best feature is the development community. Once we&#8217;re able to produce reduced test cases for the problems we&#8217;ve had there&#8217;s a really quick turn around with fixes, unit tests and releases.</p>]]></content:encoded>
	</item>
	<item>
		<title>By: TruePath</title>
		<link>http://shaver.off.net/diary/2008/05/25/fsyncers-and-curveballs/comment-page-2/#comment-139309</link>
		<dc:creator>TruePath</dc:creator>
		<pubDate>Thu, 29 May 2008 18:27:09 +0000</pubDate>
		<guid isPermaLink="false">http://shaver.off.net/diary/?p=3895#comment-139309</guid>
		<description>&lt;p&gt;Well it would be nice if there was a config option to run the db unsafely (and maybe there already is) but the risk of data corruption always seems like it&#039;s not worth worrying about until something actually goes wrong.  Remember the problem with cached writes is that you might end up with an on disk state that is totally trashed.  Many people, myself included, would be most upset to find out that we lost all our bookmarks, cookies and whatever other data to run slightly faster.&lt;/p&gt;

&lt;p&gt;This having been said it strikes me as mistaken to block on the call to fsync/F_FULLFSYNC.  This is the correct semantics for dealing with standard database type operations, submitting orders for sale, inventory etc.. etc.. where it&#039;s very important not to let a transaction appear to complete without storing that data to disk.  This isn&#039;t the case with firefox.  No further harm will occur if the user goes about his buisness and bookmarks another site in the second that firefox would otherwise have been stalled on the write to return.&lt;/p&gt;

&lt;p&gt;A well designed database could guarantee that no system corruption had occured by still occasionally submitting fsync requests and checking when they complete and ensuring that information needed to regenerate the prior version of the database in case of a crash is never overwritten until the fsync calls return telling it that new metadata blocks have been written.&lt;/p&gt;

&lt;p&gt;Also just to clarify about what OS X is doing.  OS X is offering exactly the correct semantics.  In OS X fsync guarantees that the file has been written to the &lt;i&gt;disk drive&lt;/i&gt;.  However, with NCQ the disk drive may choose to reorder that write or leave it in buffer for some time.  F_FULLFSYNC does a normal fsync and also flushes the buffers on the disk.  I know this is what you meant above but it wasn&#039;t very clear to me until I looked it up so I thought I would clarify.&lt;/p&gt;

&lt;p&gt;The reason this is the correct semantics is that it lets the programmer optimize their interactions with storage if they know that the disk is supported by a battery or otherwise guaranteed to actually flush it&#039;s internal buffers.  Hopefully one day all disks will be so designed.&lt;/p&gt;
</description>
		<content:encoded><![CDATA[<p>Well it would be nice if there was a config option to run the db unsafely (and maybe there already is) but the risk of data corruption always seems like it&#8217;s not worth worrying about until something actually goes wrong.  Remember the problem with cached writes is that you might end up with an on disk state that is totally trashed.  Many people, myself included, would be most upset to find out that we lost all our bookmarks, cookies and whatever other data to run slightly faster.</p>

<p>This having been said it strikes me as mistaken to block on the call to fsync/F_FULLFSYNC.  This is the correct semantics for dealing with standard database type operations, submitting orders for sale, inventory etc.. etc.. where it&#8217;s very important not to let a transaction appear to complete without storing that data to disk.  This isn&#8217;t the case with firefox.  No further harm will occur if the user goes about his buisness and bookmarks another site in the second that firefox would otherwise have been stalled on the write to return.</p>

<p>A well designed database could guarantee that no system corruption had occured by still occasionally submitting fsync requests and checking when they complete and ensuring that information needed to regenerate the prior version of the database in case of a crash is never overwritten until the fsync calls return telling it that new metadata blocks have been written.</p>

<p>Also just to clarify about what OS X is doing.  OS X is offering exactly the correct semantics.  In OS X fsync guarantees that the file has been written to the <i>disk drive</i>.  However, with NCQ the disk drive may choose to reorder that write or leave it in buffer for some time.  F_FULLFSYNC does a normal fsync and also flushes the buffers on the disk.  I know this is what you meant above but it wasn&#8217;t very clear to me until I looked it up so I thought I would clarify.</p>

<p>The reason this is the correct semantics is that it lets the programmer optimize their interactions with storage if they know that the disk is supported by a battery or otherwise guaranteed to actually flush it&#8217;s internal buffers.  Hopefully one day all disks will be so designed.</p>]]></content:encoded>
	</item>
	<item>
		<title>By: Mark</title>
		<link>http://shaver.off.net/diary/2008/05/25/fsyncers-and-curveballs/comment-page-2/#comment-139308</link>
		<dc:creator>Mark</dc:creator>
		<pubDate>Thu, 29 May 2008 18:26:20 +0000</pubDate>
		<guid isPermaLink="false">http://shaver.off.net/diary/?p=3895#comment-139308</guid>
		<description>&lt;blockquote&gt;
  &lt;p&gt;People use our binaries&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;[citation needed]&lt;/p&gt;
</description>
		<content:encoded><![CDATA[<blockquote>
  <p>People use our binaries</p>
</blockquote>

<p>[citation needed]</p>]]></content:encoded>
	</item>
</channel>
</rss>
