Re: Why O_SYNC and fsync are slow in Linux?

[Lance Larsh]

On Wed, 16 May 2001, Heikki Tuuri wrote:

> On Red Hat 6.2 and 7.? Intel big block writes are very slow if
> I open the file with O_SYNC. I call pwrite to write 1 MB chunks to
> the file, and I get only 1 MB/s write speed. If I open without O_SYNC
> and call fsync only after writing the whole 100 MB file,
> I get 5 MB/s. I got the same adequate speed 5 MB/s with 16 MB writes
> after which I called fdatasync.

When you open with O_SYNC, the data must go all the way to disk on
every write call.  This means you get at least one disk access for
every write, and possibly more if the writes are large (>64k).

When you don't use O_SYNC and only flush after all writes have been
submitted by the application, then the kernel is able to combine writes
in the cache and at the blk dev layer.  Therefore you end up with fewer
accesses to the physical disk, which makes it much faster.

> On a Linux-Compaq Alpha I measured the following: if I open with O_SYNC,
> I can flush the end of my file (it is a log file) to
> disk 170 times / second. If I do not open with O_SYNC,
> but call fsync or fdatasync after each write, I get only 50
> writes/second.

This is generally the case.  If you need to sync every write, O_SYNC
is usually faster than fsync.  If you don't need to sync
every individual write, then a single fsync after the last
write is the fastest to get all the data to disk.

> 
> On the Red Hat 7.? I get 500 writes per second if I open with O_SYNC.
> That is too much because the disk does not rotate
> 500 rotations/second. Does the disk fool the operating
> system to believe a write has ended while it has not?

Are you using an IDE disk?  If so, it probably has a huge cache.

-Lance