Re: direct-to-BIO for O_DIRECT

[Lincoln Dale]

At 08:24 PM 10/07/2002 -0700, Andrew Morton wrote:
> >    2.5.25 ('virgin' 2.5.25 with the exception of changing PAGE_OFFSET to
> > 0x80000000 and
> >           your O_DIRECT-on-blockdev patch to stop it oopsing -- oops report
> > below)
> >       normal     167772160 blocks of 512 bytes in 607 seconds (134.81
> > mbyte/sec), CPUs 0% idle
> >       O_DIRECT   20480 blocks of 4194304 bytes in 420 seconds (194.61
> > mbyte/sec), CPUs ~93% idle
> >       /dev/rawN  20480 blocks of 4194304 bytes in 422 seconds (193.84
> > mbyte/sec), CPUs ~92% idle
>
>The 30% improvement in pagecache-buffered reads is somewhat unexpected.
>The blockdevs are not using multipage BIOs - they're still using
>buffer_head-based I/O for both reads and writes.  Are you sure that
>the 2.4 QLogic driver is using block-highmem?

pretty sure -- there's no highmem in the system: :-)
(i.e. i changed PAGE_OFFSET in order to prevent there being any highmem).

         [root@mel-stglab-host1 root]# cat /proc/meminfo
         MemTotal:      1945680 kB
         MemFree:       1853812 kB
         MemShared:           0 kB
         Cached:          29536 kB
         SwapCached:       2520 kB
         Active:          32336 kB
         Inactive:         8336 kB
         HighTotal:           0 kB
         HighFree:            0 kB
         LowTotal:      1945680 kB
         LowFree:       1853812 kB
         SwapTotal:     2047992 kB
         SwapFree:      2037268 kB
         Dirty:            1396 kB
         Writeback:           0 kB

>OK, so there's nothing there at all really (or there may be.  Hard
>to tell when the interface has saturated).
>
>But on my lowly scsi disks I was seeing no change in read bandwidth
>either.  Only writes benefitted for some reason.   Can you do
>some write testing as well?   If you test writes through the pagecache,
>use ext2 and not direct-to-blockdev please - that'll take the multipage
>BIOs, buffer_head-bypass route.  Plain old read and write of /dev/XdYY
>isn't very optimised at all.

will do.

do you have any other preferences --
  - ext2 or ext3?
  - if ext3, change the journalling mode?
  - i/o to a single large file or multiple files per spindle?

i can also add combinations of read/write & seeking also.
what kind of file-size should i be using?

cheers,

lincoln.