[lustre-discuss] Asynchronous POSIX I/O with Lustre

[Sebastian Rettenberger]

Hi,

I am currently investigating in asynchronous I/O capability of different
file systems. I wounder whether Lustre supports aio_write and aio_read
from the POSIX standard.

I found this bug report:
https://bugzilla.lustre.org/show_bug.cgi?id=11719
and this archived mail:
http://lists.lustre.org/htdig.cgi/lustre-discuss-lustre.org/2012-May/011087.html

Does anybody has any resent information on this? Is this docuemented in
the Lustre manual?

Best regards,
Sebastian

[Dilger, Andreas]

Post by Sebastian Rettenberger
Hi,
I am currently investigating in asynchronous I/O capability of different
file systems. I wounder whether Lustre supports aio_write and aio_read
from the POSIX standard.
https://bugzilla.lustre.org/show_bug.cgi?id=11719
http://lists.lustre.org/htdig.cgi/lustre-discuss-lustre.org/2012-May/01108
7.html
Does anybody has any resent information on this? Is this docuemented in
the Lustre manual?

The Lustre client does use the kernel ->aio_read() and ->aio_write()
operations, but I don't know whether anyone has actually tested whether the
IO operations will commit asynchronously or not. If you have a test tool
do verify this, it would definitely be interesting to know the results.

Cheers, Andreas

[Sebastian Rettenberger]

Hi,

thanks for the feedback.

I found out a few things (especially for Linux) that might be interesting:

- glibc implements aio_read/aio_write in user-space using threads. Thus,
they are always asynchronous but come with an overhead.

- The Linux kernel provides non-portable native asynchronous I/O through
libaio (io_submit).

I tested native AIO function of Linux on Ext4, NFS, Lustre and GPFS.
Only NFS performed the operation asynchronously and only with O_DIRECT.
On all other file systems, the operation was turned in a synchronous
operation.

Best regards,
Sebastian

Post by Dilger, Andreas

Post by Sebastian Rettenberger
Hi,
I am currently investigating in asynchronous I/O capability of different
file systems. I wounder whether Lustre supports aio_write and aio_read
from the POSIX standard.
https://bugzilla.lustre.org/show_bug.cgi?id=11719
http://lists.lustre.org/htdig.cgi/lustre-discuss-lustre.org/2012-May/01108
7.html
Does anybody has any resent information on this? Is this docuemented in
the Lustre manual?

The Lustre client does use the kernel ->aio_read() and ->aio_write()
operations, but I don't know whether anyone has actually tested whether the
IO operations will commit asynchronously or not. If you have a test tool
do verify this, it would definitely be interesting to know the results.
Cheers, Andreas

[Dilger, Andreas]

Post by Sebastian Rettenberger
Hi,
thanks for the feedback.
- glibc implements aio_read/aio_write in user-space using threads. Thus,
they are always asynchronous but come with an overhead.
- The Linux kernel provides non-portable native asynchronous I/O through
libaio (io_submit).
I tested native AIO function of Linux on Ext4, NFS, Lustre and GPFS.
Only NFS performed the operation asynchronously and only with O_DIRECT.
On all other file systems, the operation was turned in a synchronous
operation.

Could you please file a ticket in Jira and share what test tool you are
using to check the aio functionality? That would at least allow someone
to investigate ow this is implemented and where it is getting blocked.
I can't promise someone will work on it any time soon, but at least we
would have somewhere to start, since I don't think this is a common IO
interface except with database workloads.

Cheers, Andreas

Post by Sebastian Rettenberger

Post by Dilger, Andreas

Post by Sebastian Rettenberger
Hi,
I am currently investigating in asynchronous I/O capability of different
file systems. I wounder whether Lustre supports aio_write and aio_read
from the POSIX standard.
https://bugzilla.lustre.org/show_bug.cgi?id=11719
http://lists.lustre.org/htdig.cgi/lustre-discuss-lustre.org/2012-May/011
08
7.html
Does anybody has any resent information on this? Is this docuemented in
the Lustre manual?

The Lustre client does use the kernel ->aio_read() and ->aio_write()
operations, but I don't know whether anyone has actually tested whether the
IO operations will commit asynchronously or not. If you have a test tool
do verify this, it would definitely be interesting to know the results.
Cheers, Andreas

--
Sebastian Rettenberger, M.Sc.
Technische Universität München
Department of Informatics
Chair of Scientific Computing
Boltzmannstrasse 3, 85748 Garching, Germany
http://www5.in.tum.de/

Cheers, Andreas

[Vitaly V. Ch]

Hi!

That use case is not specific to databases only. Nginx users choose GPFS
over Lustre just because GPFS does not have this trouble.

With best regards,

Vitaly

Post by Dilger, Andreas

Post by Sebastian Rettenberger
Hi,
thanks for the feedback.
- glibc implements aio_read/aio_write in user-space using threads. Thus,
they are always asynchronous but come with an overhead.
- The Linux kernel provides non-portable native asynchronous I/O through
libaio (io_submit).
I tested native AIO function of Linux on Ext4, NFS, Lustre and GPFS.
Only NFS performed the operation asynchronously and only with O_DIRECT.
On all other file systems, the operation was turned in a synchronous
operation.

Could you please file a ticket in Jira and share what test tool you are
using to check the aio functionality? That would at least allow someone
to investigate ow this is implemented and where it is getting blocked.
I can't promise someone will work on it any time soon, but at least we
would have somewhere to start, since I don't think this is a common IO
interface except with database workloads.
Cheers, Andreas
<javascript:;>>

Post by Sebastian Rettenberger

Post by Dilger, Andreas

Post by Sebastian Rettenberger
Hi,
I am currently investigating in asynchronous I/O capability of different
file systems. I wounder whether Lustre supports aio_write and aio_read
from the POSIX standard.
https://bugzilla.lustre.org/show_bug.cgi?id=11719

http://lists.lustre.org/htdig.cgi/lustre-discuss-lustre.org/2012-May/011

Post by Sebastian Rettenberger

Post by Dilger, Andreas

Post by Sebastian Rettenberger
08
7.html
Does anybody has any resent information on this? Is this docuemented in
the Lustre manual?

The Lustre client does use the kernel ->aio_read() and ->aio_write()
operations, but I don't know whether anyone has actually tested whether the
IO operations will commit asynchronously or not. If you have a test tool
do verify this, it would definitely be interesting to know the results.
Cheers, Andreas

--
Sebastian Rettenberger, M.Sc.
Technische UniversitÀt MÌnchen
Department of Informatics
Chair of Scientific Computing
Boltzmannstrasse 3, 85748 Garching, Germany
http://www5.in.tum.de/

Cheers, Andreas
--
Andreas Dilger
Lustre Principal Architect
Intel High Performance Data Division
_______________________________________________
lustre-discuss mailing list
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org