It was not as straightforward as i thought, because I had to start from the beginning and not trust any current Oracle settings. For example, FILESYSTEMIO_OPTIONS was set to "ASYNCH" in the spfile.
This would normally indicate that a DBA has specifically determined that ASYNCH I/O is possible and that they do not want to use DIRECTIO (or concurrent I/O).
Yet, reading the SAP notes, you would think that this is a moot point.
SAP note 830576 “Parameter recommendations for Oracle 10g” v227 clearly states that Oracle 10.2 on Solaris with UFS filesystems, supports asynchronous I/O, so therefore set the FILESYSTEMIO_OPTIONS to "SETALL".
However, if you have access to "My Oracle Support" you can check Oracle document "SOLARIS: Asynchronous I/O (AIO) on Solaris (SPARC) servers (Doc ID 48769.1)" which explains that, in Solaris with UFS, Oracle does not actually perform asynchronous I/O at the kernel (Solaris) level. Instead, Solaris simulates AIO by performing parallel writes (calls pwrite() ) but to Oracle it still looks and feels like KAIO.
So, the details in the SAP note are not exactly accurate. AIO is not supported in Solaris, but is simulated. Also, the simulation is just issuing more parallel I/O requests.
If your disk sub-system is slow, it's still going to be slow, but Oracle writes might be slightly faster if you handle/tune for more parallel requests at the storage layer.
Here's a nice article that explains it really well: http://sunsite.uakom.sk/sunworldonline/swol-07-1998/swol-07-insidesolaris.html