This post is provided as a log of personal experimentation only. If you attempt to replicate these steps on your system, it is wholly at your own risk with no guarantee you will get the same results as I did. It is in no way supported by Oracle, and I take no responsibility for any ill effects on your system - and Oracle won't either. This is wholly undocumented and unrecommended. I'm purposely changing things to understand them better. If you want a stable (and simpler) experience, use the Linux versions documented to be supported by Oracle.
With the lawyers held at bay, I can proceed. Remember, don't try this at work, kids - only on a system no one else uses, that you don't mind reinstalling if things go haywire.
My environment (CentOS 5.3)
$ uname -r
According to the Oracle Database Storage Administrator's Guide ACFS on a non-clustered system requires some manual steps to launch the volume driver. The steps outlined at the hyperlink above detail what to do manually.
login as root
$ cd /u01/app/oragrid/product/11.2.0/grid/bin
$ ./acfsload start -s
ADVM/ACFS is not supported on centos-release-5-3.el5.centos.1
Not surprising, the documentation said as much. Let's see if we can work around that.
Some backtracking through the shell and perl scripts leads us to ../lib/osds_acfslib.pm
$ cd ../lib
$ cp -p osds_acfslib.pm osds_acfslib.pm.orig
Now I edit osds_acfslib.pm (carefully) Search for "support this release" to get to the section to tweak. (line 278 in my case)
Since CentOS5 is equivalent to RedHat 5, we'll modify the test for RedHat to look like this...
if (($release =~ /enterprise-release-5/) ||
($release =~ /redhat-release-5/) ||
($release =~ /centos-release-5/))
That gets us past the supported version check, based on the version string. Now we get ACFS-9129: ADVM/ACFS not installed
$ mkdir /lib/modules/2.6.18-128.el5/extra/usm
$ cp /u01/app/oragrid/product/11.2.0/grid/install/usm/EL5/i386/2.6.18-8/2.6.18-8.el5-i686/bin/*ko /lib/modules/2.6.18-128.el5/extra/usm
$ chmod 744 /lib/modules/2.6.18-128.el5/extra/usm
This puts the modules in the expected location. We can check it by:
$ cd /u01/app/oragrid/product/11.2.0/grid/bin
$ ./acfsdriverstate -orahome /u01/app/oragrid/product/11.2.0/grid version
ACFS-9205: OS/ADVM,ACFS installed version = 2.6.18-8.el5(i386)/090715.1
Record dependencies for the kernel modules. (The modules won't load if you skip this step).
Now running acfsload start -s completes without any further messages.
In a cluster install the system executables get copied from the install directory to /sbin as part of the root.sh script. Since this was a standalone install, they did not get copied. We get to do that manually (and set permissions).
$ cd /u01/app/oragrid/product/11.2.0/grid/install/usm/EL5/i386/2.6.18-8/2.6.18-8.el5-i686/bin
For each file that does not end in .ko (there are 12 of them):
$ cp acfsdbg /sbin
$ chmod 755 /sbin/acfsdbg
TEST FOR SUCCESS:
Now connect as oragrid.
Running asmca is noticeably different - the Volumes and ASM Cluster File System tabs are available. (They were greyed out when I tried before).
Now we'll try the command that failed before in ASMCMD
ASMCMD > volcreate -G FS -s 2G --redundancy mirror acfs1
This completed without any error message. Switching back to asmca to see it in the GUI...
ACFS1 now appears in the Volume tab, with a device /dev/asm/acfs1-287. It is unmounted. We could also find out this virtual device using the volinfo command in asmcmd.
Now make a file system on that volume and mount it...
$ /sbin/mkfs -t acfs /dev/asm/acfs1-287
mkfs.acfs: version = 188.8.131.52.0.0
mkfs.acfs: on-disk version = 39.0
mkfs.acfs: volume = /dev/asm/acfs1-287
mkfs.acfs: volume size = 2147483648
mkfs.acfs: Format complete.
$ /sbin/acfsutil registry -a /dev/asm/acfs1-287 /u01/app/acfsmounts/acfstrialAt this point, the ASM Cluster File Systems tab in asmca shows our registered mount point, in a DISMOUNTED state. ls /u01/app shows no directory .../acfsmounts.
acfsutil registry: mount point /u01/app/acfsmounts/acfstrial successfully added
to Oracle Registry
To mount a filesystem, you need root privilege.
add the following to the beginning of /sbin/mount.acfs, after the header comments
Create a directory to mount the filesystem on. This step wasn't in the install guide. I got an error if I didn't pre-create a directory.
$ mkdir /u01/app/acfsmounts/acfstrial
Perform the mount and set permissions
$ mount -t acfs /dev/asm/acfs1-287 /u01/app/acfsmounts/acfstrial
$ chown -R oracle11r2:dba /u01/app/acfsmounts/acfstrial
And we are done! A usable file system, owned by the ID we will later install the Oracle 11gR2 Database using.
$ su - oracle11r2
$ cd /u01/app/acfsmounts/acfstrial
$ echo hello world > myfile
$ cat myfile
And there we have it... ASM Cluster File System running on a standalone CentOS box. Now I have a place to experiment and build up my skill set, with no money out of pocket!
This doesn't come up automatically following a reboot. I need to add startup scripts to perform the following steps:
1. Load the ADVM drivers
2. Enable the ASM volumes
3. Mount the ACFS filesystems
I can do these steps manually with a combination of the steps above, and with asmca.