CHANGES
Symptoms occur when reading or writing a large amount of data, without direct I/O, on Ext3 file systems on NUMA enabled database servers in Sun Oracle Database Machine.
CAUSE
Database servers in Sun Oracle Database Machine are booted with operating system NUMA support enabled. Commands that manipulate large files without using direct I/O on ext3 file systems will cause low memory conditions on the NUMA node (Xeon 5500 processor) currently running the process.
An example of such an operation is a scp(1) of a set of large flat files in preparation for loading data into the data warehouse. Linux will try to execute the scp(1) command on the same processor until it exits and as such will grant node-local memory for page cache buffering of the file content being copied into the file system.
For example, consider the following case of copying several gigabytes of data into a local ext3 file system on a database server in Sun Oracle Database Machine. The example clearly shows that memory for the file copy was allocated in an unbalanced fashion. Once node-local memory is depleted, system performance as a whole will be severely impacted.
