User`s guide

Preference Settings for Large Data Import
5-19
Preference Settings for Large Data Import
In this section...
“Will All Data (Size n) Fit in a MATLAB Variable?” on page 5-20
“Will All of This Data Fit in the JVM Heap?” on page 5-20
“How Do I Perform Batching?” on page 5-21
When using the setdbprefs to set 'FetchInBatches' and 'FetchBatchSize' or
the Cursor Fetch option for the Preference dialog box, use the following guidelines to
determine what batch size value to use. These guidelines are based on evaluating:
• The size of your data (n rows) to import into MATLAB
• The JVM heap requirements for the imported data
The general logic for making these evaluations are:
• If your data (n rows) will fit in a MATLAB variable, then will all your data fit in the
JVM heap?
• If yes, use the following preference setting:
setdbprefs('FetchInBatches','no')
• If no, evaluate h such that h < n and data of size h rows fits in the JVM heap. Use
the following preference setting:
setdbprefs('FetchInBatches','yes')
setdbprefs('FetchBatchSize','h')
• If your data (n rows) will not fit in a MATLAB variable, then:
• Evaluate m such that m < n and the data of size m rows fits in a MATLAB variable.
• Evaluate h such that h < m < n and data of size h rows fits in the JVM heap. Use
the following preference setting:
setdbprefs('FetchInBatches','yes')
setdbprefs('FetchBatchSize','h')
Then import data using fetch or runsqlscript by using the value 'm' to limit
the number of rows in the output:
curs = fetch(curs,m)
or