User`s guide

Preference Settings for Large Data Import
5-21
The value of your JVM heap can be determined by selecting MATLAB Preferences and
General > Java Heap Memory. You can increase this value to an allowable size, but
keep in mind that increasing JVM heap reduces the total memory available to MATLAB
arrays. Instead, consider fetching data in small batches to keep a low to medium value
for heap memory.
How Do I Perform Batching?
There are three different methods based on your evaluations of the data size and the
JVM heap size. Let n be the total number of rows in the data you are looking to import, m
be the number of rows that fit in a MATLAB variable, and h be the number of rows that
fit in the JVM heap.
Method 1 — Data Does Not Fit in MATLAB Variable or JVM Heap
If your data (n) does not fit in a MATLAB variable or a JVM heap, you need to find h and
m such that h < m < n.
To use automated batching to fetch those m rows in MATLAB:
setdbprefs('FetchInBatches','yes')
setdbprefs('FetchBatchSize','h')
If using exec, fetch, and connection object conn:
curs = exec(conn,'Select….’);
curs = fetch(curs,m);
If using runsqlscript to run a query from an SQL file 'filename.sql':
results = runsqlscript(conn,'filename.sql’,'rowInc’,'m’)
Once you are done processing these m rows, you can import the next m rows using the
same commands. Keep in mind, however, that using the same cursor object curs for this
results in the first curs being overwritten, including everything in curs.Data.
Note: If 'FetchInBatches' is set to 'yes' and the total number of rows fetched is less
than 'FetchBatchSize', MATLAB shows a warning message and then fetches all the
rows. The message is: Batch size specified was larger than the number of rows fetched.