User`s guide

5 Using Visual Query Builder
5-20
results = runsqlscript(conn,'filename.sql','rowInc','m')
• If you are using the native ODBC interface to import large amounts of data, you do
not need to change these settings because the native ODBC interface always fetches
data in batches of 100,000 rows. You can still override the default batch size by
setting 'FetchInBatches' to 'yes' and 'FetchBatchSize' to a number of your
choice. Note that JVM heap memory restrictions do not apply in this case since the
native ODBC interface is a C++ API.
Will All Data (Size n) Fit in a MATLAB Variable?
This example shows how to estimate the size of data to import from a database.
It is important to have an idea of the size of data that you are looking to import from
a database. Finding the size of the table(s) in the database can be misleading because
MATLAB representation of the same data is most likely going to consume more memory.
For instance, say your table has one numeric column and one text column and you are
looking to import it in a cell array. Here is how you can estimate the total size.
data = {1001, 'some text here'};
whos data
Name Size Bytes Class Attributes
data 1x2 156 cell
If you are looking to import a thousand rows of the table, the approximate size in
MATLAB would be 156 * 1000 = 156 KB. You can replicate this process for a structure or
a dataset array depending on which data type you want to import the data in. Once you
know the size of data to be imported in MATLAB, you can determine whether it fits in a
MATLAB variable by executing the command memory in MATLAB.
A conservative approach is recommended here so as to take into account memory
consumed by other MATLAB tasks and other processes running on your machine. For
example, even if you have 12 GB RAM and the memory command in MATLAB shows 14
GB of longest array possible, it might still be a good idea to limit your imported data to
a reasonable 2 or 3 GB to be able to process it without issues. Note that these numbers
vary from site to site.
Will All of This Data Fit in the JVM Heap?
This example shows how to determine the size of the JVM heap.