. . .

CSV Module

images/download/attachments/61479007/db3-module-csv.PNG


Using the CSV module you can import or export your data using CSV formatted files.


Configuration

Configuration

Example

Description

Separator Character

;

Character used to separate your data entries.
Default value is ‘;’, a typical CSV entry woould look like username;password;phone;email

List Separator Character

|

Character used to separate entries in lists. This is used if one of the im- or exported fields contains a collection as value.
Default value is ‘|’, a typical list entry would look like [1|2|3], where the braces indicate the value as list.

Charset

UTF-8

Charset of your data. Note: MS Excel in a western country will use Windows-1252
Default value is "UTF-8".

DateFormat

EEE, d MMM yyyy HH:mm:ss Z

SimpleDateFormat used to import and export date values.
Default value is "EEE', 'dd' 'MMM' 'yyyy' 'HH:mm:ss' 'Z"


To use the CSV Module you need to attach it to your application. You can find the CSV Module in the Module Market within your Dashboard.

Usage via Dashboard

After setting the configuration parameters, you will be able to export or import data, which you can see in the dashboard data screen.

The CSV Module comes with a tool which allows you to export and import ZIP-files containing CSV Data in an easy dialog.

Export via Dashboard

To export your data via ApiOmat Dashboard simply go to Modules & UI and open the sub menu CSV.
Before starting the export of your data you are also able to adjust some filter options:

  • use the query to specifiy your export data selection (see Query for more details)

  • use the exportResources checkbox to enable the file and image export

  • select the needed classes from selection table at the bottom

  • If several module versions contain the same class then you need to select only one of them to reduce redundancy.
    Reason: the data objects aren't versioned in order to be able to load old data objects in new module versions.
    The actual exported data object will be the same independent of the module version, so it makes sense to define only one of the same classes to be exported.


images/download/attachments/61479007/CSV_export_dialog.png

After pressing the export button a ZIP-file will be received that contains one separated CSV-file for each class containing the data.

Import via Dashboard

To import data into your ApiOmat instance you need to create a ZIP-file containing the CSV files for each class you want to import.

The CSV files should be located within the ZIP's root directory; subfolders for data entries aren't allowed.

Each file that contains data of a class needs to match the following naming convention: <MODULENAME>$<MODULEVERSION>$<CLASSNAME>.csv (e.g. Basics$1.0.0$User.csv )

For more details of the import and export contents read the next sections of this documentation.

As soon as every file you want to import is ready just choose your ZIP file and press the File Upload button.

images/download/attachments/61479007/CSV_import_dialog.png

A message should appear to confirm the import.

Usage via REST

Additionally, you may use the REST interface to export or import CSV data.

Export via REST

To support the export of very large data entries, each field value of one data entry is limited to a size of 100.000 characters. This value can be increased globally in the yambas properties file.
In that case, please configure the value for yambas.limits.maxStringSizeOnExport.


The usual limitation of a maximum result set size is not taken into account when exporting data. Therefore, an export of a large dataset could last several minutes. You can limit the result set with the limit/order query operator (see example below).


cURL

# exporting all data of the specified application
curl https://apiomat.org/yambas/rest/modules/csv/spec/YOUR_APPLICATION_NAME \
-u YOUR_EMAIL:YOUR_PASSWORD > export.zip

You may also filter classes to export using the ‘classesToExport’ query parameter and a comma separated list of classes which you want to be exported:cURL

# exporting data of specific non-module classes
curl https://apiomat.org/yambas/rest/modules/csv/spec/YOUR_APPLICATION_NAME?classesToExport=car,tire \
-u YOUR_EMAIL:YOUR_PASSWORD > export.zip


If you have multiple modules with classes that have the same name, you need to prepend the classnames with the module name and module version separated by a dollar sign:cURL

# exporting data of specific module classes
curl https://apiomat.org/yambas/rest/modules/csv/spec/YOUR_APPLICATION_NAME?classesToExport=MyModule$1.0.0$car,MyModule$2.0.0$tire \
-u YOUR_EMAIL:YOUR_PASSWORD > export.zip

You can also filter the data that you want to be exported by using a query:cURL

# exporting data that match a specific query
curl https://apiomat.org/yambas/rest/modules/csv/spec/YOUR_APPLICATION_NAME?q=lastModifiedAt>date(1413842400000) \
-u YOUR_EMAIL:YOUR_PASSWORD > export.zip

Or limit the resultset:cURL

# exporting data limited to the amount of returned entries and specified by an offset
curl https://apiomat.org/yambas/rest/modules/csv/spec/YOUR_APPLICATION_NAME?q=limit%20100&offset%2010 \
-u YOUR_EMAIL:YOUR_PASSWORD > export.zip

Usually, images are exported as URLs in the CSV. If you want them to be contained as files directly in the export result, simply add the ‘exportResources’ parameter as follows:cURL

# exporting data and file resources
curl https://apiomat.org/yambas/rest/modules/csv/spec/YOUR_APPLICATION_NAME?exportResources=true \
-u YOUR_EMAIL:YOUR_PASSWORD > export.zip

Import via Rest

Assuming import.zip containing all classes as csv data, the import command would look like:cURL

# importing data to a specific application
curl -X POST https://apiomat.org/yambas/rest/modules/csv/spec/${YOUR_APPLICATION_NAME} \
-u ${YOUR_EMAIL}:${YOUR_PASSWORD} --data-binary @import.zip -H "content-type:application/zip" -H "X-apiomat-system: ${SYSTEM}" \
-H 'X-apiomat-apikey: ${YOUR_API_KEY}' \

The next section contains some more detailled information about the import and export contents.

Detailled Import and Export contents

Each exported dataset of your own classes contains an ID field, which is the ID from ApiOmat database. Do NOT change this field if you want to update the dataset.

If you create new entries, leave this field empty. Self-created IDs will lead to import errors.

If you want to give your new data entries an ID, you can use an arbitrary (but unique) value in the foreignId field. You can even use this foreignId if you want to set references from other classes.

Import files, images and dates

If you want to import files or images, simply place them in the ZIP-file and write their file name (beginning with file://) into the CSV.

Example Structure:

  • myImport.zip

    • myImage.png

    • MyApp$1.0.0$MyClass1.csv

The column in your MyApp$MyClass1.csv file containing the images should then contain file://myImage.png. Subdirectories for files and images are also supported.

If you want to use dates, their format must follow the Java date format LONG or SHORT. All dates should fulfill the configured dateFormat of the CSV module.

Export embedded objects

Export and import of embedded objects is possible, too. Embedded data is placed in a separate file with an .embedded prefix on the filename. That is, if the MyClass1 in the example above has an embedded class named MySub, the exported files would look like

  • MyApp$1.0.0$MyClass1.csv

  • MyApp$1.0.0$MyClass1.embedded.csv

whereas the MyApp$1.0.0$MyClass1.embedded.csv would contain the usual data like any other non-embedded class structure.