Next >> << Prev

11. Advanced Parameters for Exports and Loads

The Advanced Parameters form is where you specify group and job additional settings and parameters to further adjust their behavior. It's accessible from any group or job by clicking the Advanced Parameters link in the group or job parameters section.

Form Tabs

The form is organized in 3 tabs each containing different kind of parameters.

Advanced Params Tabs

Four Different Forms

Each form is different depending on the type of object you're editing:

Common Fields for all Advanced Parameter Forms

The Data File Backup and Delete fields are available on the four different kind of advanced parameter forms.

Backup Data File: When this box is checked the Data File is automatically backed up. For exports, the data file is backed up just before generating a new data file. For loads the data file is backed up just after the data file has been loaded.

Delete Data File: When this box is checked the Data File is automatically deleted. For exports, the previous data file (i.e. one with the same name in the same location if exists) is systematically deleted before generating a new one. For loads the data file is deleted just after the data file has been loaded.

Note: Backup folders are created automatically in the folder where data files are located:

  • ExportBackup: To store export data files
  • LoadBackup: To store load data files
 

11.1. Advanced Parameters for Export Groups


Advanced Parameters (Export Groups)


The form contains one area for each type of group:

Schedule Group: There are no advanced parameters for schedule groups.

Data Group

Logic Group: There are no advanced parameters for logic groups.

Data File Backup and Delete: See Common Fields.

Temp Data File Disposal: This option is only available for Export Data Group.

Export data groups use a temporary data file populated by all the jobs within the group. When a group runs successfully the temp data file is automatically deleted. The "Temp Data File Disposal" field indicates what to do when the group run fails. You can choose one of 2 options:

  • Delete: (default) This option tell DMT to delete the temp data file. This is the same behavior that for a successful run.
  • Move to "Errors" Folder: This option tell DMT to move the temp data file to the Errors forder so it can be examined for errors.

Note: The LoadError folder is created automatically in the folder where data files are located.

 

11.2. Advanced Parameters for Export Jobs


Advanced Parameters (Export Jobs)


Data File Backup and Delete: See Common Fields.

DB Timeout (sec): Specifies the database timeout for the export job's SQL query. This value is set by default to the timeout value defined in the Setting/General Tab.

Auto Trim: This field indicates whether the content of each exported column should be automatically trimmed before being placed in the data file. You can choose from the following 4 options:

  • None: (default) Exported content is placed as is in the data file.
  • Left + Right: Content of each exported column is trimmed of leading and trailing spaces.
  • Left: Content of each exported column is trimmed of leading spaces.
  • Right: Content of each exported column is trimmed of trailing spaces.

Note: The auto trim functionality is applied on all exported fields. This can slow down the performance if you export a large number of fields and/or rows. In that case it might be more efficient to use database trim functions directly in your SQL query to only trim specific fields.

When Job Returns No Data: This group of fields indicate what to do when an export job's SQL query returns no data.

  • Generate (empty) Data File: The data file is generated even if there is no data in it.
  • Job "Fails": Make the job fail. This is useful if you want to easily detect exports that didn't produce any data.
 

11.3. Advanced Parameters for Load Groups


Advanced Parameters (Load Groups)


The form contains one area for each type of group:

Schedule Group: There are no advanced parameters for schedule groups.

Data Group

Logic Group: There are no advanced parameters for logic groups.

Data File Backup and Delete: See Common Fields.

No other specific parameters are available for Load Groups.

 

11.4. Advanced Parameters for Load Jobs


Advanced Parameters (Load jobs)


Data File Backup and Delete: See Common Fields.

DB Timeout (sec): Specifies the database timeout for the load job's SQL query (to load the data). This value is set by default to the timeout value defined in the Setting/General Tab.

Auto Trim: This field indicates whether the content of each data file column should be automatically trimmed before being loaded in the destination table. You can choose from the following 4 options:

  • None: (default) Data file content is loaded as is in the destination table.
  • Left + Right: Content of each data file column is trimmed of leading and trailing spaces.
  • Left: Content of each data file column is trimmed of leading spaces.
  • Right: Content of each data file column is trimmed of trailing spaces.

Note: The auto trim functionality is applied on all data file columns. This can slow down the performance if the data file contains a large number of columns and/or rows. In that case it might be more efficient to use database or DMT trim functions directly in the load mapping to only trim specific fields. You can use Trim functions in the Value column of the Load Mapping.

Job "Fails" when No Data is Loaded: When this box is checked, the job fails if no data is actually loaded in the destination table. Note this could be because there is no data in the data file or because no rows have actually been loaded due to errors or load conditions (see Load Mapping).

Allow Loading into System Tables (not recommended): When this box is checked, the dropdown list of destination tables to choose from in the Load Mapping section shows all the tables including the system ones. It's recommended not to check this box so only user tables are shown.

Incomplete Data: This field indicates what to do in case a row with incomplete data is encountered in the data file. A row with incomplete data is a row where not all the expected columns are present. You can choose from the following 4 options:

  • Error + Stop Load: An error is generated (in the log file) and the load job is stopped immediately.
  • Error + Skip Line: An error is generated (in the log file), the incomplete line is skipped and the load continues.
  • Warning + Skip Line: A warning is generated (in the log file), the incomplete line is skipped and the load continues.
  • Warning + Load Line with NULLs: A warning is generated (in the log file), the incomplete line is loaded with NULL for the missing fields, and the load continues.

Note: As explained in the Error Handling section a job can continue running even if errors or warning are generated along the way.

Load Range: You can choose to load only a certain section of the data file in the destination table. This field indicates which section to load. You can choose to start loading data from the start of the data file or from a specific line. You can choose to keep loading data until the end of the data file or until a specific line.