Data Moving Tool: Features

This page lists the major DMT features organized by categories. Click on features for more details.

 
Data Export Features
 
Data Load Features
 
Data Transformation Features
 
Scheduler Features
 
Run Pre-Conditions Features
 
Pre & Post Processes Features
 
Miscellaneous Features
 

Data Export Features

 
Export data from virtually any database

With DMT you can export data from any ODBC data source. This means that you can export data from virtually any database. No need for additional connectors or multiple dedicated database tools. All you need is the ODBC driver to the database you want to access.

ODBC drivers are widely available for recent database systems like Oracle, SQL Server, Sybase, IBM DB2 / Informix, MySQL, MS Access and many more but also for older ones like Visual FoxPro, Paradox, Dbase, etc.

This makes DMT particularly versatile, especially when you combine different data sources together for exporting and loading data. For instance you can export data from several different data sources (e.g. an Oracle database, a MS Access database and some flat files), and load it into several other different databases (e.g. an SQL Server database and a DB2 database). The possibilities are endless!

 
Use your own SQL statements to control exported data

Unlike many other ETL applications, DMT runs your own SQL statements to export data. This gives you total control and flexibility of what data you want to export and how.

 
Generate data files in various formats

When DMT exports data from a database, it stores the exported data into a data file. Data files can be used as is (e.g. for Excel reports, emails, backup, etc.) or used as source to load data into other databases.

DMT can generate the following data file formats:

FormatDMT VersionDescription
CSV1.1.5Text file where each field is separated by a coma
Tab Delimited2.0Text file where each field is separated by a Tab character
Delimited Text1.0Text file where each field is separated by a delimiter string.
The delimiter string can be any string of your choice.
HTML1.0Text file where data is placed in an HTML table. e.g.
<table>
   <tr><th>First Name</th><th>Last Name</th></tr>
   <tr><td>Joe</td><td>Blogs</td></tr>
   <tr><td>Paul</td><td>Smith</td></tr>
</table>
Excel1.0Excel files are read by DMT version 1.0.
To produce data files that can be read by Excel, you can generate CSV with the Text format.
Fixed Length-Fixed length files will be supported in a later version of DMT
 
Run Data Exports on demand at any time

All data exports can be run on demand at any time with just one click. This can be useful to test your exports or to run exports that don't need to be scheduled.

Note: All data exports can also be run automatically by the scheduler.

 
Unlimited number of exported rows

There are no limitations in the number of rows you can export with DMT. Only the trial version is restricted in the number of rows you can export.

 
Run Data Exports automatically via scheduler

All data exports can be run automatically by the scheduler. All exports have a set of scheduling parameters that control when they should be run by the scheduler. You can run an export at any chosen dates, times or interval.

Note: All data exports can also be run on demand at any time.

 
Support Pre-Conditions and Pre/Post Processes

To add logic and extend functionalities, all data exports can be given a set of run pre-conditions and pre or post processes.

 

Data Load Features

 
Load data into virtually any database

With DMT you can load data into any ODBC data source. This means that you can load data into virtually any database. No need for additional connectors or multiple dedicated database tools. All you need is the ODBC driver to the database you want to access.

ODBC drivers are widely available for recent database systems like Oracle, SQL Server, Sybase, IBM DB2 / Informix, MySQL, MS Access and many more but also for older ones like Visual FoxPro, Paradox, Dbase, etc.

This makes DMT particularly versatile, especially when you combine different data sources together for exporting and loading data. For instance you can export data from several different data sources (e.g. an SQL Server database, a DB2 database and some flat files), and load it into several other different databases (e.g. an Oracle database and an MS Access database). The possibilities are endless!

 
Load data files in various format

DMT can load data from one database to another as well as load data files directly into a database.

Note that data files can be read even if they haven't been generated by DMT.

DMT can read the following data file formats:

FormatDMT VersionDescription
CSV1.1.5Text file where each field is separated by a coma
Tab Delimited2.0Text file where each field is separated by a Tab character
Delimited Text1.0Text file where each field is separated by a delimiter string.
The delimiter string can be any string of your choice.
HTML1.0Text file where data is placed in an HTML table. e.g.
<table>
   <tr><th>First Name</th><th>Last Name</th></tr>
   <tr><td>Joe</td><td>Blogs</td></tr>
   <tr><td>Paul</td><td>Smith</td></tr>
</table>
Excel1.0Excel files are read by DMT version 1.0.
To produce data files that can be read by Excel, you can generate CSV with the Text format.
Fixed Length-Fixed length files will be supported in a later version of DMT
 
Link Loads to Exports

Data loads can be linked to data exports to transfer data directly from one database to another. When you link a load to an export, the load automatically uses the data generated by the export.

There is also an option in the scheduling parameters to run a linked load as soon as the export it's linked to is completed. This makes the Export/Transform/Load operations all bundled into one.

 
Data filtering (load conditions)

You might sometimes only want to load a subset of the data available in your data source. If you use DMT to export the source data, you can filter the data to be exported, but if you don't have any control on the source dataset, you can filter the data just before it's loaded.

For instance, if you have a list of part number coming from an external vendor into a data file, you can choose to load data only for parts where flag active = Y.

 
Load data into multiple tables using UK and FK

DMT can load data into multiple tables at the same time (even in multiple databases).

You can load the content of a data file into several target tables. When loading into several tables, you can decide which part of the data goes into which table (e.g. name fields into one table, address fields into another, order fields into another, etc).

The loaded data can be linked together using user keys and foreign keys.

 
Run Data Loads on demand at any time

All data loads can be run on demand at any time with just one click. This can be useful to test your loads or to run loads that don't need to be scheduled.

Note: All data loads can also be run automatically by the scheduler

 
Unlimited number of loaded rows

There are no limitations in the number of rows you can load with DMT. Only the trial version is restricted in the number of rows you can load.

 
Run Data Loads automatically via scheduler

All data loads can be run automatically by the scheduler. All loads have a set of scheduling parameters that control when they should be run by the scheduler. You can run a load at any chosen dates, times or interval.

Note: All data loads can also be run on demand at any time.

 
Support Pre-Conditions and Pre/Post Processes

To add logic and extend functionalities, all data loads can be given a set of run pre-conditions and pre or post processes.

 

Data Transformation Features

 
DMT Transformations (performed by DMT independently from the target database)

DMT can transform data before it's loaded into a destination table.

A transformation is an expression (a bit like an Excel formula) combining constants, functions, operators, Run Variables and columns available in the source data file.

A DMT transformation is an expression that uses DMT's predefined functions and operators. DMT fully evaluates the expression prior to loading data, and sends the result to the destination table.

The advantage of DMT transformations is that they are totally independent from the target database (i.e. a transformation is valid regardless of the target database).

 
Database transformations (performed by the target database via its dedicated functions)

DMT can let the target database perform transformations on the loaded data.

A transformation is an expression (a bit like an Excel formula) combining constants, functions, operators, Run Variables and columns available in the source data file.

A database transformation is an expression that uses the target database predefined functions and operators. DMT does not evaluate the expression and instead sends it as is to the destination table (after substituting parameters i.e. the values to load). The expression is fully evaluated by the target database.

The advantage of database transformations is that they are faster than DMT transformations and the target database might have more predefined functions and operators than DMT.

 

Scheduler Features

 
Run exports and loads Continually, Daily, Weekly, Monthly

All data exports and data loads can be run by the scheduler:

  • Continually: at a regular interval (e.g. every hour)
  • Daily: every 1 or more days
  • Weekly: every 1 or more weeks
  • Monthly: every specific date of the month (e.g. every 5th of the month)
 
Begin and End date/time, max num of runs, run at specific time

All data exports and data loads can be configured to be scheduled only for a certain period of time. You can define the beginning and the end of that period by choosing a precise data and time. Outside this time period, the export or load will not be run by the scheduler.

You can define a maximum number of times a data export or a data load can run. When that number is reached the export or load will no longer be run by the scheduler.

All data exports and data loads scheduled to run daily, weekly or monthly can be configured to run at a specific time of the day.

 
Run on specific days or months

All data exports and data loads scheduled to run continually, daily or weekly can be configured to run only on specific days.

All data exports and data loads scheduled to run monthly can be configured to run only on specific months.

 
Advanced feature: Run every 1st, 2nd, 3rd, 4th, 5th or last Mon, Tue, Thu, Fri, Sat, Sun of the month

Scheduling parameters have advanced features for data exports and data loads that run monthly. You're not limited to a specific day of the month.

You can for instance run an export or load on the:

  • 1st Wednesday of the month
  • 2nd Monday of the month
  • last Friday of the month
  • etc.
 
Advanced features: Run every last Day of the month

Scheduling parameters have advanced features for data exports and data loads that run monthly. You're not limited to a specific day of the month.

You can run an export or load on the last day of the month regardless of what that day is.

 

Run Pre-Conditions Features

 
Check the return of any SQL query before running exports or loads

Run pre-conditions are a powerful way to add logic to data exports and data loads. They allow you to trigger the run of an export or load only if specified conditions are met.

You can base a run pre-conditions on the result of an SQL query. i.e. Run an SQL query against any database and trigger or not the run of an export or load based on the result of this query.

This can be useful for instance to check that enough data is present before exporting. In the SQL pre-condition, query for the number of records in the source table and only start if the number is above a certain threshold.

 
Check the presence and/or content of a file before running exports or loads

Run pre-conditions are a powerful way to add logic to data exports and data loads. They allow you to trigger the run of an export or load only if specified conditions are met.

You can base a run pre-conditions on checking the presence or content of a file. i.e. Run an export or load only if a file is present or not and its content is whatever you specify.

This can be useful for instance if you get data files delivered randomly by external systems. You can check for the presence of a data file before loading it.

 

Pre & Post Processes Features

 
Execute SQL statements or stored procedures before or after exports or loads

Pre and Post Processes are a powerful way to add custom processing to data exports and data loads.

With Pre and Post Processes you can run SQL statements or execute stored procedures against any database.

For instance you can:

  • Perform preprocessing on some data before it's exported
  • Create a table before it's loaded
  • Delete previous data before new data is loaded
  • Log the execution of exports and loads into log tables
  • etc
 
Execute a command line before or after exports or loads

Pre and Post Processes are a powerful way to add custom processing to data exports and data loads.

With Pre and Post Processes you can execute an external program (i.e. run a command line).

For instance you can:

  • Perform some cleanup before or after an export or load
  • Zip data files after export
  • Unzip data files before loading
  • etc
 
Send emails to let people know an extract or load has taken place

Pre and Post Processes are a powerful way to add custom processing to data exports and data loads.

With Pre and Post Processes you can send emails to one or many recipients.

For instance you can:

  • Let people know that an export or load is being started or has completed
  • Customize the email (Title, Body, Recipients, etc)
  • Attach the data file associated with the export or load
  • Zip the attached data file
 

Miscellaneous Features

 
Organize exports and loads into groups and jobs

Data exports and data loads can be organized in groups and jobs. Jobs represent data exports or data loads performed by DMT and groups can contain jobs like folders and files in the windows explorer.

As well as an organizational purpose, groups also have additional functionalities according to their type:

  • A Schedule Group is group in which all the jobs are run sequentially when the group is run.
  • A Data Group is a group in which all the jobs use the same data file. exports populate the same data file and loads read the same data file.
  • A Schedule + Data Group is a combination of a schedule and data group. All the jobs within the group are run sequentially when the group is run, and they populate (for exports) or read (for loads) the same data file.
  • A Multi Table Group (Loads Only) is a group in which all the jobs read the same data file and can populate different destination tables (in the same or different databases).
  • A Logical Group is simply a logical entity containing jobs. It has no functionalities.
 
Run Variables usable almost anywhere (e.g. insert runtime information in a log table)

Run Variables are a very useful feature that gives you access to runtime information about a group or job while it's running.

They work like placeholders you can use throughout the application (mostly in SQL statements). When DMT runs a group or job and executes the SQL statement, it substitutes the run variable names with their respective values.

There are many predefined run variables, giving you access to a lot of runtime information: e.g.

  • Name of the Group or Job that's running
  • Name of the data file containing data exported or loaded by the group or job.
  • Number or rows exported or loaded by the group or job.
  • Etc.

You can also use your own custom run variables.

 
Expression editor/evaluator to facilitate editing/testing of complex load expressions

To enter complex expressions for Data Transformation or Data Filtering, you can use the expression editor.

The expression editor simplifies the task by giving you access to all the functions, operators and variables available in DMT so you don't have to type them.

The expression editor also allows you to evaluate your expressions with test values so you can make sure they work as intended.

 
Detailed customizable logs with Errors, Warnings, Run Events

DMT can produce detailed and customizable log files to track the running of groups and jobs.

As well as the standard information, you can individually select which type of errors, warnings or run events to log in the log files. This can be very useful for tracking down specific events or errors.

 
Powerful scheduler (windows service running in the background)

An important part of DMT is its powerful scheduler. All exports and loads can be scheduled to run at chosen dates, times or interval.

The DMT scheduler is located in a windows service running in the background. Once the DMT scheduler is started, you can close the DMT client and log off the computer. The scheduler keeps running in the background. This is typically designed to run on a server where no user is permanently logged on.