Skip to main content

Datasets

Datasets are “Data Containers,” similar to tables, though they do not actually contain data. Instead, they contain the result of queries or extractions from another system (e.g. a call to a web service). Datasets are automatically filled with data based on what is needed for the integration by the engine itself. They access the required data using connectors. Therefore, each Dataset must be created on a connector - no Dataset can exist without an associated Connector. The behavior exhibited by a Dataset. and the way in which it needs to be configured. depends on the type of connector with which the dataset is associated. For this reason, Integration Framework automatically loads the Dataset configuration window specific to the type of connector and the connector's direction (INPUT/OUTPUT).

Note: Datasets can be used in Integration Rules, as well as being combined, added and merged in order to generate other complex Datasets for advanced system integration.

Caching System Support​

Datasets can support a caching system by defining:

  • Cache enabled: Indicates whether or not the caching system is enabled
  • Cache timeout: Indicates the number of seconds of cache validity. After the time-out expiration, the cache is rebuilt at the first call of the Dataset.
Connection errors and communication issues​

Datasets can support and manage connection errors and communication issues by using the following parameters:

  • Call timeout: The maximum amount of seconds to wait for a reply from the external system. After this time has passed, the Dataset exits with an error.
  • Behavior on error: The resulting Dataset policy management when an error is encountered (regardless of the error type) during an external system call. Default policies are:
  • EXCEPTION FORWARD: An exception is forwarded to the client or consumer.
  • EMPTY DATA: An empty Dataset is returned to the client or consumer.
  • CUSTOM DATA: A custom Dataset is returned to the client or consumer.

Datasets can be parametric in order to call the same function or perform the same query, against different input values. When parametric, the Dataset parameters are managed through a list that can either be empty (no parameters) or contain an unlimited numbers of parameters. Each Dataset parameter should include the following:

  • Name
  • Type (string, double, integer… see section 2.1.3.6 – Parameters for the complete list)
  • Format*
  • Default value

Note: *Format is managed by the advanced Formatting, casting and data transformation service. By enabling this option, Integration Framework allows to define a set of advanced rules to manage parameter formats (by casting, formatting, scaling, etc.) in order to fit integration needs.