Data Transforms

Introduction

Amaxa supports applying transforms to data values during both extraction and load operations. Transforms are specified at the field level in the operation definition file.

version: 2
plugin-modules:
    - test_transforms
operation:
    - sobject: Account
    fields:
        - field: Name
          transforms:
            - uppercase
            - name: suffix
              options:
                suffix: "-CLIENT"
    extract:
        all: True

In this example, the Name field on Account has two transforms applied to it: uppercase and suffix. Transforms are applied sequentially, and can take options as shown above. Loading an Account with the Name “Acme Products” (in its CSV file) via the operation definition shown above would yield an Account named “ACME PRODUCTS-CLIENT” in the target Salesforce org.

Transforms are bidirectional: they run on both load and extract.

Amaxa ships with five transforms:

  • uppercase uppercases the field value.
  • lowercase lowercases the string value.
  • strip removes leading and trailing whitespace.
  • prefix adds a prefix string (specified with the options key prefix).
  • suffix adds a suffix string (specified with the options key suffix).

Custom Transforms

Amaxa also supports custom transforms. To implement a custom transformation, create a Python module containing a class that subclasses amaxa.transforms.TransformProvider. Subclasses must populate the transform_name class attribute and override two methods, _get_transform() and get_options_schema().

_get_transform() accepts a dict representing the describe of the configured field and a dict containing the user-specified options, if any. It returns a callable that will be invoked with a single parameter, the field value, each time the transform is used.

get_options_schema() returns the schema by which the user-specifiable options will be validated in the operation definition. This is a Cerberus schema that should validate a dict.

Transforms may optionally override _validate_field(), which also accepts a dict representing the describe of the configured field. This method should raise a TransformException if the transform cannot be applied to this field for any reason. The default implementation requires that the field’s soapType attribute be one of the values in the class attribute allowed_types, a string list.

Before being used in an operation, custom transforms must be loaded by specifying them in a plugin-modules list at the top level of the YAML configuration. Specify the name of the containing module, not the transform class; Amaxa will automatically discover transforms in those modules. Modules may be located in the current working directory or anywhere in the Python search path.

A complete example of using a custom transform is included in Amaxa’s repository in the assets/test_data_transforms directory.