Creating a workflow

Workflows are packages that allow running a sequence of tasks, such as executing a SQL query or running a dataflow package. You can define the sequence and the conditions for executing a task--for example, a task can be executed only after the previous task was completed successfully.

To create a workflow:

  1. On the main menu, click Packages.
  2. Click New package.
  3. Optionally, set a name and/or description
  4. Select workflow option from Type dropdown
  5. Click  + Add Task button
  6. Choose task
  7. Add additional tasks as required
  8. Connect tasks to create sequence of task execution. Click the connect icon on the dotted line to set the execution condition:
    • On success (default) - task will be executed once the preceding task was executed successfully
    • On failure - task will be executed once the preceding task execution failed
    • On completion - task will be executed once the preceding task completed, regardless to the completion status (failed/succeeded)

Execute SQL task

  1. Select your DB connection
  2. Write the SQL query that should be executed, and select the query result type from Result type dropdown. You can test the query by clicking Test Query.
  3. You can assign the return value to a workflow variable.
  4. If there are at least two preceding tasks, choose task execution condition:
    • all preceding conditions evaluate to true (AND)
    • one of the preceding conditions evaluate to true (OR)

Run package task

  1. Select the package to run
  2. Optionally, you may edit the dataflow variables if you want to override the values with workflow variables.
  3. If there are at least two preceding tasks, choose task execution condition:
    • all preceding conditions evaluate to true (AND)
    • one of the preceding conditions evaluate to true (OR)

Using Variables in Workflows

User variables can be defined at the workflow package level and can be used for both the Execute SQL Task and the Run Package Task.

  1. Execute SQL Task

    • Variables can also be assigned values by the Execute SQL task. This is useful if you want to have dynamic values on your variable and use it later on.
    • When using variables in a SQL query, enclose the variable within curly brackets (i.e: '${var_name}').
    • Example of using a variable within the Execute SQL task query:

  2. Run Package Task

    • Workflow package level variables can be used to override dataflow level variables. Here's an example:

      Note: Take note that we address package variables regularly as $variable_name which is different from the way we used it on Execute SQL Task above.

    • If both workflow and dataflow variables have the same name, you will still have to assign the workflow variable to the dataflow variable.
    • If a task dataflow uses a variable that isn't defined at the dataflow level but is assigned a value at the workflow level, the dataflow task will use the workflow variable value.

Creating packages

  1. Creating a new package in New Xplenty
  2. Creating a workflow
  3. Working in the new package designer
  4. Validating a package
  5. Using components: Amazon Redshift Source
  6. Using components: Bing Ads Source
  7. Using components: Database Source
  8. Using components: Facebook Ads Insights Source
  9. Using components: File Storage Source
  10. Using components: Google Adwords source
  11. Using components: Google Analytics Source
  12. Using components: Google BigQuery Source
  13. Using components: Google Cloud Spanner Source
  14. Using components: MongoDB Source
  15. Using components: NetSuite Source
  16. Using components: Salesforce source
  17. Using components: Rest API Source
  18. Using components: Aggregate Transformation
  19. Using components: Assert Transformation
  20. Using components: Clone transformation
  21. Using components: Cross Join Transformation
  22. Using components: Distinct Transformation
  23. Using components: Filter Transformation
  24. Using components: Join Transformation
  25. Using components: Limit Transformation
  26. Using components: Rank Transformation
  27. Using components: Sort Transformation
  28. Using components: Union Transformation
  29. Using components: Window Transformation
  30. Using components: Sample Transformation
  31. Using components: Cube transformation
  32. Using components: Amazon Redshift Destination
  33. Using components: Database Destination
  34. Using components: File Storage Destination
  35. Using components: Google BigQuery Destination
  36. Using components: Google Spanner Destination
  37. Using components: MongoDB Destination
  38. Using components: Salesforce Destination
  39. Using components: Snowflake Destination
  40. Using Components: Rest API Destination
  41. Using and setting variables in your packages
  42. System and pre-defined variables
  43. Using pattern-matching in source component paths
  44. Using ISO 8601 string functions
  45. Using Expressions in Xplenty
  46. Xplenty Functions

Feedback and Knowledge Base