Scheduling Package Execution

Use the scheduler to execute packages periodically starting at a specified date and time. The packages will be executed as scheduled, using an existing cluster that fits the scheduled cluster size or if one doesn't exist, a cluster will be provisioned automatically with the number of specified nodes. By default, the cluster is taken down as soon as package execution is completed.

Your schedule list can be viewed in the my schedules window. Any schedule can be disabled or enabled from that window.

To create a new schedule
  1. Click my schedules on the Xplenty menu to open the my schedules window.
  2. Click new schedule to open the new schedule window.

  3. Enter a Name for the schedule and optionally a Description.
  4. Optionally, change the Starts on date and time to start schedule package execution. If the schedule repeats every day or week, subsequent executions will start at this time. If the schedule repeats every hour, subsequent executions will start at the specified minute after the hour. Note that the time is in UTC.
  5. Change Repeat every to the periodicity of package execution.
  6. By default, schedules execute their packages regardless of whether previous jobs executed by the same schedules are running. Uncheck Allow execution overlapping if you want the schedule to execute jobs only after previous job executions have completed.
  7. Slide the Cluster Size slider to the number of cluster nodes to use for package execution.
  8. By default, the cluster will terminate after 1 minute of inactivity. If the package execution and schedule recurrence is lower than 1 hour, we recommend to turn automatic cluster termination off so the cluster can be reused.
  9. Set Re-use strategy:
    • Any cluster created by this schedule - use a cluster created by this schedule if one is available. Otherwise, create a cluster.
    • Any similar cluster (default) - use any existing cluster so long as it's at least as big as you set in the schedule's cluster size. Otherwise, create a cluster.
    • Never - create a new cluster every time the schedule is running. 
  10. Click add package to add at least one package to execute.
  11. Choose a package from the list and click set variables.
  12. Set the value for any user variables or system variables and click add.

    Note: Variable values are expressions that are useful to calculate relative datetime values which can be very useful in your scheduled jobs. For example:
    ToDate(ToString(SubtractDuration(CurrentTime(),'P1D'),'yyyy-MM-dd')) - returns a datetime value of Yesterday midnight
    ToString(SubtractDuration(CurrentTime(),'P1D'),'yyyy/MM/dd') - returns a string in the form of yyyy/mm/dd to use in a path for yesterday's data.
    AddDuration(ToDate('2000-01-01'),REPLACE(' PnM','n',(chararray)MonthsBetween(CurrentTime(),ToDate('2000-01-01')))) - returns a datetime value of the first day of the month

  13. Optionally add additional packages.

    Note: The variable values for a package in the schedule override the package defaults.
  14. Click Enabled on to enable the schedule.
  15. Click create to create the schedule.

To view and maintain schedules

In the my schedules window you can see all of your schedules with run information. From each schedules you can perform the following actions:

  • Click on the schedule to open the edit schedule window, with all details available to edit
  • Enable or disable the schedule
  • Manually run the schedule
  • Create a duplicate of the schedule
  • Delete the schedule

Feedback and Knowledge Base