Exceptions

Engine raises exceptions in executions of flows when errors occur. Flows can catch exceptions and implement logic to handle errors.

Use Cases

Engine uses standard Python exceptions to signal errors. Exceptions can be raised

• by Python. See Built-in Exceptions.
• by third-party modules used in a flow. See the documentation of the module you are using.
• by Engine.

This article describes the exceptions raised by Engine.

Exception Hierarchy

• EngineException

  Parent of all Engine specific exceptions.

  • InternalError

    An internal error occurred. If the error persists, please report a bug to info@cloudomation.com.

  • PermissionDenied

    The execution does not have sufficient permissions to perform the action.

  • LifecycleSignal

    A signal to end an execution. Do not use this class directly, but one of the subclasses.

    note

    It is not possible to catch any of the LifecycleSignal exceptions.

    • ExecutionCancel

      A signal to end an execution with status ENDED_CANCELLED.

    • ExecutionError

      A signal to end an execution with status ENDED_ERROR.

    • ExecutionSuccess

      A signal to end an execution with status ENDED_SUCCESS.

  • ResourceNotFoundError

    A requested resource does not exist.

  • DependencyFailedError

    A dependency failed.

  • TimeoutExceededError

    A timeout occurred.

    • LockTimeoutError

      A timeout occurred while waiting for a lock.

    • DependencyTimeoutError

      A timeout occurred while waiting for a dependency.

    • MessageResponseTimeoutError

      A timeout occurred while waiting for a message response.

  • MissingInputError

    An input was required by a connector but not provided.

  • InvalidInputError

    An input was not of an accepted type.

  • ExtraInputError

    An input was provided to a connector but not needed.

  • SchemaValidationError

    A validation of data against a schema failed.

Unhandled errors

When an exception is raised in an execution and it is not handled Engine will end the execution with the status ENDED_ERROR and provide a traceback in the status_message field.

example

This example shows how an unhandled exception is presented to the user.

import flow_api

def handler(system: flow_api.System, this: flow_api.Execution, inputs: dict):
    system.file('this-file-does-not-exist').get_text_content()
    return this.success('all done')

The execution of this flow will end with the status ENDED_ERROR and an error message:

Traceback (most recent call last):
  File "my-flow (c379ba18-51ea-4a49-a84b-80c2987b177e)", line 4, in handler
flow_api.exceptions.ResourceNotFoundError: file name this-file-does-not-exist

Handling errors

Flows can catch exceptions and implement logic to deal with errors.

tip

When developing automation logic we recommend to anticipate potential errors and implement error handling accordingly.

example

This example shows how a flow can handle an exception.

import flow_api

def handler(system: flow_api.System, this: flow_api.Execution, inputs: dict):
    try:
        content = system.file('report.txt').get_text_content()
    except flow_api.ResourceNotFoundError:
        # the file was not found. Let's create it!
        content = 'initial content'
        system.file('report.txt').save_text_content(content)
    # at this point we can be sure that the file exists.
    return this.success('all done')

Special case: PicklingError

Due to the nature of Engine, all initialized objects in your flow script are pickled whenever a flow_api method is called. This means that all objects that exist at the time of the call must support pickling.

example

import genson

import flow_api

def handler(system: flow_api.System, this: flow_api.Execution, inputs: dict):
    # the SchemaBuilder object from the third party module genson can't be pickled
    builder = genson.SchemaBuilder()
    # here you can do various operations with the builder

    # a PicklingError will be raised when calling the log method
    this.log('this is a log')

    return this.success('all done')

When you run the flow you get the following error message:

Failed to pickle your execution:

All local objects must support the pickle interface when calling Engine through a flow_api method.
At least one of your local variables is not supported.
Please inspect the following traceback to find out which one:

Traceback (most recent call last):
  File "sandbox/sandbox_main.py", line 338, in src.sandbox.sandbox_main.Sandbox.send_command_to_engine
_pickle.PicklingError: Can't pickle <class 'genson.schema.builder.SchemaBuilderSchemaNode'>: attribute lookup SchemaBuilderSchemaNode on genson.schema.builder failed

Since you cannot catch this exception by simply using a try and except block (and even if you could you wouldn't be able to use any flow_api methods for the remainder of your flow script), you need to use a workaround.

Setting the object to None after it fulfilled its purpose but before the method is called avoids the problem.

important

This workaround does not work when debugging an execution (because the debug mode calls the flow_api internally at every line of execution).

When debugging, your script cannot contain any objects that cannot be pickled.

example

import genson

import flow_api

def handler(system: flow_api.System, this: flow_api.Execution, inputs: dict):
    # the SchemaBuilder object from the third party module genson can't be pickled
    builder = genson.SchemaBuilder()
    # here you can do various operations with the builder

    # setting the object to None avoids pickling issues
    builder = None

    this.log('this is a log')

    return this.success('all done')

note

You can read more about pickling here.

Cleaning up

Flows might want to clean up resources they created even when an error occurs. The finally block can be used to do just that.

Consider an execution which launches a cloud-vm instance, then runs some script on the instance and subsequently deletes the cloud-vm instance.

example

Without a finally block. In case of an error, no clean-up is done.

import flow_api

def handler(system: flow_api.System, this: flow_api.Execution, inputs: dict):
    # execute a child execution which launches a cloud-vm
    this.flow('create-cloud-vm')

    # use the cloud-vm
    this.connect(
        'cloud-vm',
        script='sleep 10; exit 42'  # fail after 10 seconds
    )

    # execute a child execution which removes the cloud-vm
    this.flow('remove-cloud-vm')

    return this.success('all done')

What will happen:

1. our execution will start a child execution of the flow create-cloud-vm
2. when the child execution create-cloud-vm ends successfully our execution will continue
3. our execution will start a child execution of the connection cloud-vm
4. when the child execution cloud-vm fails a DependencyFailedError will be thrown in our execution
5. since our execution does not handle the exception, it will end with ENDED_ERROR
6. the status message of our execution will contain the traceback of the error:

Traceback (most recent call last):
  File "my-flow (a3d9d997-f355-47dd-8ba4-9b702b6dc1ba)", line 10, in handler
flow_api.exceptions.DependencyFailedError: dependency SSH (00891f5b-2fb2-4950-a3fa-4a887ea0279c) did not succeed

caused by:

return code: 42

warning

The flow remove-cloud-vm was never started. The cloud-vm will continue running and incur costs.

A better approach places the cleanup in a finally block.

example

Cleanup in a finally block. Even in case of an error or cancellation of the execution the cleanup will run.

import flow_api

def handler(system: flow_api.System, this: flow_api.Execution, inputs: dict):
    # execute a child execution which launches a cloud-vm
    this.flow('create-cloud-vm')

    try:
        # use the cloud-vm
        this.connect(
            'cloud-vm',
            script='sleep 10; exit 42'  # fail after 10 seconds
        )

    finally:
        # execute a child execution which removes the cloud-vm
        this.flow('remove-cloud-vm')

    return this.success('all done')

What will happen:

1. our execution will start a child execution of the flow create-cloud-vm
2. when the child execution create-cloud-vm ends successfully our execution will continue
3. our execution will start a child execution of the connection cloud-vm
4. when the child execution cloud-vm fails a DependencyFailedError will be thrown in our execution
5. the code of the finally block is executed
6. our execution will start a child execution of the flow remove-cloud-vm
7. when the child execution remove-cloud-vm ends successfully our execution will continue
8. since the DependencyFailedError was not caught by an except block, it will be re-thrown and our execution will end with ENDED_ERROR
9. the status message of our execution will contain the traceback of the error:

Traceback (most recent call last):
  File "my-flow (2f1b49c2-29a7-4b17-a51d-c31862d33c47)", line 9, in handler
flow_api.exceptions.DependencyFailedError: dependency SSH (d878bbaf-f1a7-439b-b504-dbe78ec0f96b) did not succeed

caused by:

return code: 42

note

Although the execution ends with the same error message, it started the flow remove-cloud-vm which will clean up the resources which were allocated.

It is also possible to do both, handle the error and ensure a cleanup.

example

Handle an exception and cleanup in a finally block.

import flow_api

def handler(system: flow_api.System, this: flow_api.Execution, inputs: dict):
    # execute a child execution which launches a cloud-vm
    this.flow('create-cloud-vm')

    try:
        # use the cloud-vm
        this.connect(
            'cloud-vm',
            script='sleep 10; exit 42'  # fail after 10 seconds
        )

    except flow_api.DependencyFailedError as ex:
        # an error occurred. let's send a notification
        system.user('operator').send_mail(
            subject='error notification',
            html=(
                f'''
                <h1>cloud-vm script failed</h1>
                <pre>{str(ex)}</pre>
                '''
            ),
        )

    finally:
        # execute a child execution which removes the cloud-vm
        this.flow('remove-cloud-vm')

    return this.success('all done')

What will happen:

1. our execution will start a child execution of the flow create-cloud-vm
2. when the child execution create-cloud-vm ends successfully our execution will continue
3. our execution will start a child execution of the connection cloud-vm
4. when the child execution cloud-vm fails a DependencyFailedError will be thrown in our execution
5. the code in the except flow_api.DependencyFailedError is executed
6. our execution sends a notification to an Engine user called operator
7. the code of the finally block is executed
8. our execution will start a child execution of the flow remove-cloud-vm
9. when the child execution remove-cloud-vm ends successfully our execution will continue
10. the exception was handled, so our execution will continue normally and execute the code after the try-except-finally block
11. our execution will end with ENDED_SUCCESS and the status message "all done"

tip

When an execution catches an exception it is sometimes better to let it end with the status ENDED_ERROR. This is especially important when the execution is used as a dependency of other executions. This way the parent execution knows that there was an error and can react accordingly.

Learn More

Flows

Savepoints