Version: 12 - TBD

Accessing and Manipulating Records

All content that is stored in Engine is stored in the form of records. This includes flow scripts, files, messages, users, executions and anything else you or the system create on the platform. All records can be accessed and manipulated with the same methods.

This section describes three methods for creating, accessing, manipulating, and deleting records. More detailed information can be found in the REST API documentation, and the Flow-API documentation.

Creating

Via the User Interface

Press "+ Create" and select the type of record you like to create.

The create button

The new record is opened in the main section.

Via the REST API

Send a POST request to:

https://<your-workspace-name>.cloudomation.com/api/latest/<record-type>

The JSON payload should contain all fields you wish to set.

Via the Engine Flow-API

import flow_api

def handler(system: flow_api.System, this: flow_api.Execution, inputs: dict):
    system.<record-type>('name of the new record').save()
    # for example:
    system.setting('my-new-setting').save()
    # you can set fields of the new record:
    system.setting('my-new-setting').save(value=42, description='some number')

Please note, that .save() is necessary for the record to be created.

danger

The save method overwrites existing records with the same name and type, unless they're read-only. If that is not intended, you can first check for the existence of a record with the same name and type, before calling the save method.

Require an existing record (update-only)

By default save upserts: when you save by name and no matching record exists, it is created. If you mean to update an existing record and want to avoid accidentally creating a new one (for example after a typo in the name), pass allow_upsert=False:

# fails with a not-found error instead of creating a new setting
system.setting('must-already-exist').save(value=42, allow_upsert=False)

The same flag is available on the REST API and GraphQL (query parameter / argument allow_upsert, default true) and on the MCP create_record / update_record tools.

warning

Engine resource names must not start or end with a blank. Engine will implicitly remove any leading or trailing blanks from resource names.

Scoped default field values with `default_fields`

Sometimes you want the same field values applied to every record you create inside a block of code, without repeating them at each call site. The default_fields context manager scopes a set of default field values to everything created while it is active:

import flow_api

def handler(system: flow_api.System, this: flow_api.Execution, inputs: dict):
    with system.default_fields(description='created by the import job'):
        system.setting('setting-a').save(value=1)
        system.setting('setting-b').save(value=2)
        # both settings get description='created by the import job'

The equivalent module-level form flow_api.default_fields(...) is also available.

Key behaviours:

Applies to record creation only. Defaults are injected into save() on the create path, add_*() creations, and duplicate(). Updates and saves that resolve to an already-existing record are left untouched.
Generic across record types. A default is applied only to record types for which it is a writeable field; it is silently skipped for types that do not have that field, so you can set, for example, track_in_git=False for a whole block regardless of the mix of record types created in it.
Explicit values always win. Precedence is: an explicit keyword at the call site > the innermost active default_fields block > an outer block > the platform default.

Callables resolve per record. If a default value is a zero-argument callable, it is invoked once for each record created, so it can produce a fresh value every time:

import uuid

with system.default_fields(
    name=lambda: f'scratch-{uuid.uuid4()}',
    track_in_git=False,
):
    a = system.setting().save()   # name e.g. scratch-4f3c...
    b = system.setting().save()   # name e.g. scratch-9a71... (different)

Nestable. Nested blocks merge their defaults with the enclosing block; the inner value wins on conflict.
Isolated per execution. The scope lives on the running execution, so concurrent executions never see each other's defaults.

note

Pausing an execution (for example on a wait) inside a default_fields block whose defaults include a callable is not supported, because callables cannot be serialised across a pause/resume. Plain (non-callable) default values are unaffected. In practice the tight, synchronous create loops that default_fields is meant for do not pause.

Listing

Via the User Interface

Switch to the workspace dashboard by clicking the Engine logo in the menu.
Scroll down to the resource tabs.

Via the REST API

Send a GET request to:

$ curl https://\<your-workspace-name>.cloudomation.com/api/latest/\<record-type>

You can use filter expressions:

$ curl 'https://\<your-workspace-name>.cloudomation.com/api/latest/\<record-type>?filter={"field":"name","op":"like","value":"my-%"}'

Via the Engine Flow-API

warning

Listing via the Flow-API will return all records which exist when the listing starts. This means when records are created during the listing, they will not be included in a currently running listing. Also, when records are deleted during the listing, they are still returned. Make sure to handle the ResourceNotFoundError while iterating over a listing result.

Use the following method:

import flow_api

def handler(system: flow_api.System, this: flow_api.Execution, inputs: dict):
    # for record in system.<record-type>s():
    #    # do something with "record"

    # for example:
    this.log(all_webhooks=list(system.webhooks()))
    # you can use filter expressions
    this.log(all_enabled_webhooks=list(system.webhooks(filter_={
        'field': 'is_enabled',
        'op': 'eq',
        'value': 'True',
    })))

important

The maximum number of records that are returned with one call is 1000. Therefore, if you have more than 1000 records of a specific record type you need to use multiple iterations combined with offset and order to list all records.

The parameters for system.<record-type>s are as follows:

Parameter	Description	Default value
`fields`	Which fields to return (string of comma-separated field-names).	None
`limit`	How many records to return.	1000
`offset`	How many records to skip before returning records.	0
`filter_`	Filter to limit the results.	None
`order`	How to order the records before applying limit and offset (string of comma-separated field-names).	None
`partition_by`	Return the top N records per distinct value of a field in a single query. See `partition_by`.	None
`allow_normal`	If to include normal records in the response (That is records which are not deleted).	`True`
`allow_deleted`	If to include deleted records in the response.	`False`

`filter_`

The filter can either be a basic filter, consisting of 3 key-value pairs:

field: the field to match against
op: the operation used for matching, possible values: eq, neq, like, notlike, lt, gt, lte, gte, set, unset, in, notin
value: match to this value, mutually exclusive with value_field
value_field: match the field of this record, mutually exclusive with value

note

A value still needs to be specified when using the set or unset operation, even though it has no effect.

note

The field can map to other records. The following syntax is used to map to another record:

<own field>--<other record>.<join field>:<filter field>

or it can contain a collection of filters using the and or or key:

and: every filter provided in the list must be True
or: at least one filter provided in the list must be True

example

Filter resources which were created by either of the two identities, given their name. Be aware that multiple identities can share the same name (executions for example) and that thus this example might include more records than anticipated.

{
  "or": [
    {
      "field": "created_by--identity.id:name",
      "op": "eq",
      "value": "some-user"
    },
    {
      "field": "created_by--identity.id:name",
      "op": "eq",
      "value": "some-other-user"
    }
  ]
}

The previous example could also be rewritten using the in operation:

{
  {
    "field": "created_by--identity.id:name",
    "op": "in",
    "value": ["some-user", "some-other-user"]
  },
}

Filtering by a related subrecord (`exists`)

The --/: syntax above walks a forward foreign key (this record points at another record). To go the other way — match a record by the existence of a related subrecord row that points back at it — use the exists node:

{
  "exists": {
    "relation": "record_metadata",
    "where": {
      "and": [
        { "field": "key", "op": "eq", "value": "environment" },
        { "field": "data", "op": "eq", "value": "production" }
      ]
    }
  }
}

This matches every record that has at least one record_metadata row with key = "environment" and data = "production".

relation: the name of the related subrecord type. Any subrecord of the record's own type hierarchy works — for example record_metadata (metadata attached to any record) or role_permission (permissions of a role). An unknown relation is rejected as malformed input. The error message lists the relations available for the record type you are listing.
where (optional): a nested filter — the same grammar as this document describes, so it composes and/or/not and even further exists nodes. It is evaluated against the columns of the related subrecord (for record_metadata those are key and data). Omit where entirely to match records that have any row of that relation.

Wrap the node in not to match records that have no such related row:

{ "not": { "exists": { "relation": "record_metadata",
                       "where": { "field": "key", "op": "eq", "value": "environment" } } } }

The nested predicate honours the related subrecord's own read permissions, so exists never reveals the presence of rows an identity is not allowed to read.

note

exists currently targets subrecord relations. data on record_metadata is a JSONEncodedData field, so {"field": "data", "op": "eq", "value": true} compares against the stored JSON value (a boolean, string, number or object) directly.

`partition_by`

Sometimes you want the top N records per group rather than the top N records overall — for example the 3 most recent builds per repository, or the 10 most recent executions per status. Doing this with a normal listing requires one request per group, which is slow and not pageable.

The optional partition_by parameter solves this in a single query using an SQL window function (RANK() OVER (PARTITION BY <field> ORDER BY <order>)). It is an object with the following keys:

field: the field path to partition on. Supports plain column names (e.g. status) as well as paths into JSONEncodedData fields using dot notation (e.g. value.repo). JSONEncodedData is stored as TEXT containing a JSON string, not as a PostgreSQL JSON/JSONB column.
limit: the maximum number of records to return per distinct value of field.
order (optional): how to order records within each partition (string of comma-separated field-names, same syntax as order). Defaults to the top-level order.

When partition_by is set, the top-level limit/offset page over partitions (limit = maximum number of distinct field values returned) rather than over records. The response is a flat list with each partition's records grouped together; reconstruct the groups on the client side. All other parameters (filter_, order, limit, offset) keep working, and the behaviour is unchanged when partition_by is omitted.

Custom objects and value.* fields

When listing custom objects with partition_by or order on value.* paths (for example value.repo), you must include a filter that restricts the result to a single object template:

filter_={'field': 'object_template_id', 'op': 'eq', 'value': '<object-template-id>'}

Why: Custom object attributes are not stored in a shared value JSON column on the custom_object table. Each object template has its own masterdata table (masterdata.<object-template-id>) with one column per attribute. The value field you see in API responses is assembled from that table when records are read. To partition or order by value.<attribute> in SQL, the engine must join the correct masterdata table — and it can only know which table that is when the query is scoped to one object_template_id. Without that filter, the request is rejected as malformed input.

Only a single attribute name after value. is supported (for example value.repo). Nested paths such as value.nested.key are not supported for custom objects.

example

Return the 3 most recent build custom objects per repository:

import flow_api

def handler(system: flow_api.System, this: flow_api.Execution, inputs: dict):
    build_template = system.object_template('build')
    latest_builds = system.custom_objects(
        fields='id,value,created_at',
        filter_={'field': 'object_template_id', 'op': 'eq', 'value': build_template.get('id')},
        partition_by={'field': 'value.repo', 'limit': 3, 'order': '-created_at'},
    )
    this.log(latest_builds=latest_builds)

Return the 10 most recent executions per status:

    system.executions(
        fields='id,status,created_at',
        partition_by={'field': 'status', 'limit': 10, 'order': '-created_at'},
    )

Via the REST API, partition_by is a JSON-encoded query parameter. When partitioning custom objects by value.*, also pass a filter that sets object_template_id (see above):

$ curl 'https://\<your-workspace-name>.cloudomation.com/api/latest/custom_object?filter={"field":"object_template_id","op":"eq","value":"<object-template-id>"}&partition_by={"field":"value.repo","limit":3,"order":"-created_at"}'

Reading

Via the User Interface

Enter the name or parts of the name of the record in the quick search bar.
Click on the record in the quick search results.
The record is opened in the main section.

Via the REST API

Send a GET request to

https://<your-workspace-name>.cloudomation.com/api/latest/<record-type>/<record-id>

You can also access resources by name:

Send a GET request to:

https://<your-workspace-name>.cloudomation.com/api/latest/<resource-type>/<resource-name>?by=name

note

See the records reference to learn about the different resources and activities

You can specify the fields you are interested in:

https://<your-workspace-name>.cloudomation.com/api/latest/<record-type>/<record-id>?fields=id,name,description

Via the Engine Flow API

Use the following method:

import flow_api

def handler(system: flow_api.System, this: flow_api.Execution, inputs: dict):
    system.<record-type>('name of the record').get('name of field to read')
    # for example:
    description = system.flow('my-flow').get('description')
    # you can read several fields at once:
    created_at, modified_at = system.flow('my-flow').get('created_at', 'modified_at')

important

Referencing a record does not validate its existence. Even if a record with the provided name or ID does not exist, the Flow API will not throw an error.

For example:

this.log(system.flow('some-name')) 

This will log <flow some-name>, even if the flow 'some-name' does not exist or if 'some-name' does not refer to an actual flow.

Therefore, it is necessary to explicitly validate the existence of a record before performing updates or modifications through the Flow API.

example

How to access and validate if the record actually exists

import flow_api

def handler(system: flow_api.System, this: flow_api.Execution, inputs: dict):
  flow = system.flow('example_flow_name') 
  if flow.exists():
      this.log("flow exists!")
      return this.success('all done')
  else:
      return this.error("flow does not exist!")

Updating

Via the User Interface

Open the record you want to modify.
Change any field of the record.
Press the "save" button.

Via the REST API

Send a PATCH request to:

https://<your-workspace-name>.cloudomation.com/api/latest/<record-type>/<record-id>

The JSON payload should contain all fields you wish to update.

Via the Engine Flow API

Use the following method:

import flow_api

def handler(system: flow_api.System, this: flow_api.Execution, inputs: dict):
    system.user('<my-user-name>').save(description='This is me')

Atomic partial updates (concurrency-safe)

A plain read-modify-write — read a value, change part of it, and save() it back — races when two executions do it to the same record at the same time: both read the old value, and the second save() overwrites the first one's change (a lost update). For the two most common partial-update patterns, the Flow API offers dedicated methods that perform the whole read-modify-write in a single engine command under a row lock, so concurrent callers never lose each other's changes:

this.set_output(key, value, **kwargs) merges one or more keys into an execution's output_value without overwriting keys set by earlier calls. Prefer it over this.save(output_value={...}) when you only want to set part of the output.
```
this.set_output('result', 42)          # positional single key
this.set_output(status='ok', count=3)  # multiple keys, keyword form
```
warning
Do not pass a dict as the first positional argument (this.set_output({...})) — it is taken as the key and raises TypeError: unhashable type: 'dict'. To replace the whole output at once use this.save(output_value={...}).
system.setting('name').list_append(*values) appends elements to a list stored in a setting without first reading it. Concurrent appends from different executions are all preserved. Raises InvalidInputError if the current value is not a list.
```
system.setting('processed-ids').list_append('id-1', 'id-2')
```

Performance

set_output re-serialises the entire output_value on every call. If you are setting many keys or large content, build the dict in your script and write it once with this.save(output_value=my_dict) instead of many set_output calls.

Deleting

Refer to Trash (deleted records) for information on how to move to trash, permanently delete and restore records.

Object Templates and Custom Objects

Refer to the dedicated flow_api methods for information on how to access and manipulate object templates and custom objects with the flow_api.

Record Metadata

You can associate any data, that can be stored in a valid json format, to any Cloudomation Engine record.

example

The following example shows how to create, modify, access, and delete metadata on a record. In this example the metadata will belong to the execution created by this script, but you can assign metadata to any record you like.

import flow_api

def handler(system: flow_api.System, this: flow_api.Execution, inputs: dict):
    # add metadata with key 'environment' and data='test' to this execution
    # get metadata of this this execution and write it into output value
    this.add_record_metadata(key='environment', data='test')
    output_value1 = [metadata.get_dict('key', 'data') for metadata in this.record_metadata_list()]
    this.set_output(output_value1=output_value1)
    
    # change metadata with key 'environment' and set data to 'prod'
    # add metadata with key 'about' and a json object as data to this execution
    # get metadata of this this execution and write it into output value
    this.metadata('environment').save(data='prod', record_id=this.get('id'))
    this.add_record_metadata(key='about', data={'author': 'my-user', 'initial_created_at': '2000-01-01T12:00:00'})
    output_value2 = [metadata.get_dict('key', 'data') for metadata in this.record_metadata_list()]
    this.set_output(output_value2=output_value2)
    
    # delete metadata with key 'environment'
    # get metadata of this this execution and write it into output value
    this.metadata('environment').delete(permanently=True)
    output_value3 = [metadata.get_dict('key', 'data') for metadata in this.record_metadata_list()]
    this.set_output(output_value3=output_value3)

    return this.success('all done')

The script logs the metadata in the output value of the execution, first after adding a metadata, second after editing the metadata and adding a new metadata, and finally, after deleting one of the metadata:

output_value1:
  - key: environment
    data: test
output_value2:
  - key: about
    data:
      author: my-user
      initial_created_at: '2000-01-01T12:00:00'
  - key: environment
    data: prod
output_value3:
  - key: about
    data:
      author: my-user
      initial_created_at: '2000-01-01T12:00:00'

Attaching metadata at creation (atomic)

The example above creates the record first and adds its metadata in a second call. That leaves a short window in which the record exists without its metadata — and for a git-tracked record it may even be committed to git before the marker lands. To avoid this create-then-add race, pass the metadata inline as the record_metadata field on the same save() (create or update). The record and its metadata are then written in a single transaction:

import flow_api

def handler(system: flow_api.System, this: flow_api.Execution, inputs: dict):
    # the setting and its 'origin'/'batch' metadata are created atomically
    system.setting('imported-value').save(
        value=42,
        record_metadata={
            'origin': 'nightly-import',
            'batch': {'id': 17, 'source': 'crm'},
        },
    )
    return this.success('all done')

record_metadata is a mapping of {key: data} — each key is a string (up to 128 characters) and each value is arbitrary JSON. Entries are upserted by (record_id, key), so passing the same key again updates that metadata entry. The field is accepted on both create and update, and is available on every record type as well as on the REST API, GraphQL, and the MCP create_record / update_record tools. On read, the metadata is exposed through the record_metadata subrecord list (shown above), not through this field.

Creating a record with its subrecords (`children`)

The record_metadata field above is a convenience for one particular subrecord type. To create a record together with any of its subrecords in a single atomic call, use the generic children field. It is schema-derived, so it works for every subrecord type without per-type support — for example an object_template with its attributes, a role with its permissions, or a flow with its wrappers.

children is a mapping of {subrecord_type: [ {column: value, ...}, ... ]}: each key names a subrecord type of the record being created, and its value is the list of child rows to create. The parent foreign key is wired for you — you only supply each child's own columns.

import flow_api

def handler(system: flow_api.System, this: flow_api.Execution, inputs: dict):
    # an object_template and its two attributes, created together
    system.object_template('my-template').save(
        children={
            'object_template_attribute': [
                {'name': 'title', 'datatype': 'STRING'},
                {'name': 'status', 'datatype': 'TEXT'},
            ],
        },
    )
    # a role and its permissions, created together
    system.role('my-role').save(
        children={
            'role_permission': [
                {'table_type': 'FLOW', 'operation': 'READ'},
                {'table_type': 'FLOW', 'operation': 'UPDATE'},
            ],
        },
    )
    return this.success('all done')

The whole operation is atomic: the parent and all of its children are written in one transaction. Requests are validated up front, so an invalid child — a bad column value, or a children key that is not a subrecord type of the parent — is rejected as a clean 400 Bad Request before the parent is written, and nothing is created (the parent is rolled back too).

Read the created subrecords back through the parent's subrecord-list accessors — for example role_permission_list(), or, when a subrecord type has more than one foreign key to the parent, the column-disambiguated variant such as object_template_attribute_list_object_template_id().

Like record_metadata, the children field is accepted on both create and update, on every record type, and across the REST API, GraphQL, the Flow API save(), and the MCP create_record / update_record tools.

Nesting

A child entry may itself carry a children key, and it is validated recursively to any depth. Today the built-in subrecord types that can be created through the API (attributes, permissions, wrappers, metadata, …) do not themselves own further createable subrecords, so a single level of children is what you use in practice; deeper nesting is already supported by the engine for future subrecord types.

Record Comments

Comments attach threaded, human-readable notes to any Cloudomation Engine record. Where record metadata stores structured data on a record, comments store a discussion trail: Markdown notes that people — and automations — leave on a record without editing its body. A comment can be assigned to an identity, which turns it into a lightweight question directed at that person, and each comment tracks per identity whether it has been read.

Via the User Interface

Open a record and select the Comments tab. The composer supports Markdown; press Comment to post. Each comment shows its author and the time it was written, renders its Markdown body, and can be deleted by its author.

Via the REST API and Flow-API

A comment is a record_comment subrecord of the record it belongs to. You create, list, and delete comments with the same generic subrecord tools used for every other subrecord type (see Record Metadata and the children field above).

A record_comment carries these fields:

Field	Description
`record_id`	The record the comment is attached to (required).
`body`	The comment text, in Markdown (required).
`assignee_identity_id`	Optional identity the comment is assigned to, which turns it into a lightweight question.
`parent_comment_id`	Optional parent comment, for reply threading.
`author_identity_id`	The identity that wrote the comment. It is set to the calling identity — a comment cannot be posted as another identity.
`resolved_at` / `resolved_by`	When and by whom the comment was resolved. Setting `resolved_at` stamps `resolved_by`.

From a flow, add a comment to any record and read its comments back through the generated subrecord accessors:

import flow_api

def handler(system: flow_api.System, this: flow_api.Execution, inputs: dict):
    # add a comment to this execution (the author is the calling identity)
    this.add_record_comment(body='Deployment **succeeded** — see the linked run.')

    # add a reply to the first comment, assigned to a user for their attention
    first = this.record_comment_list(order='created_at')[0]
    this.add_record_comment(
        body='Please review when you have a moment.',
        parent_comment_id=first.get('id'),
        assignee_identity_id=inputs['reviewer_identity_id'],
    )

    # list the comments on this execution
    comments = [c.get_dict('body', 'author_identity_id') for c in this.record_comment_list()]
    this.set_output(comments=comments)
    return this.success('all done')

Read tracking. Each time an identity reads a comment, a record_comment_read subrecord records that identity and the time. A comment with no record_comment_read entry for an identity is unread by that identity; there is at most one read record per comment-and-identity pair.

Permissions. Reading and creating comments on a record requires READ permission on that record. A comment can be edited or deleted by its author, or by an identity that holds UPDATE permission on the parent record.

Learn More

Records reference

Authentication

Git Integration

Creating​

Via the User Interface​

Via the REST API​

Via the Engine Flow-API​

Scoped default field values with default_fields​

Listing​

Via the User Interface​

Via the REST API​

Via the Engine Flow-API​

filter_​

Filtering by a related subrecord (exists)​

partition_by​

Reading​

Via the User Interface​

Via the REST API​

Via the Engine Flow API​

Updating​

Via the User Interface​

Via the REST API​

Via the Engine Flow API​

Atomic partial updates (concurrency-safe)​

Deleting​

Object Templates and Custom Objects​

Record Metadata​

Attaching metadata at creation (atomic)​

Creating a record with its subrecords (children)​

Record Comments​

Via the User Interface​

Via the REST API and Flow-API​

Learn More​

Creating

Via the User Interface

Via the REST API

Via the Engine Flow-API

Scoped default field values with `default_fields`

Listing

Via the User Interface

Via the REST API

Via the Engine Flow-API

`filter_`

Filtering by a related subrecord (`exists`)

`partition_by`

Reading

Via the User Interface

Via the REST API

Via the Engine Flow API

Updating

Via the User Interface

Via the REST API

Via the Engine Flow API

Atomic partial updates (concurrency-safe)

Deleting

Object Templates and Custom Objects

Record Metadata

Attaching metadata at creation (atomic)

Creating a record with its subrecords (`children`)

Record Comments

Via the User Interface

Via the REST API and Flow-API

Learn More