Third-Party Python Modules

Engine flows are written in Python 3.8. In addition to all standard Python modules several third-party modules are available.

Use Cases

The third-party Python modules serve different purposes. Please refer to their individual documentation pages which are linked below.

Available Third-Party Python Modules

The following third-party Python modules are available to be used in your flow scripts:

Module name	Description	Links
arrow	Better dates & times for Python	https://arrow.readthedocs.io/en/latest/
chevron	A Python implementation of mustache	https://github.com/noahmorrison/chevron
genson	GenSON is a powerful, user-friendly JSON Schema generator built in Python	https://github.com/wolverdude/GenSON
holidays	Generate and work with holidays in Python	https://github.com/dr-prodigy/python-holidays
html5lib	Standards-compliant library for parsing and serializing HTML documents and fragments in Python	https://github.com/html5lib/html5lib-python
iso3166	Standalone ISO 3166-1 country definitions	https://github.com/deactivated/python-iso3166
jsonschema	An implementation of the JSON Schema specification for Python	https://github.com/python-jsonschema/jsonschema
lxml	The lxml XML toolkit for Python	https://github.com/lxml/lxml
markdown	A Python implementation of John Gruber’s Markdown with Extension support.	https://github.com/Python-Markdown/markdown
mathplotlib	plotting with Python	https://github.com/matplotlib/matplotlib
numpy	The fundamental package for scientific computing with Python	https://github.com/numpy/numpy
pandas	Flexible and powerful data analysis / manipulation library for Python	https://github.com/pandas-dev/pandas
passlib	The Passlib Python Library	https://passlib.readthedocs.io/en/stable/
pdf2image	A python module that wraps the pdftoppm utility to convert PDF to PIL Image object	https://github.com/Belval/pdf2image
pdftotext	Simple PDF text extraction	https://github.com/jalan/pdftotext
phonenumbers	Python port of Google's libphonenumber	https://github.com/daviddrysdale/python-phonenumbers
Pillow	Python Imaging Library	https://github.com/python-pillow/Pillow
pycryptodomex	A self-contained cryptographic library for Python	https://github.com/Legrandin/pycryptodome
pyjwt	JSON Web Token implementation in Python	https://github.com/jpadilla/pyjwt
pyotp	Python One-Time Password Library	https://github.com/pyauth/pyotp
pyquery	A jquery-like library for python	https://github.com/gawel/pyquery
python-dateutil	Useful extensions to the standard Python datetime features	https://github.com/dateutil/dateutil
pytz	World Timezone Definitions for Python	https://pythonhosted.org/pytz/
pyyaml	PyYAML-based module to produce pretty and readable YAML-serialized data	https://pyyaml.org/
tiktoken	tiktoken is a fast BPE tokeniser for use with OpenAI's models	https://github.com/openai/tiktoken
ujson	Ultra fast JSON decoder and encoder written in C with Python bindings	https://github.com/ultrajson/ultrajson
yarl	Yet another URL library	https://github.com/aio-libs/yarl

note

Contact us at info@cloudomation.com to request additional Python modules.

Examples

Parse a PDF File

You can access the content of PDF files directly in your flow script.

example

Access a date written in a PDF file

import re
import base64
import io
import pdftotext
import flow_api

def handler(system: flow_api.System, this: flow_api.Execution, inputs: dict):
    pdf_name = 'example.pdf'
    # Open the Engine file object
    pdf_file = system.file(pdf_name)
    # Files contain binary content which is returned base64 encoded
    file_base64 = pdf_file.get('content')
    file_bytes = base64.b64decode(file_base64)
    # Construct a `PDF` object from the bytes
    pdf = pdftotext.PDF(io.BytesIO(file_bytes))
    # Access the text of the first page of the PDF file
    pdf_text = pdf[0]
    # Clear the PDF object, so that the file object is closed
    pdf = None
    # Use a Regex to find the date
    the_date = re.search(r'Date:\s*(\d{4}-[01]\d-[0-3]\d)', pdf_text).group(1)
    this.log(the_date)
    return this.success('all done')

Work with CSV Files

You can use the csv module to read CSV files into Python objects or write Python objects into CSV files.

example

Create a CSV file and send it by email.

import base64
import csv
import io
import flow_api

def handler(system: flow_api.System, this: flow_api.Execution, inputs: dict):
    fake_data_list = [
        {'name': 'spam', 'number': 1},
        {'name': 'eggs', 'number': 2},
        {'name': 'foo', 'number': 3},
        {'name': 'bar', 'number': 4},
    ]
    result_file = io.StringIO()
    csv_writer = csv.DictWriter(
        result_file,
        fieldnames=['name', 'number'],
    )
    csv_writer.writeheader()
    for data in fake_data_list:
        csv_writer.writerow(data)
    csv_content = result_file.getvalue()
    result_file.close()
    result_file = None
    csv_writer = None

    system.file('result.csv').save_text_content(csv_content)

    this.connect(
        connector_type='SMTP',
        from_='me@example.com',
        to='you@example.com',
        subject= 'Result',
        text='See the attached file',
        login='me@example.com',
        password= '****',
        smtp_host= 'SMTP.example.com',
        smtp_port= 587,
        use_tls= True,
        attachments= [
                'cloudomation:result.csv',
            ],
    )

    return this.success('all done')