C3 - the CLAIMED Component Compiler

TL;DR

• takes arbitrary assets (Jupyter notebooks, python scripts, R scripts) as input
• automatically creates container images and pushes to container registries
• automatically installs all required dependencies into the container image
• creates KubeFlow Pipeline components (target workflow execution engines are pluggable)
• creates Kubernetes job configs for execution on Kubernetes/Openshift clusters
• can be triggered from CICD pipelines

To learn more on how this library works in practice, please have a look at the following video

Related work

Ploomber

Orchest

Getting started

Install

pip install claimed

Usage

Just run the following command with your python script or notebook:

c3_create_operator "<your-operator-script>.py" --repository "<registry>/<namespace>"

Your code needs to follow certain requirements which are explained in Getting Started.

Getting Help

c3_create_operator --help

We welcome your questions, ideas, and feedback. Please create an issue or a discussion thread. Please see VULNERABILITIES.md for reporting vulnerabilities.

Contributing to CLAIMED

Interested in helping make CLAIMED better? We encourage you to take a look at our Contributing page.

Credits

CLAIMED is supported by the EU’s Horizon Europe program under Grant Agreement number 101131841 and also received funding from the Swiss State Secretariat for Education, Research and Innovation (SERI) and the UK Research and Innovation (UKRI).

License

This software is released under Apache License v2.0.