Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/_build.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright (c) 2022 - 2023, Oracle and/or its affiliates. All rights reserved.
# Copyright (c) 2022 - 2025, Oracle and/or its affiliates. All rights reserved.
# Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl/.

# This is a trusted builder implemented as a reusable workflow that can be called by other
Expand Down Expand Up @@ -142,7 +142,7 @@ jobs:
needs: [build]
permissions:
contents: read
packages: read
packages: write
uses: ./.github/workflows/_build_docker.yaml
with:
artifact-sha256: ${{ needs.build.outputs.artifacts-sha256 }}
Expand Down
82 changes: 75 additions & 7 deletions .github/workflows/_build_docker.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ jobs:
build-docker:
runs-on: ubuntu-latest
permissions:
packages: read
packages: write # to push the test docker image
steps:

- name: Check out repository
Expand Down Expand Up @@ -67,10 +67,78 @@ jobs:
run: make setup-integration-test-utility-for-docker

# Run the integration tests against the built Docker image.
- name: Test the Docker image
# - name: Test the Docker image
# env:
# # This environment variable will be picked up by run_macaron.sh.
# MACARON_IMAGE_TAG: test
# DOCKER_PULL: never
# GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# run: make integration-test-docker

# Push the test Docker image.
- name: Push the Docker image
id: push-docker
env:
IMAGE_NAME: ghcr.io/oracle/macaron
RELEASE_TAG: test
run: |
make push-docker-test
IMAGE_AND_DIGEST=$(docker inspect --format='{{index .RepoDigests 0}}' "$IMAGE_NAME")
NAME=$(echo "$IMAGE_AND_DIGEST" | cut -d'@' -f1 | cut -d':' -f1)
DIGEST=$(echo "$IMAGE_AND_DIGEST" | cut -d'@' -f2)
{
echo "image-name=${NAME}"
echo "image-digest=${DIGEST}"
} >> "$GITHUB_OUTPUT"

# Generate the Docker image SBOM under the dist/ directory which will be published as part of the release assets.
- name: Generate Docker sbom
env:
# This environment variable will be picked up by run_macaron.sh.
MACARON_IMAGE_TAG: test
DOCKER_PULL: never
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: make integration-test-docker
SYFT_BIN: ${{ github.workspace }}/bin
RELEASE_TAG: test
RELEASE_VERSION: test
SYFT_VERSION: 1.29.0
# We install Syft, which is an SBOM generator tool for Docker images, using the instructions from:
# https://github.com/anchore/syft#installation
# We only generate SBOM in CycloneDX format.
run: |
mkdir -p "$SYFT_BIN"
ASSET_NAME="syft_${SYFT_VERSION}_linux_amd64.tar.gz"
CHECKSUMS="syft_${SYFT_VERSION}_checksums.txt"

# Download artifacts.
echo "Downloading $ASSET_NAME"
curl --output "$ASSET_NAME" --progress-bar --location \
"https://github.com/anchore/syft/releases/download/v${SYFT_VERSION}/${ASSET_NAME}"
test -s "$ASSET_NAME" || (echo "Unable to download $ASSET_NAME" && exit 0)
echo "Downloading $CHECKSUMS"
curl --output "$CHECKSUMS" --progress-bar --location \
"https://github.com/anchore/syft/releases/download/v${SYFT_VERSION}/${CHECKSUMS}"
test -s "$CHECKSUMS" || (echo "Unable to download $CHECKSUMS" && exit 0)

EXPECTED=$(grep "${ASSET_NAME}" "${CHECKSUMS}" 2>/dev/null | tr '\t' ' ' | cut -d ' ' -f 1)
SYFT_DIGEST=$(sha256sum "$ASSET_NAME" | cut -d ' ' -f 1)

# Check if artifact is valid.
if [ "$EXPECTED" == "$SYFT_DIGEST" ]; then
tar -zxvf "$ASSET_NAME" -C "$SYFT_BIN" syft
"$SYFT_BIN"/syft --version
"$SYFT_BIN"/syft \
ghcr.io/oracle/macaron:"$RELEASE_TAG" \
-o cyclonedx-json=dist/macaron-"$RELEASE_VERSION"-sbom-docker.json
else
echo "Checksum for '$ASSET_NAME' did not verify: expected $EXPECTED but got $SYFT_DIGEST"
fi

# Remove the downloaded artifacts.
rm -f "$ASSET_NAME"
rm -f "$CHECKSUMS"

# Upload the SBOM.
- name: Upload the SBOM
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: sbom-macaron-test
path: dist/macaron-test-sbom-docker.json
if-no-files-found: error
retention-days: 7
4 changes: 2 additions & 2 deletions .github/workflows/pr-change-set.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright (c) 2022 - 2023, Oracle and/or its affiliates. All rights reserved.
# Copyright (c) 2022 - 2025, Oracle and/or its affiliates. All rights reserved.
# Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl/.

# This workflow checks and tests the package code, builds all package
Expand All @@ -22,4 +22,4 @@ jobs:
uses: ./.github/workflows/_build.yaml
permissions:
contents: read
packages: read
packages: write
2 changes: 1 addition & 1 deletion .github/workflows/test_macaron_action.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ jobs:
- name: Setup Python for analyzed venv
uses: actions/setup-python@42375524e23c412d93fb67b49958b491fce71c38 # v5.4.0
with:
python-version: 3.11.13
python-version: 3.11.14

- name: Create and populate analyzed venv
run: |
Expand Down
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ Please see the [README for the malware analyzer](./src/macaron/malware_analyzer/

### Prerequisites

- Python 3.11.13
- Python 3.11.14
- Go 1.23
- JDK 17

Expand Down
41 changes: 32 additions & 9 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -94,10 +94,27 @@ setup: force-upgrade setup-go setup-binaries setup-schemastore
setup-go:
go build -o $(PACKAGE_PATH)/bin/ $(REPO_PATH)/golang/cmd/...
setup-binaries: $(PACKAGE_PATH)/bin/slsa-verifier souffle gnu-sed

# Install SLSA Verifier.
SLSA_VERIFIER_TAG := v2.7.1
SLSA_VERIFIER_BIN := slsa-verifier-linux-amd64
SLSA_VERIFIER_BIN_PATH := $(PACKAGE_PATH)/bin/$(SLSA_VERIFIER_BIN)
SLSA_VERIFIER_PROVENANCE := $(SLSA_VERIFIER_BIN).intoto.jsonl
SLSA_VERIFIER_PROVENANCE_PATH := $(PACKAGE_PATH)/bin/$(SLSA_VERIFIER_PROVENANCE)

$(PACKAGE_PATH)/bin/slsa-verifier:
git clone --depth 1 https://github.com/slsa-framework/slsa-verifier.git -b v2.7.1
cd slsa-verifier/cli/slsa-verifier && go build -o $(PACKAGE_PATH)/bin/
cd $(REPO_PATH) && rm -rf slsa-verifier
mkdir -p $(PACKAGE_PATH)/bin \
&& wget -O $(PACKAGE_PATH)/bin/slsa-verifier https://github.com/slsa-framework/slsa-verifier/releases/download/$(SLSA_VERIFIER_TAG)/$(SLSA_VERIFIER_BIN) \
&& wget -O $(SLSA_VERIFIER_PROVENANCE_PATH) https://github.com/slsa-framework/slsa-verifier/releases/download/$(SLSA_VERIFIER_TAG)/$(SLSA_VERIFIER_PROVENANCE) \
&& chmod +x $(PACKAGE_PATH)/bin/slsa-verifier \
&& EXPECTED_HASH=$$(jq -r '.payload' $(SLSA_VERIFIER_PROVENANCE_PATH) | base64 -d | jq -r '.subject[] | select(.name == "$(SLSA_VERIFIER_BIN)") | .digest.sha256') \
&& ACTUAL_HASH=$$(sha256sum $(PACKAGE_PATH)/bin/slsa-verifier | awk '{print $$1}'); \
if [ "$$EXPECTED_HASH" != "$$ACTUAL_HASH" ]; then \
echo "Hash mismatch: expected $$EXPECTED_HASH, got $$ACTUAL_HASH"; \
exit 1; \
fi

# Set up schemastore for GitHub Actions specs.
setup-schemastore: $(PACKAGE_PATH)/resources/schemastore/github-workflow.json $(PACKAGE_PATH)/resources/schemastore/LICENSE $(PACKAGE_PATH)/resources/schemastore/NOTICE
$(PACKAGE_PATH)/resources/schemastore/github-workflow.json:
cd $(PACKAGE_PATH)/resources \
Expand Down Expand Up @@ -257,15 +274,12 @@ requirements.txt: pyproject.toml
# editable mode (like the one in development here) because they may not have
# a PyPI entry; also print out CVE description and potential fixes if audit
# found an issue.
# Ignore GHSA-4xh5-x5gv-qwph since we are using Python >=3.11.13, which is not vulnerable to this
# CVE. Remove this once a new version of pip that fixes the CVE is released.
# See https://github.com/pypa/pip/issues/13607
.PHONY: audit
audit:
if ! $$(python -c "import pip_audit" &> /dev/null); then \
echo "No package pip_audit installed, upgrade your environment!" && exit 1; \
fi;
python -m pip_audit --skip-editable --desc on --fix --dry-run --ignore-vuln GHSA-4xh5-x5gv-qwph
python -m pip_audit --skip-editable --desc on --fix --dry-run

# Run some or all checks over the package code base.
.PHONY: check check-code check-bandit check-flake8 check-lint check-mypy check-go check-actionlint
Expand Down Expand Up @@ -346,9 +360,9 @@ integration-test-update:
# set to the build date/epoch. For more details, see: https://flit.pypa.io/en/latest/reproducible.html
.PHONY: dist
dist: dist/$(PACKAGE_NAME)-$(PACKAGE_VERSION)-py3-none-any.whl dist/$(PACKAGE_NAME)-$(PACKAGE_VERSION).tar.gz dist/$(PACKAGE_NAME)-$(PACKAGE_VERSION)-docs-html.zip dist/$(PACKAGE_NAME)-$(PACKAGE_VERSION)-build-epoch.txt
dist/$(PACKAGE_NAME)-$(PACKAGE_VERSION)-py3-none-any.whl: check test integration-test
dist/$(PACKAGE_NAME)-$(PACKAGE_VERSION)-py3-none-any.whl:
flit build --setup-py --format wheel
dist/$(PACKAGE_NAME)-$(PACKAGE_VERSION).tar.gz: check test integration-test
dist/$(PACKAGE_NAME)-$(PACKAGE_VERSION).tar.gz:
flit build --setup-py --format sdist
dist/$(PACKAGE_NAME)-$(PACKAGE_VERSION)-docs-html.zip: docs
python -m zipfile -c dist/$(PACKAGE_NAME)-$(PACKAGE_VERSION)-docs-html.zip docs/_build/html
Expand Down Expand Up @@ -389,6 +403,15 @@ push-docker:
docker push "${IMAGE_NAME}":latest
docker push "${IMAGE_NAME}":"${RELEASE_TAG}"

# Push the test Docker image. The image name and tag are read from IMAGE_NAME and RELEASE_TAG
# environment variables, respectively.
.PHONY: push-docker-test
push-docker-test:
if [ -z "${IMAGE_NAME}" ] || [ -z "${RELEASE_TAG}" ]; then \
echo "Please set IMAGE_NAME and RELEASE_TAG environment variables!" && exit 1; \
fi
docker push "${IMAGE_NAME}":"${RELEASE_TAG}"

# Prune the packages currently installed in the virtual environment down to the required
# packages only. Pruning works in a roundabout way, where we first generate the wheels for
# all installed packages into the build/wheelhouse/ folder. Next we wipe all packages and
Expand Down
2 changes: 1 addition & 1 deletion action.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ runs:
- name: Setup Python
uses: actions/setup-python@42375524e23c412d93fb67b49958b491fce71c38 # v5.4.0
with:
python-version: 3.11.13
python-version: 3.11.14

- name: Setup Go
uses: actions/setup-go@4dc6199c7b1a012772edbd06daecab0f50c9053c # v6.1.0
Expand Down
8 changes: 4 additions & 4 deletions docker/Dockerfile.base
Original file line number Diff line number Diff line change
Expand Up @@ -5,15 +5,15 @@
# to build and run the Docker image.
# This image is based on the container-registry.oracle.com/os/oraclelinux:9-slim image and contains the following
# components:
# Python3.11.13 compiled and installed from source.
# Python3.11.14 compiled and installed from source.
# Souffle 2.5 compiled and installed from source.
# Other runtime libraries (e.g sqlite-devel) which are installed from dnf.

FROM container-registry.oracle.com/os/oraclelinux:9-slim@sha256:92deb326256d4d3053d210397b00dce9a423789d1c555adb7a3b7a1f0747ea2f
FROM container-registry.oracle.com/os/oraclelinux:9-slim@sha256:41a867b7f24306cf38c01ba578598164397bd07aa26dbdc9a985bedd9177e82e

ENV HOME="/home/macaron" \
# Setting Python related environment variables.
PYTHON3_VERSION=3.11.13 \
PYTHON3_VERSION=3.11.14 \
PYTHONUNBUFFERED=1 \
PYTHONDONTWRITEBYTECODE=1 \
# https://github.com/docker-library/python/blob/f568f56f28fab0fe87b34db777e2c2861cef002b/3.11/slim-buster/Dockerfile#L12
Expand Down Expand Up @@ -62,7 +62,7 @@ enabled=1\
# Exceptions (not installed):
# mcpp - The package mcpp is not available in Oracle Linux 9. However, we don't use Souffle's feature
# that needs it.
# python3 - We use the installed Python3.11.13 for this.
# python3 - We use the installed Python3.11.14 for this.
gcc-c++ \
libffi \
libffi-devel \
Expand Down
2 changes: 1 addition & 1 deletion docker/Dockerfile.final
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
# Note that the local machine must login to ghcr.io so that Docker could pull the ghcr.io/oracle/macaron-base
# image for this build.

FROM ghcr.io/oracle/macaron-base:latest@sha256:e7cb431d2a870999b70a9a282a84e7b278f7a9ea91e60ba2a8efdab35b4b7e71
FROM ghcr.io/oracle/macaron-base:latest@sha256:6d1d300d32060a75deffd2e6fce00e9f6d646df233f8df4deee2baf2982cf022

ENV HOME="/home/macaron"

Expand Down
2 changes: 1 addition & 1 deletion docs/source/pages/supported_technologies/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,7 @@ Automatic dependency resolution

Currently, we support the following type of project for automatic dependency resolution.

* Python (with a Python virtual environment created and packages installed using Python3.11.13, see :ref:`providing Python virtual environment <python-venv-deps>`.)
* Python (with a Python virtual environment created and packages installed using Python3.11.14, see :ref:`providing Python virtual environment <python-venv-deps>`.)

--------
See also
Expand Down
2 changes: 1 addition & 1 deletion docs/source/pages/tutorials/detect_malicious_package.rst
Original file line number Diff line number Diff line change
Expand Up @@ -190,7 +190,7 @@ Macaron supports analyzing a package's dependencies and performs the same set of

Let's assume ``/tmp/.django_venv`` is the virtual environment where ``[email protected]`` is installed.

.. note:: If you want Macaron to analyze the virtual environment directly to identify the dependencies, we require Python 3.11.13 to be used to install the package. Alternatively, you can generate the SBOM as instructed :ref:`here <python-sbom>` and pass it to Macaron as input.
.. note:: If you want Macaron to analyze the virtual environment directly to identify the dependencies, we require Python 3.11.14 to be used to install the package. Alternatively, you can generate the SBOM as instructed :ref:`here <python-sbom>` and pass it to Macaron as input.

Run Macaron as follows to analyze ``django`` and its direct dependencies.

Expand Down
2 changes: 1 addition & 1 deletion docs/source/pages/using.rst
Original file line number Diff line number Diff line change
Expand Up @@ -378,7 +378,7 @@ Where ``--python-venv`` is the path to virtual environment.

Alternatively, you can create an SBOM for the python package and provide it to Macaron as input as explained :ref:`here <with-sbom>`.

.. note:: We only support Python 3.11.13 for this feature of Macaron. Please make sure to install the package using this version of Python.
.. note:: We only support Python 3.11.14 for this feature of Macaron. Please make sure to install the package using this version of Python.


-----------------------------------------------
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ build-backend = "flit_core.buildapi"

[project]
name = "macaron"
requires-python = ">=3.11.13"
requires-python = ">=3.11.14"
authors = [
{"name" = "Trong Nhan Mai", "email" = "[email protected]"},
{"name" = "Behnaz Hassanshahi", "email" = "[email protected]"},
Expand Down
Loading