Skip to content

[SP-2874] feat: add licenses sub-command, add support for ingesting CDX, add CDX input validation #131

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 18 commits into
base: main
Choose a base branch
from
Open

[SP-2874] feat: add licenses sub-command, add support for ingesting CDX, add CDX input validation #131

Show file tree
Hide file tree
Changes from 16 commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
592e714
[SP-2874] feat: add licenses sub-command, add support for ingesting C…
matiasdaloia Jul 10, 2025
72b0652
[SP-2874] feat: support CDX input file in crypto decoration commands
matiasdaloia Jul 10, 2025
0b60f5d
[SP-2874] chore: update dependency versions, refactor to reduce metho…
matiasdaloia Jul 10, 2025
df62002
[SP-2874] chore: use python 3.9 in all workflows
matiasdaloia Jul 10, 2025
9bdc2af
[SP-2874] chore: update protobuf
matiasdaloia Jul 11, 2025
43e68bb
[SP-2874] chore: update all remaining protobufs
matiasdaloia Jul 11, 2025
bffcc06
[SP-2874] chore: bump version
matiasdaloia Jul 11, 2025
cb6476d
[SP-2874] chore: update changelog, documentation and dockerfile
matiasdaloia Aug 1, 2025
2e28522
[SP-2874] chore: update scanoss.json
matiasdaloia Aug 1, 2025
0a3ebe3
[SP-2874] chore: update scanoss.json
matiasdaloia Aug 1, 2025
b84b036
[SP-2874] chore: update changelog and version
matiasdaloia Aug 19, 2025
5f920dc
[SP-2991] fix: update to papi latest definitions
matiasdaloia Aug 21, 2025
96f839e
[SP-2874] chore: update version and changelog
matiasdaloia Sep 19, 2025
70f73c6
[SP-2874] fix: adapt for new components request
matiasdaloia Sep 19, 2025
b2c6409
[SP-2874] feat: add REST support for licenses endpoint
matiasdaloia Sep 22, 2025
5777a49
[SP-2874] chore: update workflow python version
matiasdaloia Sep 22, 2025
a490fc3
[SP-2874] fix: scancode dockerfile execution
matiasdaloia Sep 22, 2025
62f6076
[SP-2874] chore: update pkg requirements
matiasdaloia Sep 22, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions .github/workflows/container-local-test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,10 @@ on:
workflow_dispatch:
push:
branches:
- 'main'
- "main"
pull_request:
branches:
- 'main'
- "main"

env:
IMAGE_BASE: scanoss/scanoss-py-base
Expand All @@ -27,7 +27,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.9.x'
python-version: "3.10.x"

- name: Install Dependencies
run: |
Expand Down Expand Up @@ -98,3 +98,4 @@ jobs:
echo "Error: Scan test did not produce any results. Failing"
exit 1
fi

2 changes: 1 addition & 1 deletion .github/workflows/container-publish-ghcr.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.9.x"
python-version: '3.9.x'

- name: Install Dependencies
run: |
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/python-publish-pypi.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.9.x"
python-version: '3.9.x'

- name: Install dependencies
run: |
Expand Down Expand Up @@ -70,7 +70,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.9.x"
python-version: '3.9.x'

- name: Install Remote Package
uses: nick-fields/retry@v3
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/python-publish-testpypi.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.9.x"
python-version: '3.9.x'

- name: Install Dependencies
run: |
Expand Down Expand Up @@ -65,7 +65,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.9.x"
python-version: '3.9.x'

- name: Install Remote Package
run: |
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/version-tag.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.9.x"
python-version: '3.9.x'
- name: Determine Tag
id: taggerVersion
run: |
Expand Down
21 changes: 14 additions & 7 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Added
- Upcoming changes...

## [1.33.0] - 2025-09-19
### Added
- Add `licenses` sub-command to `component` command
- Add support for ingesting CDX to all decoration commands
- Add CDX input validation

## [1.32.0] - 2025-09-01
### Added
- Switched vulnerability and dependency APIs to use REST by default
Expand Down Expand Up @@ -176,7 +182,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [1.20.2] - 2025-02-26
### Fixed
- Fixed provenance command
- Fixed provenance command

## [1.20.1] - 2025-02-18
### Added
Expand Down Expand Up @@ -238,7 +244,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [1.17.5] - 2024-11-12
### Fixed
- Fix dependencies scan result structure

## [1.17.4] - 2024-11-08
### Fixed
- Fix backslashes in file paths on Windows
Expand All @@ -255,7 +261,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Added
- Added supplier to SPDX packages
### Changed
- Changed undeclared summary output
- Changed undeclared summary output

## [1.17.1] - 2024-10-24
### Fixed
Expand Down Expand Up @@ -288,7 +294,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Added
- Added support for Python3.12
- Module `pkg_resources` has been replaced with `importlib_resources`
- Added support for UTF-16 filenames
- Added support for UTF-16 filenames

## [1.13.0] - 2024-06-05
### Added
Expand Down Expand Up @@ -367,11 +373,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [1.6.0] - 2023-06-16
### Added
- Added support for High Precision Snippet Matching (`--hpsm` or `-H`) while scanning
- `scanoss-py scan --hpsm ...`
- `scanoss-py scan --hpsm ...`

## [1.5.2] - 2023-06-13
### Added
- Added retry limit option (`--retry`) while scanning
- Added retry limit option (`--retry`) while scanning
- `--retry 0` will fail immediately

## [1.5.1] - 2023-04-21
Expand Down Expand Up @@ -660,4 +666,5 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
[1.31.3]: https://github.com/scanoss/scanoss.py/compare/v1.31.2...v1.31.3
[1.31.4]: https://github.com/scanoss/scanoss.py/compare/v1.31.3...v1.31.4
[1.31.5]: https://github.com/scanoss/scanoss.py/compare/v1.31.4...v1.31.5
[1.31.5]: https://github.com/scanoss/scanoss.py/compare/v1.31.5...v1.32.0
[1.32.0]: https://github.com/scanoss/scanoss.py/compare/v1.31.5...v1.32.0
[1.33.0]: https://github.com/scanoss/scanoss.py/compare/v1.32.0...v1.33.0
38 changes: 38 additions & 0 deletions CLIENT_HELP.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -337,6 +337,44 @@ The following command provides the capability to search the SCANOSS KB for compo
scanoss-py comp prov -p "pkg:github/unoconv/unoconv" --origin
```

#### Component Licenses
The following command provides the capability to search the SCANOSS KB for licenses for Open Source components:
```bash
scanoss-py comp licenses -p "pkg:github/jquery/jquery" -p "pkg:npm/express"
```
It is possible to supply multiple PURLs by repeating the `-p pkg` option, or providing a purl input file `-i purl-input.json` ([for example](tests/data/purl-input.json)):
```bash
scanoss-py comp licenses -i purl-input.json -o component-licenses.json
```

The licenses command also supports CycloneDX (CDX) input files. You can provide a CycloneDX SBOM file and retrieve license information for all components:
```bash
scanoss-py comp licenses -i cyclonedx-sbom.json -o component-licenses.json
```

### CDX Input Support for Component Commands
Several component commands now support CycloneDX (CDX) input files. This allows you to analyze components from existing SBOM files:

**Supported commands with CDX input:**
- `comp vulns` - Analyze vulnerabilities from CDX file
- `comp licenses` - Retrieve licenses from CDX file
- `comp crypto` - Detect cryptographic algorithms from CDX file
- `comp semgrep` - Find semgrep issues from CDX file

**Example using CDX input:**
```bash
# Analyze vulnerabilities from a CycloneDX SBOM
scanoss-py comp vulns -i sbom.cdx.json -o vulnerabilities.json

# Get licenses for all components in a CycloneDX SBOM
scanoss-py comp licenses -i sbom.cdx.json -o licenses.json

# Detect cryptographic usage from CDX
scanoss-py comp crypto -i sbom.cdx.json -o crypto-findings.json
```

The CDX input file is automatically validated to ensure it's a valid CycloneDX format before processing.


### Results Commands
The `results` command provides the capability to operate on scan results. For example:
Expand Down
3 changes: 2 additions & 1 deletion requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,9 @@ requests
crc32c>=2.2
binaryornot
progress
grpcio>1.42.0
grpcio>=1.70.0
protobuf>3.19.1
protoc-gen-openapiv2
pypac
urllib3
pyOpenSSL
Expand Down
12 changes: 7 additions & 5 deletions scanoss.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,9 @@
"skip": {
"patterns": {
"scanning": [
"src/protoc_gen_swagger/",
"src/scanoss/api/",
"docs/make.bat"
"src/protoc_gen_swagger",
"docs",
"scanoss_common_pb2_grpc.py"
]
},
"sizes": {}
Expand All @@ -16,6 +16,8 @@
{
"purl": "pkg:github/scanoss/scanoss.py"
}
]
],
"remove": []
}
}
}

3 changes: 2 additions & 1 deletion setup.cfg
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,9 @@ install_requires =
crc32c>=2.2
binaryornot
progress
grpcio>1.42.0
grpcio>=1.70.0
protobuf>3.19.1
protoc-gen-openapiv2
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

🧩 Analysis chain

Duplicate dependency and likely wrong package variant for OpenAPI v2 protos.

  • protoc-gen-openapiv2 is listed twice (Lines 34 and 43).
  • The protoc-gen-openapiv2 package on PyPI appears unmaintained (last release 0.0.1 on Dec 2, 2022). A more up-to-date alternative is grpc-gateway-protoc-openapiv2 (e.g., 2.25.1.0 released Jan 11, 2025). Please align the dependency with whatever import path your generated code uses and deduplicate. (pypi.org)

Suggested fix (choose the package that matches your generated imports; if you import grpc_gateway_protoc_openapiv2.*, use the first option below):

-    protoc-gen-openapiv2
+    grpc-gateway-protoc-openapiv2
...
-    protoc-gen-openapiv2

If your generated imports are protoc_gen_openapiv2.*, keep that variant but remove the duplicate and consider switching to a maintained fork when feasible. The upstream gRPC‑Gateway docs also indicate the plugin is a Go tool (protoc-gen-openapiv2), hence the need for Python-packaged stubs. (github.com)

Run to confirm which package your code imports:

Also applies to: 43-43

🏁 Script executed:

#!/bin/bash
# Detect which OpenAPI v2 stub package the code expects
rg -nP -C2 'from\s+(grpc_gateway_protoc_openapiv2|protoc_gen_openapiv2)\b|import\s+(grpc_gateway_protoc_openapiv2|protoc_gen_openapiv2)\b' -g 'src/**'

Length of output: 5380

Remove duplicate dependency; keep protoc-gen-openapiv2 to match generated imports

Generated protos import protoc_gen_openapiv2.options — remove the duplicate protoc-gen-openapiv2 entry from setup.cfg.

🤖 Prompt for AI Agents 
In setup.cfg around line 34 the dependency protoc-gen-openapiv2 is duplicated;
remove the duplicate entry so only a single protoc-gen-openapiv2 remains to
match generated protos that import protoc_gen_openapiv2.options. Edit setup.cfg
to delete the redundant line and ensure the remaining entry uses the correct
package name and version (if applicable) to keep imports consistent with
generated code.

pypac
pyOpenSSL
google-api-core
Expand Down
30 changes: 18 additions & 12 deletions src/protoc_gen_swagger/options/annotations_pb2.py
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

48 changes: 48 additions & 0 deletions src/protoc_gen_swagger/options/annotations_pb2.pyi
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
"""
@generated by mypy-protobuf. Do not edit manually!
isort:skip_file
"""

import builtins
import google.protobuf.descriptor
import google.protobuf.descriptor_pb2
import google.protobuf.internal.extension_dict
import protoc_gen_swagger.options.openapiv2_pb2

DESCRIPTOR: google.protobuf.descriptor.FileDescriptor

OPENAPIV2_SWAGGER_FIELD_NUMBER: builtins.int
OPENAPIV2_OPERATION_FIELD_NUMBER: builtins.int
OPENAPIV2_SCHEMA_FIELD_NUMBER: builtins.int
OPENAPIV2_TAG_FIELD_NUMBER: builtins.int
OPENAPIV2_FIELD_FIELD_NUMBER: builtins.int
openapiv2_swagger: google.protobuf.internal.extension_dict._ExtensionFieldDescriptor[google.protobuf.descriptor_pb2.FileOptions, protoc_gen_swagger.options.openapiv2_pb2.Swagger]
"""ID assigned by [email protected] for grpc-gateway project.

All IDs are the same, as assigned. It is okay that they are the same, as they extend
different descriptor messages.
"""
openapiv2_operation: google.protobuf.internal.extension_dict._ExtensionFieldDescriptor[google.protobuf.descriptor_pb2.MethodOptions, protoc_gen_swagger.options.openapiv2_pb2.Operation]
"""ID assigned by [email protected] for grpc-gateway project.

All IDs are the same, as assigned. It is okay that they are the same, as they extend
different descriptor messages.
"""
openapiv2_schema: google.protobuf.internal.extension_dict._ExtensionFieldDescriptor[google.protobuf.descriptor_pb2.MessageOptions, protoc_gen_swagger.options.openapiv2_pb2.Schema]
"""ID assigned by [email protected] for grpc-gateway project.

All IDs are the same, as assigned. It is okay that they are the same, as they extend
different descriptor messages.
"""
openapiv2_tag: google.protobuf.internal.extension_dict._ExtensionFieldDescriptor[google.protobuf.descriptor_pb2.ServiceOptions, protoc_gen_swagger.options.openapiv2_pb2.Tag]
"""ID assigned by [email protected] for grpc-gateway project.

All IDs are the same, as assigned. It is okay that they are the same, as they extend
different descriptor messages.
"""
openapiv2_field: google.protobuf.internal.extension_dict._ExtensionFieldDescriptor[google.protobuf.descriptor_pb2.FieldOptions, protoc_gen_swagger.options.openapiv2_pb2.JSONSchema]
"""ID assigned by [email protected] for grpc-gateway project.

All IDs are the same, as assigned. It is okay that they are the same, as they extend
different descriptor messages.
"""
20 changes: 20 additions & 0 deletions src/protoc_gen_swagger/options/annotations_pb2_grpc.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,24 @@
# Generated by the gRPC Python protocol compiler plugin. DO NOT EDIT!
"""Client and server classes corresponding to protobuf-defined services."""
import grpc
import warnings


GRPC_GENERATED_VERSION = '1.73.1'
GRPC_VERSION = grpc.__version__
_version_not_supported = False

try:
from grpc._utilities import first_version_is_lower
_version_not_supported = first_version_is_lower(GRPC_VERSION, GRPC_GENERATED_VERSION)
except ImportError:
_version_not_supported = True

if _version_not_supported:
raise RuntimeError(
f'The grpc package installed is at version {GRPC_VERSION},'
+ f' but the generated code in protoc_gen_swagger/options/annotations_pb2_grpc.py depends on'
+ f' grpcio>={GRPC_GENERATED_VERSION}.'
+ f' Please upgrade your grpc module to grpcio>={GRPC_GENERATED_VERSION}'
+ f' or downgrade your generated code using grpcio-tools<={GRPC_VERSION}.'
)
Loading
Loading