Public sync via Copybara PoC #183
Open
+1,291
−88
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![internal-python-client-app[bot]](https://avatars.githubusercontent.com/u/50711777?s=60&v=4){kind=small}
internal-python-client-app
bot
force-pushed
the
eda-sync-
branch
2 times, most recently
from
51e78b1 to
cb002bb
Compare
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
internal-python-client-app
bot
force-pushed
the
eda-sync-
branch
from
cb002bb to
fec2f73
Compare
Comment on lines
+13
to
+38
| runs-on: ${{ github.repository_visibility == 'private' && 'ed-runner-set' || 'ubuntu-latest' }} | ||
| strategy: | ||
| matrix: | ||
| python-version: ["3.10", "3.11", "3.12", "3.13"] | ||
| container: | ||
| image: python:${{ matrix.python-version }} | ||
|
|
||
| steps: | ||
| - uses: actions/checkout@v4 | ||
|
|
||
| - name: Install system dependencies | ||
| run: | | ||
| apt-get update | ||
| apt-get install -y gdal-bin libgdal-dev | ||
|
|
||
| - name: Install Poetry | ||
| run: pip install poetry | ||
|
|
||
| - name: Install test dependencies | ||
| run: poetry install --only tests | ||
|
|
||
| - name: Run Linting | ||
| run: poetry run tox -e lint | ||
|
|
||
| - name: Run Formatting Check | ||
| run: poetry run tox -e format |
Check warning
Code scanning / CodeQL
Workflow does not contain permissions Medium
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 4 days ago
To fix the issue, we will add a permissions block at the root level of the workflow file. Since this workflow is for linting, formatting, and type checking, it only needs read access to the repository contents. We will set contents: read as the minimal required permission. This change ensures that the workflow does not inadvertently gain unnecessary write permissions.
Suggested changeset
1
.github/workflows/lint.yml
| @@ -10,2 +10,5 @@ | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| jobs: |
Copilot is powered by AI and may make mistakes. Always verify output.
internal-python-client-app
bot
force-pushed
the
eda-sync-
branch
from
fec2f73 to
a4604c0
Compare
Comment on lines
+13
to
+33
| name: TruffleHog Secret Scan | ||
| runs-on: ${{ github.repository_visibility == 'private' && 'ed-runner-set' || 'ubuntu-latest' }} | ||
| container: | ||
| image: alpine:latest | ||
| steps: | ||
| - name: Checkout code | ||
| uses: actions/checkout@v4 | ||
| - name: Install Git and dependencies | ||
| run: | | ||
| apk add --no-cache git curl jq | ||
|
|
||
| - name: Clone the repository manually | ||
| env: | ||
| GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
| run: | | ||
| git clone https://x-access-token:${GH_TOKEN}@github.com/${GITHUB_REPOSITORY}.git repo | ||
| cd repo | ||
|
|
||
| # install TruffleHog | ||
| curl -sSfL https://raw.githubusercontent.com/trufflesecurity/trufflehog/main/scripts/install.sh | sh -s -- -b /usr/local/bin | ||
|
|
||
| - name: Run TruffleHog OSS | ||
| uses: trufflesecurity/trufflehog@v3.88.17 | ||
| # run TruffleHog scan | ||
| trufflehog filesystem "." --results=verified,unknown --fail --json | jq |
Check warning
Code scanning / CodeQL
Workflow does not contain permissions Medium
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 4 days ago
To fix the issue, we will add a permissions block at the workflow level to explicitly define the minimal permissions required. Since the workflow only needs to read the repository contents to perform the secret scan, we will set contents: read. This ensures that the GITHUB_TOKEN has the least privileges necessary to complete the task.
Suggested changeset
1
.github/workflows/secret-scanning.yml
| @@ -10,2 +10,5 @@ | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| jobs: |
Copilot is powered by AI and may make mistakes. Always verify output.
Comment on lines
+13
to
+45
| name: Unit Tests (Python ${{ matrix.python-version }}) | ||
| runs-on: ${{ github.repository_visibility == 'private' && 'ed-runner-set' || 'ubuntu-latest' }} | ||
| strategy: | ||
| matrix: | ||
| python-version: ["3.10", "3.11", "3.12", "3.13"] | ||
| include: | ||
| - python-version: "3.10" | ||
| tox_env: py310 | ||
| - python-version: "3.11" | ||
| tox_env: py311 | ||
| - python-version: "3.12" | ||
| tox_env: py312 | ||
| - python-version: "3.13" | ||
| tox_env: py313 | ||
| container: | ||
| image: python:${{ matrix.python-version }} | ||
|
|
||
| steps: | ||
| - uses: actions/checkout@v4 | ||
|
|
||
| - name: Install GDAL and dependencies | ||
| run: | | ||
| apt-get update | ||
| apt-get install -y gdal-bin libgdal-dev | ||
|
|
||
| - name: Install Poetry | ||
| run: pip install poetry | ||
|
|
||
| - name: Install test dependencies | ||
| run: poetry install --only tests | ||
|
|
||
| - name: Run tests with tox | ||
| run: poetry run tox -e ${{ matrix.tox_env }} |
Check warning
Code scanning / CodeQL
Workflow does not contain permissions Medium test
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 4 days ago
To fix the issue, we will add a permissions block at the workflow level (root) to restrict the GITHUB_TOKEN permissions to contents: read. This is sufficient for the current workflow, as it only needs to read repository contents to run tests. This change will ensure that the workflow adheres to the principle of least privilege.
Suggested changeset
1
.github/workflows/test.yaml
| @@ -1,2 +1,4 @@ | ||
| name: earthdaily-python-client CI | ||
| permissions: | ||
| contents: read | ||
|
|
Copilot is powered by AI and may make mistakes. Always verify output.
internal-python-client-app
bot
force-pushed
the
eda-sync-
branch
5 times, most recently
from
06adf64 to
e6356d2
Compare
internal-python-client-app
bot
force-pushed
the
eda-sync-
branch
4 times, most recently
from
922eb67 to
a778819
Compare
internal-python-client-app
bot
force-pushed
the
eda-sync-
branch
3 times, most recently
from
d433509 to
d1c95db
Compare
…a Fix CodeArtifact login domain and domain owner in workflow by Iman Shafiei <[email protected]> - 63da05a7c83b9e1e4acfd24bdfe88f59bd7c9e0c Refactor CodeArtifact workflow to install twine and publi... by Iman Shafiei <[email protected]> - 835cf35efbee67948948287552851d90985f45b5 Fix CodeArtifact repository configuration by swapping tok... by Iman Shafiei <[email protected]> - b910eddd8f9279339ba94aac29b2f3341c90325e Add logging for CodeArtifact URL and token length in work... by Iman Shafiei <[email protected]> - 59fd410dd5cb319a1d759ecce42331d21a12081e Add retrieval of authorization token for CodeArtifact in ... by Iman Shafiei <[email protected]> - 79245720d5edae81b0f7f174a017526521fd1276 Add step to configure Poetry repository with CodeArtifact... by Iman Shafiei <[email protected]> - bb0d752a2c9997827b2cd0674f413022bb4f6e45 Refactor CodeArtifact workflow to use Python slim image a... by Iman Shafiei <[email protected]> - 138066b9170b8a5c26cb75f1f639cf635836143e Add bulk insert and bulk search services to the Earth Dai... by Iman Shafiei <[email protected]> - b922f64bc514c34a0cde19981c54a2981ded6d1c More debugging by Iman Shafiei <[email protected]> - 63d60e2dd8db8ed4033bc21dce1fa0ced37af67d Update container image for CodeArtifact upload job to use... by Iman Shafiei <[email protected]> - c9cf80878ef51c245e02dd55517d7a1941dca989 Add debug step for AWS environment in CodeArtifact workflow by Iman Shafiei <[email protected]> - 275a4ec9d6f6dc2ede041fa6f0afa6591426f3b6 Fix formatting of docstring in generate_token.py and upda... by Iman Shafiei <[email protected]> - 083edc6eba06d8c5c9bbae5e720b6dfb1c2269d3 Update permissions and clean up workflow steps for CodeAr... by Iman Shafiei <[email protected]> - ad1ce501dadf4aa0bfb48c14c06aaea9f10a07fe Add GitHub Actions workflow for publishing to AWS CodeArt... by Iman Shafiei <[email protected]> - 22f380c1b7cd594a93d0a355a39d9c92d80ffa27 Refactor internal service initialization and add error ha... by Iman Shafiei <[email protected]> - 6d6a64deb92793a5ecc52a7bb6bc305cc47d80c3 Refactor job runner selection for private and public repo... by Iman Shafiei <[email protected]> - 082fa1a4f29cb050ee4ea53c88c06075cad3f26a Fix runner selection logic for lint job based on reposito... by Iman Shafiei <[email protected]> - 5be4aaa4d335f5892ec6a0e8af923f5b4e875590 Add pull_request trigger to workflows for better CI integ... by Iman Shafiei <[email protected]> - 6522b429410df79b2c6acf02fc842dbc4f8f2427 Fix virtualenv activation command in Copybara sync workflow by Iman Shafiei <[email protected]> - 1822e48ca9b79a2f57a73d16e4554e5c87f99d89 Refactor secret scanning workflow to clone repository man... by Iman Shafiei <[email protected]> - 73a6795b16bcf2f74c0269a1577cdce695e00747 Refactor secret scanning workflow to use Alpine image and... by Iman Shafiei <[email protected]> - 3b0e46719f0810c291cb8e336314d52bd2847879 Add step to install jq in secret scanning workflow by Iman Shafiei <[email protected]> - bd142eae532db672b6e061ed3d97a55f3c886129 Add configuration for Git safe directory in secret scanni... by Iman Shafiei <[email protected]> - ab8515a7030c0fb9aea7eb0b34ef4a168ddfe9d9 Add configuration for Git safe directory in secret scanni... by Iman Shafiei <[email protected]> - 4cc3959122cc10b7d01529813e654e1a9fa1a575 Refactor secret scanning workflow to use TruffleHog Docke... by Iman Shafiei <[email protected]> - feaa54f1ce823560a2f8ce7c611dc99b107b35fb Update secret scanning workflow to install additional dep... by Iman Shafiei <[email protected]> - d770a696c4a6c94f9935ab947bc35d73caa19a3d Update secret scanning workflow to use Docker stable imag... by Iman Shafiei <[email protected]> - 30c5bdc77bba8a0395941f20275ad5abeb71293d Update secret scanning workflow to install additional dep... by Iman Shafiei <[email protected]> - e38fab2d223599461f7deb7a247d6b4ae590d228 Add configuration for Git safe directory in secret scanni... by Iman Shafiei <[email protected]> - c1f13aaeb118143d22d54268c9a363941323a4a9 Add git installation step to secret scanning workflow by Iman Shafiei <[email protected]> - c20fcc1b8adea1cc67ff7bc6691ad7a80c864250 Add container configuration with Ubuntu image to CI workf... by Iman Shafiei <[email protected]> - 2c0b4062c69e6bd71beca6632670b0f77d05611e format with ruff 0.11.0 by Iman Shafiei <[email protected]> - b5efc402a65bdac855632248c4fddf23ca16d4ca Refactor CI test jobs to use a matrix strategy for Python... by Iman Shafiei <[email protected]> - 3485721d1cb2b8d3eb57e9b962125e5cc1abef1a Refactor linting workflow to consolidate Python version h... by Iman Shafiei <[email protected]> - 88fa81508552f3658534fb96850a80b6ded31b92 Add CI workflows for linting and testing with Python vers... by Iman Shafiei <[email protected]> - ba318c0b5667f9585ec50045f3ea5f9254f8f0a9 Update CI workflows to use ed-runner-set for consistency ... by Iman Shafiei <[email protected]> - 5effcd5b81839d7f68783d2a1ce65144cfa8e5b8 Update CI workflows and BulkInsert/BulkSearch services fo... by Iman Shafiei <[email protected]> - ea78f2c91cb69f3b55d8125a7b17b5f157b4bfa7 Add InternalService and bulk operations support to EDSCli... by Iman Shafiei <[email protected]> - bfe7972f2b1df36143c609736e5ff952ff149403 Comment out CodeQL workflow configuration in codeql.yml by Iman Shafiei <[email protected]> - 0e46cd33ede3f3f1a8b8ad059a64069cf1ab6aa4 Beta/add pyjwt dependency and implement GitHub token gene... by Iman Shafiei <[email protected]> - a9174ac2cf3c1455019f98efba2dfb355980ede8 Merge remote-tracking branch 'public-repo/beta/1.x' by Iman Shafiei <[email protected]> - dbdaa9c820ec7245f5964ead6ce985803157e92a Initial commit by edawader <[email protected]> GitOrigin-RevId: 521521fcef242de9a2579e2871023ce1fcd9469a
internal-python-client-app
bot
force-pushed
the
eda-sync-
branch
from
d1c95db to
0c41a4e
Compare
Comment on lines
+10
to
+52
| name: Upload to AWS CodeArtifact | ||
| runs-on: [ed-runner-set] | ||
| container: | ||
| image: python:3.13-slim | ||
|
|
||
| steps: | ||
| - name: Install system dependencies | ||
| run: | | ||
| apt-get update | ||
| apt-get install -y curl unzip gnupg awscli build-essential | ||
|
|
||
| - name: Checkout repo | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Configure AWS credentials | ||
| uses: aws-actions/configure-aws-credentials@v4 | ||
| with: | ||
| aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} | ||
| aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} | ||
| aws-region: us-east-1 | ||
|
|
||
| - name: Install Poetry | ||
| run: pip install poetry | ||
|
|
||
| - name: Clean old dist | ||
| run: rm -rf dist/ | ||
|
|
||
| - name: Build package with Poetry | ||
| run: poetry build | ||
|
|
||
| - name: Login to CodeArtifact | ||
| run: | | ||
| aws codeartifact login \ | ||
| --tool twine \ | ||
| --repository pypi \ | ||
| --domain urthecast \ | ||
| --domain-owner ${{ secrets.FGPROD_ACCOUNT_ID }} \ | ||
| --region us-east-1 | ||
|
|
||
| - name: Install twine and publish to CodeArtifact | ||
| run: | | ||
| pip install twine | ||
| twine upload --repository codeartifact dist/* |
Check warning
Code scanning / CodeQL
Workflow does not contain permissions Medium
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI about 2 hours ago
To fix the issue, we need to add a permissions block to the workflow. This block should specify the minimal permissions required for the workflow to function correctly. Based on the workflow's operations, it primarily interacts with external services and uses actions/checkout. Therefore, the contents: read permission is sufficient.
The permissions block should be added at the root level of the workflow to apply to all jobs, as no job-specific permissions are required.
Suggested changeset
1
.github/workflows/publish_to_codeartifact.yaml
| @@ -7,2 +7,5 @@ | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| jobs: |
Copilot is powered by AI and may make mistakes. Always verify output.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR contains a filtered export excluding internal, secrets, tests, and the Copybara config itself.