set up ci to update offliner definition

[elfkuzco]

Changes

• set up CI publish offliner definitions changes to https://api.farm.openzim.org and https://api.farm.zimit.kiwix.org

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 92.28%. Comparing base (49f1155) to head (5857667).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main      #46   +/-   ##
=======================================
  Coverage   92.28%   92.28%           
=======================================
  Files           4        4           
  Lines         376      376           
  Branches       37       37           
=======================================
  Hits          347      347           
  Misses         25       25           
  Partials        4        4           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[benoit74]

I feel like we should:

• replace "farm" with "zimfarm" for clarity (in job, filename, variables, comments, ...)
• rename file from devdocs.json to offliner-definition.json (so that we have same name in all scrapers, and it is way easier for scraper maintainers to understand what this file is about)

Sorry for these nitty-gritty details, but it is worth to polish this first PR before spreading this over all scrapers ^^

Thank you overall, once both details above are fixed we should merge and try to make it run on main, we might realize there is something still needing polishing, but we shouldn't care.

[elfkuzco]

Okay. I was thinking if there's a way we could reuse the workflow given all the scrapers are all on the same org repo.
Also, I realize I need to create an endpoint for PATCH too

[benoit74]

Okay. I was thinking if there's a way we could reuse the workflow given all the scrapers are all on the same org repo.

Is there a way in Github to run "org" CIs ?

Also, I realize I need to create an endpoint for PATCH too

I didn't dare to mention it because I was afraid I just missed it during my review 🤣 Or maybe we could say that POST allows to both create or update definition (a bit more risky, but not that a big deal)

[elfkuzco]

Is there a way in Github to run "org" CIs ?

I am currently looking at https://docs.github.com/en/actions/how-tos/reuse-automations/reuse-workflows#creating-a-reusable-workflow but it doesn't seem to cover what I need. Not sure if it respects the on because the job shouldn't run everytime. Otherwise, it serves. It would have to live in one repo. Not sure if there's a way to setup org level CI without a repo

[elfkuzco]

# .github/workflows/reusable-offliner.yml
name: Offliner Reusable Workflow

on:
  workflow_call:
    inputs:
      version:
        required: string
        type: string
      offliner:
       required: true
       type: string

jobs:
  offliner:
    runs-on: ubuntu-latest
    steps:
      - run: echo "Running offliner with version=${{ inputs.version || 'auto' }}"

and then in devdocs

# .github/workflows/offliner-triggers.yml
name: Offliner Triggers

on:
  push:
    branches: [main]
    paths:
      - "offliner-definition.json"
  release:
    types: [published]

jobs:
  call-reusable:
    uses: ./.github/workflows/reusable-offliner.yml
    with:
      version: ${{ github.event.release.tag_name || 'dev' }}
      offliner: devdocs

[elfkuzco]

I reckon it could take more arguments

[elfkuzco]

I think openzim/overview would be a good place to place the reusable workflow. What do you think?

[benoit74]

I think openzim/overview would be a good place to place the reusable workflow. What do you think?

Yes, we should give it a try

[elfkuzco]

set up openzim/overview#64 for the reusable workflow

[benoit74]

LGTM, I will try to run this once you've merged