Skip to content

docs: add module popularity #33

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
+20 −0
Open

docs: add module popularity #33

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions source/reference/dashboard.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,8 @@ Once you choose the module, you will be presented with the module's information:

.. image:: /_static/images/dashboard/module.png

.. todo: update image

To deploy click in ``Deploy > Inference API (dedicated)`` and you will be redirected to a configuration page.

.. image:: /_static/images/dashboard/configure.png
Expand Down
18 changes: 18 additions & 0 deletions source/reference/modules.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,3 +58,21 @@ A typical pipeline is divided in two parts:
* regenerate the provenance chain of that module,

.. image:: /_static/images/ai4eosc/jenkins.png


Module popularity
-----------------

In order to efficiently explore the catalog of modules, it is useful to know what are the most popular modules.

Assessing the popularity of module is a difficult task because there are many proxy metrics of popularity, each one with it's own shortcomings:

* **Github Stars**: while useful, many users do not take the time to star the modules they use.
* **DockerHub downloads**: downloads sometimes do not reflect real usage because they can be inflated by CI/CD automated builds.
* **Number of created persistent deployments**: they can be misleading sometimes, as someone can deploy a module a single time but use it a lot, while another person might download it and immediately remove it.
* **Number of created tryme deployments**: this metric is very informative but it privileges trained modules (eg. YOLO trained to detect fishes) over untrained ones (eg. YOLO with no trained weights). This is unfair because the trained module used the untrained module as a base reference, so the base module is indeed popular.
* **Number of Dashboard views**: this metric could prioritize fancy shinny modules over modules that are really used by users,

To be able to easily compare across modules and mitigate the individual shortcomings, we weight these individual metrics into an overall **aggregated popularity score**. This aggragated score allows to sort the catalog modules by popularity.

.. todo: add image