Publish a list of malicious packages that have been taken down

[di]

What's the problem this feature will solve?
Users who may have possibly installed malicious packages don't have insight into what packages have been taken down by PyPI administrators.

Describe the solution you'd like
PyPI should publish both a human-readable and machine-readable (API) list of malicious packages that have been taken down. Ideally the human-readable list would be sortable by package name, or by the date it was created/taken down.

Additional context
Feature request to automatically uninstall packages via this API in pip: pypa/pip#5777

[waseem18]

@di How do I find packages that have been taken down - from the database point of view? Is there any flag (is_removed or is_malicious) in the table?

[di]

@waseem18 Nope, there isn't, so we would have to add that flag and manually infer it from the comments of previously removed packages.

[waseem18]

Okay, So we add the flag to the respective table and we set it to true for any projects we want to mark as malicious.

If I understand you correctly, as the data of already removed packages doesn't exist on our database, we would need to infer it from the Warehouse GH issues.

So after we add the flag, the API call would return any packages that are flagged as malicious + the list we have of already removed packages.

Please do correct me if I'm wrong.

[di]

There is a comment field on the BlacklistedProject model: https://github.com/pypa/warehouse/blob/97f28dfa5a4017dd1e1a7630f772ce01ec1af749/warehouse/packaging/models.py#L568

What I meant was that once we add the ability to mark a BlacklistedProject as malicious, there should be some way the administrators can go back and manually set this marker based on the comments we left. There's only about 200 projects right now, so this wouldn't be a terrible burden.

[waseem18]

Gotcha!

So we can add the is_malicious flag to the BlacklistedProject model with default value as false.

And the API end point would query for the entities of BlacklistedProject table with the flag set to true.

[pfalcon]

Please be sure to provide the reason for each takedown case - e.g. DMCA request, government/security services involvement, somebody's whim, etc.

[di]

This issue is only about malicious packages, which are taken down by the PyPI admistrators at their discretion.

[pradyunsg]

Is anyone working on this? I would like to work toward this during the Bloomberg Sprint.

If nothing else, figuring out how this works/is exposed from Warehouse's side should be a good start.

[waseem18]

I'm not working on this @pradyunsg . Feel free to pick it up.

[oliviersm199]

Hey I will pick it up at the Bloomberg NYC Sprint!

[oliviersm199]

#4962 relates to this issue