start implementing support for downloading output_data after an invocation completes #1529
Conversation
…ation completes Currently, the --download_outputs flag can be used with planemo run, but it requires the user to wait until the execution is complete. If the user prefers to use --nowait or if the connection is lost during execution, they lose the ability to retrieve the outputs. This pull request begins to introduce support for downloading outputs after the invocation has finished, regardless of when or how it completes.
|
|
||
| invocation_data = invocation_client.show_invocation(invocation) | ||
| # NEED I way of decoding workflow id and history_id from the invocation data, | ||
| # we have the information but need to decode it before requesting the files. |
There was a problem hiding this comment.
Hello, not sure what you mean by 'encoded versions', perhaps you're referring to the difference between workflow_id and stored_workflow_id? It's not well documented unfortunately and has tripped me up before, e.g. #1205. If so, you can take the workflow_id you get from the invocation and look it up with the instance param set to true: gi.make_get_request(f'{config.user_gi.url}/workflows/{workflow_id}', params={'instance': instance}).json()
As for the history ids, you should be able to use them directly
There was a problem hiding this comment.
thanks for helping me! That /workflows/{workflow_id} endpoint helped:)
|
This pull-request depends on a new release of bioblend, bioblend>1.5.0 |
Smeds
force-pushed
the
invocation-download-output
branch
from
ea4e3ef to
d1f8e5f
Compare
@Smeds Let me know if you need other PRs for bioblend merged, otherwise I can make a new release. |
@nsoranzo Thanks! I would like to be able to have bioblend #506 merged, but that on is dependent on galaxy #20359. Don't know if we should wait on that on or make a release now for bioblend. |
|
I think this is covered by #1534 and #1532 ? feel free to re-open if I missed something. Thanks for all the great work @Smeds and the help from @simonbray and @nsoranzo! |
Currently, the --download_outputs flag can be used with planemo run, but it requires the user to wait until the execution is complete. If the user prefers to use --nowait or if the connection is lost during execution, they lose the ability to retrieve the outputs. This pull request begins to introduce support for downloading outputs after the invocation has finished, regardless of when or how it completes.
The current implementation has a major limitation: the user must provide the history_id and workflow_id in addition to the invocation_id. While it’s possible to fetch invocation details using the invocation_id, the response only includes encoded versions of history_id and workflow_id, which cannot be used directly. I'm not sure how to decode these values—any guidance or pointers would be appreciated.