Skip to content

WIP: feature(pu): add sampled_unizero multitask pipeline #311

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 142 commits into
base: main
Choose a base branch
from
Open

WIP: feature(pu): add sampled_unizero multitask pipeline #311

wants to merge 142 commits into from

Conversation

puyuan1996
Copy link
Collaborator

@puyuan1996 puyuan1996 commented Dec 24, 2024 •
edited
Loading

add sampled_unizero multitask pipeline in continuous action sapce

puyuan1996 and others added 30 commits July 5, 2024 16:32
@puyuan1996
feature(pu): add UniZero multitask related pipeline
dd2c95c
@dyyoungg
polish(pu): polish unizero_multitask config
8769a5c
@dyyoungg
fix(pu): fix empty_keys_values in init_infer
c342ce1
@dyyoungg
feature(pu): add softmoe head option in unizero_multitask
6eb772a
@dyyoungg
fix(pu): fix unizero reset in muzero_collector
71f55b4
@dyyoungg
polish(pu): polish unizero-multitask config
445fd70
@dyyoungg
fix(pu): fix replay ratio
4954581
@dyyoungg
feature(pu): add moe option of feedforward in transformer backbone
44304bf
@dyyoungg
feature(pu): add value_priority in unizero_multitask
d6be21a
@dyyoungg
polish(pu): polish value_priority in unizero_multitask
fde51cc
@dyyoungg
sync code
b460d2f
@dyyoungg
fix(pu): fix moe in feedforward layer of transformer and polish configs
5117459
@dyyoungg
feature(pu): add mistralai moe in transformer feedforward and head of…
2495d60 
… unizero
@PaParaZz1
polish(pu): polish quantize_state_hash and deepcopy
95886bd
@PaParaZz1
fix(pu): fix np.array dtype bug in buffer
0e49a30
@PaParaZz1
polish(pu): use 0 deepcopy in kv_cache operation in collect/eval phas…
00147f4 
…e of unizero
@PaParaZz1
polish(pu): use custom deepcopy for kv_cache
b40c71b
@PaParaZz1
polish(pu): use value_array rather than value_list in compute_target_…
2cc81be 
…value
@PaParaZz1
polish(pu): optimize compute_target_policy_non_re
bc5332f
@PaParaZz1
polish(pu): optimize kv_caching update()
a6c6a8e
@PaParaZz1
polish(pu): kv_cache_dict no to_cpu
b5dcdcc
@PaParaZz1
polish(pu): optimize custom kv_cache copy
5b0cbd4
@PaParaZz1
polish(pu): kv_cache_dict no to_cpu
0035829
@PaParaZz1
feature(pu): add unizero ddp config
043727b
@dyyoungg
fix(pu): fix unizero ddp
d568008
@dyyoungg
sync code
d349137
@PaParaZz1
polish(pu): use de kv_cacheepcopy only in recur_infer load
3a344aa
@dyyoungg
Merge branch 'dev-efficiency' of https://github.com/opendilab/LightZero…
40053f7 
… into dev-efficiency
@dyyoungg
sync code
61a1139
@dyyoungg
polish(pu): polish suz dmc config
0e545c7
dyyoungg and others added 19 commits November 25, 2024 17:33
@dyyoungg
sync code
13fbe4c
@puyuan1996
feature(pu): add muzero multitask (and its ddp version) pipeline
d5842f1
@puyuan1996
polish(pu): polish configs
7723f13
@dyyoungg
polish(pu): polish config
f1e8d8c
@dyyoungg
polish(pu): polish config
62c8a96
@dyyoungg
Merge branch 'dev-mz-multitask-ddp' of https://github.com/opendilab/L…
99c08e2 
…ightZero into dev-mz-multitask-ddp
@dyyoungg
fix(pu): fix embed dim in uz_multitask pipeline
1edcba3
@dyyoungg
feature(pu): add uz finetune config
4b195eb
@dyyoungg
feature(pu): add uz eval-tsne config
13183e7
@dyyoungg
fix(pu): add uz eval-tsne config
1e37cae
@dyyoungg
polish(pu): polish tsne-plot legend
29298e6
@puyuan1996
polish(pu): polish atari multitask related configs
ffdf4db
@puyuan1996
polish(pu): polish unizero/muzero multitask related entry
2b0af34
@puyuan1996
polish(pu): delete unused files
0bd688e
@puyuan1996
Merge remote-tracking branch 'origin/main' into dev-mz-multitask-ddp
69a1842
@puyuan1996
polish(pu): polish policy/model in multitask settings
d06ce61
@puyuan1996
feature(pu): add sampled_unizero multitask pipeline
3a88f46
@puyuan1996
fix(pu): fix sampled_unizero multitask ddp pipeline
563548b
@puyuan1996
fix(pu): fix sampled_unizero multitask ddp pipeline
d8705e6
@puyuan1996 puyuan1996 mentioned this pull request Dec 24, 2024
Closed
@puyuan1996 puyuan1996 changed the title TMP: feature(pu): add sampled_unizero multitask pipeline WIP: feature(pu): add sampled_unizero multitask pipeline Jan 8, 2025
@puyuan1996 puyuan1996 added enhancement New feature or request research Research work in progress labels Jan 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
enhancement New feature or request research Research work in progress
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@puyuan1996 @dyyoungg @PaParaZz1 @jiayilee65