feat(dsv3): add dsv3 fast pretrain into paddleformers #2496

chen2016013 · 2025-08-27T16:12:07Z

Before submitting

Lint code. If there are lint issues, please format the code first.

# Install and register `pre-commit` in the project folder
pip install pre-commit && pre-commit install

# Process previous code files separately
pre-commit run --file XXXX.py

Add test cases into tests folder. If there are codecov issues, please add tests cases first.

PR types

PR changes

Description

…/PaddleFormers into test0827_merge

paddle-bot · 2025-08-27T16:12:33Z

Thanks for your contribution!

paddleformers/trainer/training_args.py

paddleformers/examples/deepseek_v3/run_pretrain.py

paddleformers/trainer/training_args.py

paddleformers/transformers/deepseek_v2/modeling_fast.py

Co-authored-by: zhangbo9674 <[email protected]>

…eFormers into test0827_merge

zhangbo9674 · 2025-08-28T09:08:51Z

paddleformers/examples/deepseek_v3/script/train_gpu.sh

+bash script/kill_process.sh
+
+# source /root/paddlejob/workspace/env_run/zhangbo/env_ds/bin/activate
+source /root/paddlejob/workspace/env_run/chenxi/chenxi_py3.10/bin/activate


hushenwei2000 and others added 16 commits August 25, 2025 14:20

feat(dsv3): trainer part migration

f789ca3

add pre_train entrance

608bde9

change deepseekv2 model

dc4f910

change deepseekv2 model 2

a040746

feat(dsv3): download support bos

d57a21b

Merge branch 'develop' into merge_dsv3

626ef5a

feat(dsv3): download support bos

11318b9

change deepseekv2 model 3

1ed58e9

Update __init__.py

92d409a

Update config.json

be2b827

Update train_gpu.sh

35b3693

feat(dsv3): fix code

6a72aed

Merge commit 'refs/pull/2486/head' of https://github.com/PaddlePaddle…

edb5cd1

…/PaddleFormers into test0827_merge

Merge commit 'refs/pull/2487/head' of https://github.com/PaddlePaddle…

bee704f

…/PaddleFormers into test0827_merge

Merge commit 'refs/pull/2483/head' of https://github.com/PaddlePaddle…

c6e19b4

…/PaddleFormers into test0827_merge

merge all changes

0459f1d

chen2016013 commented Aug 27, 2025

View reviewed changes

paddleformers/trainer/training_args.py Outdated Show resolved Hide resolved

zhangbo9674 reviewed Aug 28, 2025

View reviewed changes

paddleformers/examples/deepseek_v3/run_pretrain.py Outdated Show resolved Hide resolved

paddleformers/trainer/training_args.py Outdated Show resolved Hide resolved

paddleformers/transformers/deepseek_v2/modeling_fast.py Outdated Show resolved Hide resolved

chen2016013 and others added 5 commits August 28, 2025 16:06

fix bug

101d592

Update paddleformers/transformers/deepseek_v2/modeling_fast.py

7b65ab6

Co-authored-by: zhangbo9674 <[email protected]>

Update paddleformers/examples/deepseek_v3/run_pretrain.py

84cf440

Co-authored-by: zhangbo9674 <[email protected]>

fix bug

4f60f2d

Merge branch 'test0827_merge' of https://github.com/chen2016013/Paddl…

5e6075a

…eFormers into test0827_merge

zhangbo9674 reviewed Aug 28, 2025

View reviewed changes

Update train_gpu.sh

f87fe03

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(dsv3): add dsv3 fast pretrain into paddleformers #2496

feat(dsv3): add dsv3 fast pretrain into paddleformers #2496

Uh oh!

chen2016013 commented Aug 27, 2025

Uh oh!

paddle-bot bot commented Aug 27, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

zhangbo9674 Aug 28, 2025

Uh oh!

Uh oh!

feat(dsv3): add dsv3 fast pretrain into paddleformers #2496

Are you sure you want to change the base?

feat(dsv3): add dsv3 fast pretrain into paddleformers #2496

Uh oh!

Conversation

chen2016013 commented Aug 27, 2025

Before submitting

PR types

PR changes

Description

Uh oh!

paddle-bot bot commented Aug 27, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

zhangbo9674 Aug 28, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!