Hugging Face's logo

smartrichard
/
X-R1-lora-7500

Text Generation
Transformers
Safetensors
qwen2
Generated from Trainer
X-R1
conversational
text-generation-inference
Model card Files
xet
Community
1
smartrichard commited on
Commit
28e21c6
·
verified ·
1 Parent(s): 156b75f

Upload 65 files

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. .gitattributes +4 -0
  2. README.md +67 -0
  3. adapter_config.json +34 -0
  4. adapter_model.safetensors +3 -0
  5. added_tokens.json +24 -0
  6. all_results.json +8 -0
  7. checkpoint-1876/README.md +202 -0
  8. checkpoint-1876/adapter_config.json +34 -0
  9. checkpoint-1876/adapter_model.safetensors +3 -0
  10. checkpoint-1876/added_tokens.json +24 -0
  11. checkpoint-1876/global_step1875/bf16_zero_pp_rank_0_mp_rank_00_optim_states.pt +3 -0
  12. checkpoint-1876/global_step1875/mp_rank_00_model_states.pt +3 -0
  13. checkpoint-1876/latest +1 -0
  14. checkpoint-1876/merges.txt +0 -0
  15. checkpoint-1876/rng_state.pth +3 -0
  16. checkpoint-1876/scheduler.pt +3 -0
  17. checkpoint-1876/special_tokens_map.json +31 -0
  18. checkpoint-1876/tokenizer.json +3 -0
  19. checkpoint-1876/tokenizer_config.json +209 -0
  20. checkpoint-1876/trainer_state.json +2464 -0
  21. checkpoint-1876/training_args.bin +3 -0
  22. checkpoint-1876/vocab.json +0 -0
  23. checkpoint-1876/zero_to_fp32.py +674 -0
  24. checkpoint-2811/README.md +202 -0
  25. checkpoint-2811/adapter_config.json +34 -0
  26. checkpoint-2811/adapter_model.safetensors +3 -0
  27. checkpoint-2811/added_tokens.json +24 -0
  28. checkpoint-2811/global_step2810/bf16_zero_pp_rank_0_mp_rank_00_optim_states.pt +3 -0
  29. checkpoint-2811/global_step2810/mp_rank_00_model_states.pt +3 -0
  30. checkpoint-2811/latest +1 -0
  31. checkpoint-2811/merges.txt +0 -0
  32. checkpoint-2811/rng_state.pth +3 -0
  33. checkpoint-2811/scheduler.pt +3 -0
  34. checkpoint-2811/special_tokens_map.json +31 -0
  35. checkpoint-2811/tokenizer.json +3 -0
  36. checkpoint-2811/tokenizer_config.json +209 -0
  37. checkpoint-2811/trainer_state.json +0 -0
  38. checkpoint-2811/training_args.bin +3 -0
  39. checkpoint-2811/vocab.json +0 -0
  40. checkpoint-2811/zero_to_fp32.py +674 -0
  41. checkpoint-938/README.md +202 -0
  42. checkpoint-938/adapter_config.json +34 -0
  43. checkpoint-938/adapter_model.safetensors +3 -0
  44. checkpoint-938/added_tokens.json +24 -0
  45. checkpoint-938/global_step937/bf16_zero_pp_rank_0_mp_rank_00_optim_states.pt +3 -0
  46. checkpoint-938/global_step937/mp_rank_00_model_states.pt +3 -0
  47. checkpoint-938/latest +1 -0
  48. checkpoint-938/merges.txt +0 -0
  49. checkpoint-938/rng_state.pth +3 -0
  50. checkpoint-938/scheduler.pt +3 -0
.gitattributes CHANGED
@@ -33,3 +33,7 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ checkpoint-1876/tokenizer.json filter=lfs diff=lfs merge=lfs -text
37
+ checkpoint-2811/tokenizer.json filter=lfs diff=lfs merge=lfs -text
38
+ checkpoint-938/tokenizer.json filter=lfs diff=lfs merge=lfs -text
39
+ tokenizer.json filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,67 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: Qwen/Qwen2.5-1.5B
3
+ datasets: xiaodongguaAIGC/X-R1-7500
4
+ library_name: transformers
5
+ tags:
6
+ - generated_from_trainer
7
+ - X-R1
8
+ licence: license
9
+ ---
10
+
11
+ # Model Card for None
12
+
13
+ This model is a fine-tuned version of [Qwen/Qwen2.5-1.5B](https://huggingface.co/Qwen/Qwen2.5-1.5B) on the [xiaodongguaAIGC/X-R1-7500](https://huggingface.co/datasets/xiaodongguaAIGC/X-R1-7500) dataset.
14
+ It has been trained using [TRL](https://github.com/huggingface/trl).
15
+
16
+ ## Quick start
17
+
18
+ ```python
19
+ from transformers import pipeline
20
+
21
+ question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
22
+ generator = pipeline("text-generation", model="None", device="cuda")
23
+ output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
24
+ print(output["generated_text"])
25
+ ```
26
+
27
+ ## Training procedure
28
+
29
+ [<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="150" height="24"/>](https://wandb.ai/smartrichard_team1/huggingface/runs/rx351n7r)
30
+
31
+
32
+ This model was trained with GRPO, a method introduced in [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://huggingface.co/papers/2402.03300).
33
+
34
+ ### Framework versions
35
+
36
+ - TRL: 0.15.0
37
+ - Transformers: 4.48.2
38
+ - Pytorch: 2.5.1
39
+ - Datasets: 3.3.2
40
+ - Tokenizers: 0.21.0
41
+
42
+ ## Citations
43
+
44
+ Cite GRPO as:
45
+
46
+ ```bibtex
47
+ @article{zhihong2024deepseekmath,
48
+ title = {{DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models}},
49
+ author = {Zhihong Shao and Peiyi Wang and Qihao Zhu and Runxin Xu and Junxiao Song and Mingchuan Zhang and Y. K. Li and Y. Wu and Daya Guo},
50
+ year = 2024,
51
+ eprint = {arXiv:2402.03300},
52
+ }
53
+
54
+ ```
55
+
56
+ Cite TRL as:
57
+
58
+ ```bibtex
59
+ @misc{vonwerra2022trl,
60
+ title = {{TRL: Transformer Reinforcement Learning}},
61
+ author = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallouédec},
62
+ year = 2020,
63
+ journal = {GitHub repository},
64
+ publisher = {GitHub},
65
+ howpublished = {\url{https://github.com/huggingface/trl}}
66
+ }
67
+ ```
adapter_config.json ADDED
@@ -0,0 +1,34 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alpha_pattern": {},
3
+ "auto_mapping": null,
4
+ "base_model_name_or_path": "Qwen/Qwen2.5-1.5B",
5
+ "bias": "none",
6
+ "eva_config": null,
7
+ "exclude_modules": null,
8
+ "fan_in_fan_out": false,
9
+ "inference_mode": true,
10
+ "init_lora_weights": true,
11
+ "layer_replication": null,
12
+ "layers_pattern": null,
13
+ "layers_to_transform": null,
14
+ "loftq_config": {},
15
+ "lora_alpha": 8,
16
+ "lora_bias": false,
17
+ "lora_dropout": 0.0,
18
+ "megatron_config": null,
19
+ "megatron_core": "megatron.core",
20
+ "modules_to_save": null,
21
+ "peft_type": "LORA",
22
+ "r": 32,
23
+ "rank_pattern": {},
24
+ "revision": null,
25
+ "target_modules": [
26
+ "q_proj",
27
+ "v_proj",
28
+ "embed_tokens",
29
+ "k_proj"
30
+ ],
31
+ "task_type": "CAUSAL_LM",
32
+ "use_dora": false,
33
+ "use_rslora": false
34
+ }
adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d0e2d12773cde23612f66f956756bfff79b5088a590085701d068e152e8b9f0d
3
+ size 488520640
added_tokens.json ADDED
@@ -0,0 +1,24 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "</tool_call>": 151658,
3
+ "<tool_call>": 151657,
4
+ "<|box_end|>": 151649,
5
+ "<|box_start|>": 151648,
6
+ "<|endoftext|>": 151643,
7
+ "<|file_sep|>": 151664,
8
+ "<|fim_middle|>": 151660,
9
+ "<|fim_pad|>": 151662,
10
+ "<|fim_prefix|>": 151659,
11
+ "<|fim_suffix|>": 151661,
12
+ "<|im_end|>": 151645,
13
+ "<|im_start|>": 151644,
14
+ "<|image_pad|>": 151655,
15
+ "<|object_ref_end|>": 151647,
16
+ "<|object_ref_start|>": 151646,
17
+ "<|quad_end|>": 151651,
18
+ "<|quad_start|>": 151650,
19
+ "<|repo_name|>": 151663,
20
+ "<|video_pad|>": 151656,
21
+ "<|vision_end|>": 151653,
22
+ "<|vision_pad|>": 151654,
23
+ "<|vision_start|>": 151652
24
+ }
all_results.json ADDED
@@ -0,0 +1,8 @@
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "total_flos": 0.0,
3
+ "train_loss": 0.21294908598650353,
4
+ "train_runtime": 95825.8938,
5
+ "train_samples": 7500,
6
+ "train_samples_per_second": 0.235,
7
+ "train_steps_per_second": 0.029
8
+ }
checkpoint-1876/README.md ADDED
@@ -0,0 +1,202 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: Qwen/Qwen2.5-1.5B
3
+ library_name: peft
4
+ ---
5
+
6
+ # Model Card for Model ID
7
+
8
+ <!-- Provide a quick summary of what the model is/does. -->
9
+
10
+
11
+
12
+ ## Model Details
13
+
14
+ ### Model Description
15
+
16
+ <!-- Provide a longer summary of what this model is. -->
17
+
18
+
19
+
20
+ - **Developed by:** [More Information Needed]
21
+ - **Funded by [optional]:** [More Information Needed]
22
+ - **Shared by [optional]:** [More Information Needed]
23
+ - **Model type:** [More Information Needed]
24
+ - **Language(s) (NLP):** [More Information Needed]
25
+ - **License:** [More Information Needed]
26
+ - **Finetuned from model [optional]:** [More Information Needed]
27
+
28
+ ### Model Sources [optional]
29
+
30
+ <!-- Provide the basic links for the model. -->
31
+
32
+ - **Repository:** [More Information Needed]
33
+ - **Paper [optional]:** [More Information Needed]
34
+ - **Demo [optional]:** [More Information Needed]
35
+
36
+ ## Uses
37
+
38
+ <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
39
+
40
+ ### Direct Use
41
+
42
+ <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
43
+
44
+ [More Information Needed]
45
+
46
+ ### Downstream Use [optional]
47
+
48
+ <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
49
+
50
+ [More Information Needed]
51
+
52
+ ### Out-of-Scope Use
53
+
54
+ <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
55
+
56
+ [More Information Needed]
57
+
58
+ ## Bias, Risks, and Limitations
59
+
60
+ <!-- This section is meant to convey both technical and sociotechnical limitations. -->
61
+
62
+ [More Information Needed]
63
+
64
+ ### Recommendations
65
+
66
+ <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
67
+
68
+ Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
69
+
70
+ ## How to Get Started with the Model
71
+
72
+ Use the code below to get started with the model.
73
+
74
+ [More Information Needed]
75
+
76
+ ## Training Details
77
+
78
+ ### Training Data
79
+
80
+ <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
81
+
82
+ [More Information Needed]
83
+
84
+ ### Training Procedure
85
+
86
+ <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
87
+
88
+ #### Preprocessing [optional]
89
+
90
+ [More Information Needed]
91
+
92
+
93
+ #### Training Hyperparameters
94
+
95
+ - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
96
+
97
+ #### Speeds, Sizes, Times [optional]
98
+
99
+ <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
100
+
101
+ [More Information Needed]
102
+
103
+ ## Evaluation
104
+
105
+ <!-- This section describes the evaluation protocols and provides the results. -->
106
+
107
+ ### Testing Data, Factors & Metrics
108
+
109
+ #### Testing Data
110
+
111
+ <!-- This should link to a Dataset Card if possible. -->
112
+
113
+ [More Information Needed]
114
+
115
+ #### Factors
116
+
117
+ <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
118
+
119
+ [More Information Needed]
120
+
121
+ #### Metrics
122
+
123
+ <!-- These are the evaluation metrics being used, ideally with a description of why. -->
124
+
125
+ [More Information Needed]
126
+
127
+ ### Results
128
+
129
+ [More Information Needed]
130
+
131
+ #### Summary
132
+
133
+
134
+
135
+ ## Model Examination [optional]
136
+
137
+ <!-- Relevant interpretability work for the model goes here -->
138
+
139
+ [More Information Needed]
140
+
141
+ ## Environmental Impact
142
+
143
+ <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
144
+
145
+ Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
146
+
147
+ - **Hardware Type:** [More Information Needed]
148
+ - **Hours used:** [More Information Needed]
149
+ - **Cloud Provider:** [More Information Needed]
150
+ - **Compute Region:** [More Information Needed]
151
+ - **Carbon Emitted:** [More Information Needed]
152
+
153
+ ## Technical Specifications [optional]
154
+
155
+ ### Model Architecture and Objective
156
+
157
+ [More Information Needed]
158
+
159
+ ### Compute Infrastructure
160
+
161
+ [More Information Needed]
162
+
163
+ #### Hardware
164
+
165
+ [More Information Needed]
166
+
167
+ #### Software
168
+
169
+ [More Information Needed]
170
+
171
+ ## Citation [optional]
172
+
173
+ <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
174
+
175
+ **BibTeX:**
176
+
177
+ [More Information Needed]
178
+
179
+ **APA:**
180
+
181
+ [More Information Needed]
182
+
183
+ ## Glossary [optional]
184
+
185
+ <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
186
+
187
+ [More Information Needed]
188
+
189
+ ## More Information [optional]
190
+
191
+ [More Information Needed]
192
+
193
+ ## Model Card Authors [optional]
194
+
195
+ [More Information Needed]
196
+
197
+ ## Model Card Contact
198
+
199
+ [More Information Needed]
200
+ ### Framework versions
201
+
202
+ - PEFT 0.14.0
checkpoint-1876/adapter_config.json ADDED
@@ -0,0 +1,34 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alpha_pattern": {},
3
+ "auto_mapping": null,
4
+ "base_model_name_or_path": "Qwen/Qwen2.5-1.5B",
5
+ "bias": "none",
6
+ "eva_config": null,
7
+ "exclude_modules": null,
8
+ "fan_in_fan_out": false,
9
+ "inference_mode": true,
10
+ "init_lora_weights": true,
11
+ "layer_replication": null,
12
+ "layers_pattern": null,
13
+ "layers_to_transform": null,
14
+ "loftq_config": {},
15
+ "lora_alpha": 8,
16
+ "lora_bias": false,
17
+ "lora_dropout": 0.0,
18
+ "megatron_config": null,
19
+ "megatron_core": "megatron.core",
20
+ "modules_to_save": null,
21
+ "peft_type": "LORA",
22
+ "r": 32,
23
+ "rank_pattern": {},
24
+ "revision": null,
25
+ "target_modules": [
26
+ "q_proj",
27
+ "v_proj",
28
+ "embed_tokens",
29
+ "k_proj"
30
+ ],
31
+ "task_type": "CAUSAL_LM",
32
+ "use_dora": false,
33
+ "use_rslora": false
34
+ }
checkpoint-1876/adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a8e46761f61f3bfddbd76a744c1714b7fdc6ee2a8fdc7e7a9a602efe22934a56
3
+ size 488520640
checkpoint-1876/added_tokens.json ADDED
@@ -0,0 +1,24 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "</tool_call>": 151658,
3
+ "<tool_call>": 151657,
4
+ "<|box_end|>": 151649,
5
+ "<|box_start|>": 151648,
6
+ "<|endoftext|>": 151643,
7
+ "<|file_sep|>": 151664,
8
+ "<|fim_middle|>": 151660,
9
+ "<|fim_pad|>": 151662,
10
+ "<|fim_prefix|>": 151659,
11
+ "<|fim_suffix|>": 151661,
12
+ "<|im_end|>": 151645,
13
+ "<|im_start|>": 151644,
14
+ "<|image_pad|>": 151655,
15
+ "<|object_ref_end|>": 151647,
16
+ "<|object_ref_start|>": 151646,
17
+ "<|quad_end|>": 151651,
18
+ "<|quad_start|>": 151650,
19
+ "<|repo_name|>": 151663,
20
+ "<|video_pad|>": 151656,
21
+ "<|vision_end|>": 151653,
22
+ "<|vision_pad|>": 151654,
23
+ "<|vision_start|>": 151652
24
+ }
checkpoint-1876/global_step1875/bf16_zero_pp_rank_0_mp_rank_00_optim_states.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6d4389c6c361a47aeae8b4f2e6246e904a7f0364e62f83400a8972e88f8c36db
3
+ size 130520624
checkpoint-1876/global_step1875/mp_rank_00_model_states.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d9db778bee26f573f90d8b34629fd5b990bcde4b1922cc2210ee73ed40b36e4c
3
+ size 488645432
checkpoint-1876/latest ADDED
@@ -0,0 +1 @@
 
 
1
+ global_step1875
checkpoint-1876/merges.txt ADDED
The diff for this file is too large to render. See raw diff
 
checkpoint-1876/rng_state.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7a7836d60f20134f3d9313f7612d26f0024f4c05fe0ccd1e58a97556452c2ebb
3
+ size 14244
checkpoint-1876/scheduler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3499db549e30a7e2b1735bafe664c530646f254a8a7c3ec3b6b3d3c9d1138a84
3
+ size 1064
checkpoint-1876/special_tokens_map.json ADDED
@@ -0,0 +1,31 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "additional_special_tokens": [
3
+ "<|im_start|>",
4
+ "<|im_end|>",
5
+ "<|object_ref_start|>",
6
+ "<|object_ref_end|>",
7
+ "<|box_start|>",
8
+ "<|box_end|>",
9
+ "<|quad_start|>",
10
+ "<|quad_end|>",
11
+ "<|vision_start|>",
12
+ "<|vision_end|>",
13
+ "<|vision_pad|>",
14
+ "<|image_pad|>",
15
+ "<|video_pad|>"
16
+ ],
17
+ "eos_token": {
18
+ "content": "<|endoftext|>",
19
+ "lstrip": false,
20
+ "normalized": false,
21
+ "rstrip": false,
22
+ "single_word": false
23
+ },
24
+ "pad_token": {
25
+ "content": "<|endoftext|>",
26
+ "lstrip": false,
27
+ "normalized": false,
28
+ "rstrip": false,
29
+ "single_word": false
30
+ }
31
+ }
checkpoint-1876/tokenizer.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5eee858c5123a4279c3e1f7b81247343f356ac767940b2692a928ad929543214
3
+ size 11422063
checkpoint-1876/tokenizer_config.json ADDED
@@ -0,0 +1,209 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_bos_token": false,
3
+ "add_prefix_space": false,
4
+ "added_tokens_decoder": {
5
+ "151643": {
6
+ "content": "<|endoftext|>",
7
+ "lstrip": false,
8
+ "normalized": false,
9
+ "rstrip": false,
10
+ "single_word": false,
11
+ "special": true
12
+ },
13
+ "151644": {
14
+ "content": "<|im_start|>",
15
+ "lstrip": false,
16
+ "normalized": false,
17
+ "rstrip": false,
18
+ "single_word": false,
19
+ "special": true
20
+ },
21
+ "151645": {
22
+ "content": "<|im_end|>",
23
+ "lstrip": false,
24
+ "normalized": false,
25
+ "rstrip": false,
26
+ "single_word": false,
27
+ "special": true
28
+ },
29
+ "151646": {
30
+ "content": "<|object_ref_start|>",
31
+ "lstrip": false,
32
+ "normalized": false,
33
+ "rstrip": false,
34
+ "single_word": false,
35
+ "special": true
36
+ },
37
+ "151647": {
38
+ "content": "<|object_ref_end|>",
39
+ "lstrip": false,
40
+ "normalized": false,
41
+ "rstrip": false,
42
+ "single_word": false,
43
+ "special": true
44
+ },
45
+ "151648": {
46
+ "content": "<|box_start|>",
47
+ "lstrip": false,
48
+ "normalized": false,
49
+ "rstrip": false,
50
+ "single_word": false,
51
+ "special": true
52
+ },
53
+ "151649": {
54
+ "content": "<|box_end|>",
55
+ "lstrip": false,
56
+ "normalized": false,
57
+ "rstrip": false,
58
+ "single_word": false,
59
+ "special": true
60
+ },
61
+ "151650": {
62
+ "content": "<|quad_start|>",
63
+ "lstrip": false,
64
+ "normalized": false,
65
+ "rstrip": false,
66
+ "single_word": false,
67
+ "special": true
68
+ },
69
+ "151651": {
70
+ "content": "<|quad_end|>",
71
+ "lstrip": false,
72
+ "normalized": false,
73
+ "rstrip": false,
74
+ "single_word": false,
75
+ "special": true
76
+ },
77
+ "151652": {
78
+ "content": "<|vision_start|>",
79
+ "lstrip": false,
80
+ "normalized": false,
81
+ "rstrip": false,
82
+ "single_word": false,
83
+ "special": true
84
+ },
85
+ "151653": {
86
+ "content": "<|vision_end|>",
87
+ "lstrip": false,
88
+ "normalized": false,
89
+ "rstrip": false,
90
+ "single_word": false,
91
+ "special": true
92
+ },
93
+ "151654": {
94
+ "content": "<|vision_pad|>",
95
+ "lstrip": false,
96
+ "normalized": false,
97
+ "rstrip": false,
98
+ "single_word": false,
99
+ "special": true
100
+ },
101
+ "151655": {
102
+ "content": "<|image_pad|>",
103
+ "lstrip": false,
104
+ "normalized": false,
105
+ "rstrip": false,
106
+ "single_word": false,
107
+ "special": true
108
+ },
109
+ "151656": {
110
+ "content": "<|video_pad|>",
111
+ "lstrip": false,
112
+ "normalized": false,
113
+ "rstrip": false,
114
+ "single_word": false,
115
+ "special": true
116
+ },
117
+ "151657": {
118
+ "content": "<tool_call>",
119
+ "lstrip": false,
120
+ "normalized": false,
121
+ "rstrip": false,
122
+ "single_word": false,
123
+ "special": false
124
+ },
125
+ "151658": {
126
+ "content": "</tool_call>",
127
+ "lstrip": false,
128
+ "normalized": false,
129
+ "rstrip": false,
130
+ "single_word": false,
131
+ "special": false
132
+ },
133
+ "151659": {
134
+ "content": "<|fim_prefix|>",
135
+ "lstrip": false,
136
+ "normalized": false,
137
+ "rstrip": false,
138
+ "single_word": false,
139
+ "special": false
140
+ },
141
+ "151660": {
142
+ "content": "<|fim_middle|>",
143
+ "lstrip": false,
144
+ "normalized": false,
145
+ "rstrip": false,
146
+ "single_word": false,
147
+ "special": false
148
+ },
149
+ "151661": {
150
+ "content": "<|fim_suffix|>",
151
+ "lstrip": false,
152
+ "normalized": false,
153
+ "rstrip": false,
154
+ "single_word": false,
155
+ "special": false
156
+ },
157
+ "151662": {
158
+ "content": "<|fim_pad|>",
159
+ "lstrip": false,
160
+ "normalized": false,
161
+ "rstrip": false,
162
+ "single_word": false,
163
+ "special": false
164
+ },
165
+ "151663": {
166
+ "content": "<|repo_name|>",
167
+ "lstrip": false,
168
+ "normalized": false,
169
+ "rstrip": false,
170
+ "single_word": false,
171
+ "special": false
172
+ },
173
+ "151664": {
174
+ "content": "<|file_sep|>",
175
+ "lstrip": false,
176
+ "normalized": false,
177
+ "rstrip": false,
178
+ "single_word": false,
179
+ "special": false
180
+ }
181
+ },
182
+ "additional_special_tokens": [
183
+ "<|im_start|>",
184
+ "<|im_end|>",
185
+ "<|object_ref_start|>",
186
+ "<|object_ref_end|>",
187
+ "<|box_start|>",
188
+ "<|box_end|>",
189
+ "<|quad_start|>",
190
+ "<|quad_end|>",
191
+ "<|vision_start|>",
192
+ "<|vision_end|>",
193
+ "<|vision_pad|>",
194
+ "<|image_pad|>",
195
+ "<|video_pad|>"
196
+ ],
197
+ "bos_token": null,
198
+ "chat_template": "{%- if tools %}\n {{- '<|im_start|>system\\n' }}\n {%- if messages[0]['role'] == 'system' %}\n {{- messages[0]['content'] }}\n {%- else %}\n {{- 'You are a helpful assistant.' }}\n {%- endif %}\n {{- \"\\n\\n# Tools\\n\\nYou may call one or more functions to assist with the user query.\\n\\nYou are provided with function signatures within <tools></tools> XML tags:\\n<tools>\" }}\n {%- for tool in tools %}\n {{- \"\\n\" }}\n {{- tool | tojson }}\n {%- endfor %}\n {{- \"\\n</tools>\\n\\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\\n<tool_call>\\n{\\\"name\\\": <function-name>, \\\"arguments\\\": <args-json-object>}\\n</tool_call><|im_end|>\\n\" }}\n{%- else %}\n {%- if messages[0]['role'] == 'system' %}\n {{- '<|im_start|>system\\n' + messages[0]['content'] + '<|im_end|>\\n' }}\n {%- else %}\n {{- '<|im_start|>system\\nYou are a helpful assistant.<|im_end|>\\n' }}\n {%- endif %}\n{%- endif %}\n{%- for message in messages %}\n {%- if (message.role == \"user\") or (message.role == \"system\" and not loop.first) or (message.role == \"assistant\" and not message.tool_calls) %}\n {{- '<|im_start|>' + message.role + '\\n' + message.content + '<|im_end|>' + '\\n' }}\n {%- elif message.role == \"assistant\" %}\n {{- '<|im_start|>' + message.role }}\n {%- if message.content %}\n {{- '\\n' + message.content }}\n {%- endif %}\n {%- for tool_call in message.tool_calls %}\n {%- if tool_call.function is defined %}\n {%- set tool_call = tool_call.function %}\n {%- endif %}\n {{- '\\n<tool_call>\\n{\"name\": \"' }}\n {{- tool_call.name }}\n {{- '\", \"arguments\": ' }}\n {{- tool_call.arguments | tojson }}\n {{- '}\\n</tool_call>' }}\n {%- endfor %}\n {{- '<|im_end|>\\n' }}\n {%- elif message.role == \"tool\" %}\n {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != \"tool\") %}\n {{- '<|im_start|>user' }}\n {%- endif %}\n {{- '\\n<tool_response>\\n' }}\n {{- message.content }}\n {{- '\\n</tool_response>' }}\n {%- if loop.last or (messages[loop.index0 + 1].role != \"tool\") %}\n {{- '<|im_end|>\\n' }}\n {%- endif %}\n {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n {{- '<|im_start|>assistant\\n' }}\n{%- endif %}\n",
199
+ "clean_up_tokenization_spaces": false,
200
+ "eos_token": "<|endoftext|>",
201
+ "errors": "replace",
202
+ "extra_special_tokens": {},
203
+ "model_max_length": 131072,
204
+ "pad_token": "<|endoftext|>",
205
+ "padding_side": "left",
206
+ "split_special_tokens": false,
207
+ "tokenizer_class": "Qwen2Tokenizer",
208
+ "unk_token": null
209
+ }
checkpoint-1876/trainer_state.json ADDED
@@ -0,0 +1,2464 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_metric": null,
3
+ "best_model_checkpoint": null,
4
+ "epoch": 2.0,
5
+ "eval_steps": 10,
6
+ "global_step": 1876,
7
+ "is_hyper_param_search": false,
8
+ "is_local_process_zero": true,
9
+ "is_world_process_zero": true,
10
+ "log_history": [
11
+ {
12
+ "completion_length": 121.971875,
13
+ "epoch": 0.010666666666666666,
14
+ "grad_norm": 0.156667098402977,
15
+ "kl": 2.0313262939453126e-05,
16
+ "learning_rate": 1.0638297872340426e-05,
17
+ "loss": 0.001,
18
+ "reward": 0.0125,
19
+ "reward_std": 0.025,
20
+ "rewards/accuracy_reward": 0.009375,
21
+ "rewards/format_reward": 0.003125,
22
+ "step": 10
23
+ },
24
+ {
25
+ "completion_length": 122.521875,
26
+ "epoch": 0.021333333333333333,
27
+ "grad_norm": 0.0012713409960269928,
28
+ "kl": 0.00021836161613464355,
29
+ "learning_rate": 2.1276595744680852e-05,
30
+ "loss": 0.0051,
31
+ "reward": 0.015625,
32
+ "reward_std": 0.025966878235340118,
33
+ "rewards/accuracy_reward": 0.015625,
34
+ "rewards/format_reward": 0.0,
35
+ "step": 20
36
+ },
37
+ {
38
+ "completion_length": 117.5125,
39
+ "epoch": 0.032,
40
+ "grad_norm": 0.002654253738000989,
41
+ "kl": 0.0003068089485168457,
42
+ "learning_rate": 3.1914893617021275e-05,
43
+ "loss": -0.0002,
44
+ "reward": 0.00625,
45
+ "reward_std": 0.007216878235340118,
46
+ "rewards/accuracy_reward": 0.00625,
47
+ "rewards/format_reward": 0.0,
48
+ "step": 30
49
+ },
50
+ {
51
+ "completion_length": 118.871875,
52
+ "epoch": 0.042666666666666665,
53
+ "grad_norm": 0.00353299081325531,
54
+ "kl": 0.000412750244140625,
55
+ "learning_rate": 4.2553191489361704e-05,
56
+ "loss": 0.0055,
57
+ "reward": 0.009375,
58
+ "reward_std": 0.01875,
59
+ "rewards/accuracy_reward": 0.00625,
60
+ "rewards/format_reward": 0.003125,
61
+ "step": 40
62
+ },
63
+ {
64
+ "completion_length": 121.046875,
65
+ "epoch": 0.05333333333333334,
66
+ "grad_norm": 0.003619612194597721,
67
+ "kl": 0.0004070043563842773,
68
+ "learning_rate": 5.319148936170213e-05,
69
+ "loss": 0.0,
70
+ "reward": 0.0,
71
+ "reward_std": 0.0,
72
+ "rewards/accuracy_reward": 0.0,
73
+ "rewards/format_reward": 0.0,
74
+ "step": 50
75
+ },
76
+ {
77
+ "completion_length": 119.4375,
78
+ "epoch": 0.064,
79
+ "grad_norm": 0.11174867302179337,
80
+ "kl": 0.00045168399810791016,
81
+ "learning_rate": 6.382978723404255e-05,
82
+ "loss": 0.0064,
83
+ "reward": 0.01875,
84
+ "reward_std": 0.0375,
85
+ "rewards/accuracy_reward": 0.015625,
86
+ "rewards/format_reward": 0.003125,
87
+ "step": 60
88
+ },
89
+ {
90
+ "completion_length": 119.05625,
91
+ "epoch": 0.07466666666666667,
92
+ "grad_norm": 0.006828859448432922,
93
+ "kl": 0.0011888980865478516,
94
+ "learning_rate": 7.446808510638297e-05,
95
+ "loss": 0.0012,
96
+ "reward": 0.0125,
97
+ "reward_std": 0.025,
98
+ "rewards/accuracy_reward": 0.0125,
99
+ "rewards/format_reward": 0.0,
100
+ "step": 70
101
+ },
102
+ {
103
+ "completion_length": 120.28125,
104
+ "epoch": 0.08533333333333333,
105
+ "grad_norm": 0.0064537739381194115,
106
+ "kl": 0.0019659996032714844,
107
+ "learning_rate": 8.510638297872341e-05,
108
+ "loss": 0.0028,
109
+ "reward": 0.0125,
110
+ "reward_std": 0.025,
111
+ "rewards/accuracy_reward": 0.0125,
112
+ "rewards/format_reward": 0.0,
113
+ "step": 80
114
+ },
115
+ {
116
+ "completion_length": 117.559375,
117
+ "epoch": 0.096,
118
+ "grad_norm": 0.09068689495325089,
119
+ "kl": 0.0025023460388183595,
120
+ "learning_rate": 9.574468085106382e-05,
121
+ "loss": 0.003,
122
+ "reward": 0.021875,
123
+ "reward_std": 0.04375,
124
+ "rewards/accuracy_reward": 0.01875,
125
+ "rewards/format_reward": 0.003125,
126
+ "step": 90
127
+ },
128
+ {
129
+ "completion_length": 117.68125,
130
+ "epoch": 0.10666666666666667,
131
+ "grad_norm": 0.16541939973831177,
132
+ "kl": 0.00291900634765625,
133
+ "learning_rate": 0.00010638297872340425,
134
+ "loss": 0.0008,
135
+ "reward": 0.021875,
136
+ "reward_std": 0.03125,
137
+ "rewards/accuracy_reward": 0.021875,
138
+ "rewards/format_reward": 0.0,
139
+ "step": 100
140
+ },
141
+ {
142
+ "completion_length": 116.971875,
143
+ "epoch": 0.11733333333333333,
144
+ "grad_norm": 0.07206544280052185,
145
+ "kl": 0.0038990020751953126,
146
+ "learning_rate": 0.00011702127659574467,
147
+ "loss": 0.0026,
148
+ "reward": 0.015625,
149
+ "reward_std": 0.03125,
150
+ "rewards/accuracy_reward": 0.0125,
151
+ "rewards/format_reward": 0.003125,
152
+ "step": 110
153
+ },
154
+ {
155
+ "completion_length": 114.996875,
156
+ "epoch": 0.128,
157
+ "grad_norm": 0.02286006510257721,
158
+ "kl": 0.007346725463867188,
159
+ "learning_rate": 0.0001276595744680851,
160
+ "loss": 0.0076,
161
+ "reward": 0.025,
162
+ "reward_std": 0.05,
163
+ "rewards/accuracy_reward": 0.015625,
164
+ "rewards/format_reward": 0.009375,
165
+ "step": 120
166
+ },
167
+ {
168
+ "completion_length": 119.315625,
169
+ "epoch": 0.13866666666666666,
170
+ "grad_norm": 0.015629781410098076,
171
+ "kl": 0.008090972900390625,
172
+ "learning_rate": 0.00013829787234042552,
173
+ "loss": 0.0011,
174
+ "reward": 0.009375,
175
+ "reward_std": 0.01875,
176
+ "rewards/accuracy_reward": 0.009375,
177
+ "rewards/format_reward": 0.0,
178
+ "step": 130
179
+ },
180
+ {
181
+ "completion_length": 121.821875,
182
+ "epoch": 0.14933333333333335,
183
+ "grad_norm": 0.15498439967632294,
184
+ "kl": 0.006272506713867187,
185
+ "learning_rate": 0.00014893617021276593,
186
+ "loss": -0.0012,
187
+ "reward": 0.021875,
188
+ "reward_std": 0.03846687823534012,
189
+ "rewards/accuracy_reward": 0.01875,
190
+ "rewards/format_reward": 0.003125,
191
+ "step": 140
192
+ },
193
+ {
194
+ "completion_length": 121.409375,
195
+ "epoch": 0.16,
196
+ "grad_norm": 0.18756870925426483,
197
+ "kl": 0.00465240478515625,
198
+ "learning_rate": 0.00015957446808510637,
199
+ "loss": 0.0012,
200
+ "reward": 0.021875,
201
+ "reward_std": 0.03846687823534012,
202
+ "rewards/accuracy_reward": 0.01875,
203
+ "rewards/format_reward": 0.003125,
204
+ "step": 150
205
+ },
206
+ {
207
+ "completion_length": 118.26875,
208
+ "epoch": 0.17066666666666666,
209
+ "grad_norm": 0.011626984924077988,
210
+ "kl": 0.01092681884765625,
211
+ "learning_rate": 0.00017021276595744682,
212
+ "loss": -0.0011,
213
+ "reward": 0.021875,
214
+ "reward_std": 0.03318375647068024,
215
+ "rewards/accuracy_reward": 0.01875,
216
+ "rewards/format_reward": 0.003125,
217
+ "step": 160
218
+ },
219
+ {
220
+ "completion_length": 119.25,
221
+ "epoch": 0.18133333333333335,
222
+ "grad_norm": 0.00764912273734808,
223
+ "kl": 0.00976104736328125,
224
+ "learning_rate": 0.0001808510638297872,
225
+ "loss": 0.0045,
226
+ "reward": 0.021875,
227
+ "reward_std": 0.04375,
228
+ "rewards/accuracy_reward": 0.021875,
229
+ "rewards/format_reward": 0.0,
230
+ "step": 170
231
+ },
232
+ {
233
+ "completion_length": 115.35,
234
+ "epoch": 0.192,
235
+ "grad_norm": 0.0785018652677536,
236
+ "kl": 0.014077377319335938,
237
+ "learning_rate": 0.00019148936170212765,
238
+ "loss": 0.0037,
239
+ "reward": 0.025,
240
+ "reward_std": 0.04471687823534012,
241
+ "rewards/accuracy_reward": 0.009375,
242
+ "rewards/format_reward": 0.015625,
243
+ "step": 180
244
+ },
245
+ {
246
+ "completion_length": 108.6125,
247
+ "epoch": 0.20266666666666666,
248
+ "grad_norm": 0.13107918202877045,
249
+ "kl": 0.039361572265625,
250
+ "learning_rate": 0.00020212765957446807,
251
+ "loss": 0.0412,
252
+ "reward": 0.11875,
253
+ "reward_std": 0.18907372057437896,
254
+ "rewards/accuracy_reward": 0.021875,
255
+ "rewards/format_reward": 0.096875,
256
+ "step": 190
257
+ },
258
+ {
259
+ "completion_length": 89.915625,
260
+ "epoch": 0.21333333333333335,
261
+ "grad_norm": 0.19012346863746643,
262
+ "kl": 0.08895263671875,
263
+ "learning_rate": 0.0002127659574468085,
264
+ "loss": 0.1321,
265
+ "reward": 0.46875,
266
+ "reward_std": 0.41404569447040557,
267
+ "rewards/accuracy_reward": 0.015625,
268
+ "rewards/format_reward": 0.453125,
269
+ "step": 200
270
+ },
271
+ {
272
+ "completion_length": 47.74375,
273
+ "epoch": 0.224,
274
+ "grad_norm": 0.4668453335762024,
275
+ "kl": 0.26416015625,
276
+ "learning_rate": 0.0002234042553191489,
277
+ "loss": 0.0712,
278
+ "reward": 0.871875,
279
+ "reward_std": 0.19805223047733306,
280
+ "rewards/accuracy_reward": 0.01875,
281
+ "rewards/format_reward": 0.853125,
282
+ "step": 210
283
+ },
284
+ {
285
+ "completion_length": 45.15625,
286
+ "epoch": 0.23466666666666666,
287
+ "grad_norm": 0.21052278578281403,
288
+ "kl": 0.3112213134765625,
289
+ "learning_rate": 0.00023404255319148934,
290
+ "loss": 0.0464,
291
+ "reward": 0.890625,
292
+ "reward_std": 0.11346687823534012,
293
+ "rewards/accuracy_reward": 0.0125,
294
+ "rewards/format_reward": 0.878125,
295
+ "step": 220
296
+ },
297
+ {
298
+ "completion_length": 57.2875,
299
+ "epoch": 0.24533333333333332,
300
+ "grad_norm": 0.16618619859218597,
301
+ "kl": 0.254522705078125,
302
+ "learning_rate": 0.00024468085106382976,
303
+ "loss": 0.0589,
304
+ "reward": 0.834375,
305
+ "reward_std": 0.12261751294136047,
306
+ "rewards/accuracy_reward": 0.0125,
307
+ "rewards/format_reward": 0.821875,
308
+ "step": 230
309
+ },
310
+ {
311
+ "completion_length": 68.153125,
312
+ "epoch": 0.256,
313
+ "grad_norm": 0.17739807069301605,
314
+ "kl": 0.214471435546875,
315
+ "learning_rate": 0.0002553191489361702,
316
+ "loss": 0.1375,
317
+ "reward": 0.659375,
318
+ "reward_std": 0.28527562469244006,
319
+ "rewards/accuracy_reward": 0.009375,
320
+ "rewards/format_reward": 0.65,
321
+ "step": 240
322
+ },
323
+ {
324
+ "completion_length": 52.709375,
325
+ "epoch": 0.26666666666666666,
326
+ "grad_norm": 0.09843996912240982,
327
+ "kl": 0.2847900390625,
328
+ "learning_rate": 0.0002659574468085106,
329
+ "loss": 0.1085,
330
+ "reward": 0.834375,
331
+ "reward_std": 0.290549997985363,
332
+ "rewards/accuracy_reward": 0.034375,
333
+ "rewards/format_reward": 0.8,
334
+ "step": 250
335
+ },
336
+ {
337
+ "completion_length": 51.55625,
338
+ "epoch": 0.2773333333333333,
339
+ "grad_norm": 0.1133696436882019,
340
+ "kl": 0.276953125,
341
+ "learning_rate": 0.00027659574468085103,
342
+ "loss": 0.0437,
343
+ "reward": 0.903125,
344
+ "reward_std": 0.1361730858683586,
345
+ "rewards/accuracy_reward": 0.025,
346
+ "rewards/format_reward": 0.878125,
347
+ "step": 260
348
+ },
349
+ {
350
+ "completion_length": 55.046875,
351
+ "epoch": 0.288,
352
+ "grad_norm": 0.14536090195178986,
353
+ "kl": 0.2501953125,
354
+ "learning_rate": 0.0002872340425531915,
355
+ "loss": 0.0588,
356
+ "reward": 0.878125,
357
+ "reward_std": 0.13846687823534012,
358
+ "rewards/accuracy_reward": 0.00625,
359
+ "rewards/format_reward": 0.871875,
360
+ "step": 270
361
+ },
362
+ {
363
+ "completion_length": 55.853125,
364
+ "epoch": 0.2986666666666667,
365
+ "grad_norm": 0.1799221634864807,
366
+ "kl": 0.3143310546875,
367
+ "learning_rate": 0.00029787234042553186,
368
+ "loss": 0.0609,
369
+ "reward": 0.90625,
370
+ "reward_std": 0.18080126941204072,
371
+ "rewards/accuracy_reward": 0.021875,
372
+ "rewards/format_reward": 0.884375,
373
+ "step": 280
374
+ },
375
+ {
376
+ "completion_length": 59.85,
377
+ "epoch": 0.30933333333333335,
378
+ "grad_norm": 0.10688479989767075,
379
+ "kl": 0.20706787109375,
380
+ "learning_rate": 0.0002999925930442553,
381
+ "loss": 0.0522,
382
+ "reward": 0.815625,
383
+ "reward_std": 0.2959165498614311,
384
+ "rewards/accuracy_reward": 0.065625,
385
+ "rewards/format_reward": 0.75,
386
+ "step": 290
387
+ },
388
+ {
389
+ "completion_length": 64.625,
390
+ "epoch": 0.32,
391
+ "grad_norm": 0.03851361572742462,
392
+ "kl": 0.201220703125,
393
+ "learning_rate": 0.00029996250354024344,
394
+ "loss": 0.0815,
395
+ "reward": 0.8625,
396
+ "reward_std": 0.21301814764738083,
397
+ "rewards/accuracy_reward": 0.0125,
398
+ "rewards/format_reward": 0.85,
399
+ "step": 300
400
+ },
401
+ {
402
+ "completion_length": 57.95,
403
+ "epoch": 0.33066666666666666,
404
+ "grad_norm": 0.23480646312236786,
405
+ "kl": 0.221240234375,
406
+ "learning_rate": 0.0002999092731927958,
407
+ "loss": 0.0292,
408
+ "reward": 0.921875,
409
+ "reward_std": 0.15895397514104842,
410
+ "rewards/accuracy_reward": 0.0375,
411
+ "rewards/format_reward": 0.884375,
412
+ "step": 310
413
+ },
414
+ {
415
+ "completion_length": 64.196875,
416
+ "epoch": 0.3413333333333333,
417
+ "grad_norm": 0.1151675432920456,
418
+ "kl": 0.20123291015625,
419
+ "learning_rate": 0.0002998329102159332,
420
+ "loss": 0.0491,
421
+ "reward": 0.83125,
422
+ "reward_std": 0.19258119761943818,
423
+ "rewards/accuracy_reward": 0.01875,
424
+ "rewards/format_reward": 0.8125,
425
+ "step": 320
426
+ },
427
+ {
428
+ "completion_length": 70.628125,
429
+ "epoch": 0.352,
430
+ "grad_norm": 0.1377689391374588,
431
+ "kl": 0.1906005859375,
432
+ "learning_rate": 0.0002997334263932927,
433
+ "loss": 0.0841,
434
+ "reward": 0.846875,
435
+ "reward_std": 0.21890811175107955,
436
+ "rewards/accuracy_reward": 0.015625,
437
+ "rewards/format_reward": 0.83125,
438
+ "step": 330
439
+ },
440
+ {
441
+ "completion_length": 61.54375,
442
+ "epoch": 0.3626666666666667,
443
+ "grad_norm": 0.0947548896074295,
444
+ "kl": 0.21240234375,
445
+ "learning_rate": 0.0002996108370763087,
446
+ "loss": 0.062,
447
+ "reward": 0.88125,
448
+ "reward_std": 0.13713996410369872,
449
+ "rewards/accuracy_reward": 0.015625,
450
+ "rewards/format_reward": 0.865625,
451
+ "step": 340
452
+ },
453
+ {
454
+ "completion_length": 60.109375,
455
+ "epoch": 0.37333333333333335,
456
+ "grad_norm": 0.14599719643592834,
457
+ "kl": 0.2236083984375,
458
+ "learning_rate": 0.0002994651611818448,
459
+ "loss": 0.0408,
460
+ "reward": 0.928125,
461
+ "reward_std": 0.18282372057437896,
462
+ "rewards/accuracy_reward": 0.028125,
463
+ "rewards/format_reward": 0.9,
464
+ "step": 350
465
+ },
466
+ {
467
+ "completion_length": 62.90625,
468
+ "epoch": 0.384,
469
+ "grad_norm": 0.3738599121570587,
470
+ "kl": 0.2464111328125,
471
+ "learning_rate": 0.00029929642118927394,
472
+ "loss": 0.0753,
473
+ "reward": 0.834375,
474
+ "reward_std": 0.20676814764738083,
475
+ "rewards/accuracy_reward": 0.025,
476
+ "rewards/format_reward": 0.809375,
477
+ "step": 360
478
+ },
479
+ {
480
+ "completion_length": 70.1625,
481
+ "epoch": 0.39466666666666667,
482
+ "grad_norm": 2.8762810230255127,
483
+ "kl": 0.88681640625,
484
+ "learning_rate": 0.00029910464313701013,
485
+ "loss": 0.2053,
486
+ "reward": 0.640625,
487
+ "reward_std": 0.38192625939846037,
488
+ "rewards/accuracy_reward": 0.009375,
489
+ "rewards/format_reward": 0.63125,
490
+ "step": 370
491
+ },
492
+ {
493
+ "completion_length": 43.578125,
494
+ "epoch": 0.4053333333333333,
495
+ "grad_norm": 0.9355350136756897,
496
+ "kl": 2.06865234375,
497
+ "learning_rate": 0.0002988898566184902,
498
+ "loss": 0.2631,
499
+ "reward": 0.725,
500
+ "reward_std": 0.32462068647146225,
501
+ "rewards/accuracy_reward": 0.0,
502
+ "rewards/format_reward": 0.725,
503
+ "step": 380
504
+ },
505
+ {
506
+ "completion_length": 44.19375,
507
+ "epoch": 0.416,
508
+ "grad_norm": 0.709173858165741,
509
+ "kl": 3.47939453125,
510
+ "learning_rate": 0.0002986520947776074,
511
+ "loss": 0.3225,
512
+ "reward": 0.6125,
513
+ "reward_std": 0.3950331017374992,
514
+ "rewards/accuracy_reward": 0.009375,
515
+ "rewards/format_reward": 0.603125,
516
+ "step": 390
517
+ },
518
+ {
519
+ "completion_length": 54.775,
520
+ "epoch": 0.4266666666666667,
521
+ "grad_norm": 0.6549698114395142,
522
+ "kl": 4.3202392578125,
523
+ "learning_rate": 0.0002983913943035968,
524
+ "loss": 0.3808,
525
+ "reward": 0.66875,
526
+ "reward_std": 0.3901100158691406,
527
+ "rewards/accuracy_reward": 0.01875,
528
+ "rewards/format_reward": 0.65,
529
+ "step": 400
530
+ },
531
+ {
532
+ "completion_length": 53.871875,
533
+ "epoch": 0.43733333333333335,
534
+ "grad_norm": 0.01826515607535839,
535
+ "kl": 2.477734375,
536
+ "learning_rate": 0.00029810779542537355,
537
+ "loss": 0.2661,
538
+ "reward": 0.79375,
539
+ "reward_std": 0.22999776750802994,
540
+ "rewards/accuracy_reward": 0.025,
541
+ "rewards/format_reward": 0.76875,
542
+ "step": 410
543
+ },
544
+ {
545
+ "completion_length": 49.434375,
546
+ "epoch": 0.448,
547
+ "grad_norm": 0.5134692192077637,
548
+ "kl": 2.07587890625,
549
+ "learning_rate": 0.0002978013419053255,
550
+ "loss": 0.2091,
551
+ "reward": 0.771875,
552
+ "reward_std": 0.26785253882408144,
553
+ "rewards/accuracy_reward": 0.0,
554
+ "rewards/format_reward": 0.771875,
555
+ "step": 420
556
+ },
557
+ {
558
+ "completion_length": 59.475,
559
+ "epoch": 0.45866666666666667,
560
+ "grad_norm": 0.7835673689842224,
561
+ "kl": 2.516943359375,
562
+ "learning_rate": 0.00029747208103256,
563
+ "loss": 0.2312,
564
+ "reward": 0.740625,
565
+ "reward_std": 0.31220938116312025,
566
+ "rewards/accuracy_reward": 0.0125,
567
+ "rewards/format_reward": 0.728125,
568
+ "step": 430
569
+ },
570
+ {
571
+ "completion_length": 58.815625,
572
+ "epoch": 0.4693333333333333,
573
+ "grad_norm": 0.021143430843949318,
574
+ "kl": 2.1959716796875,
575
+ "learning_rate": 0.0002971200636156068,
576
+ "loss": 0.2386,
577
+ "reward": 0.796875,
578
+ "reward_std": 0.2231356605887413,
579
+ "rewards/accuracy_reward": 0.0,
580
+ "rewards/format_reward": 0.796875,
581
+ "step": 440
582
+ },
583
+ {
584
+ "completion_length": 55.696875,
585
+ "epoch": 0.48,
586
+ "grad_norm": 1.8231980800628662,
587
+ "kl": 2.9314697265625,
588
+ "learning_rate": 0.00029674534397457745,
589
+ "loss": 0.3506,
590
+ "reward": 0.796875,
591
+ "reward_std": 0.2616912335157394,
592
+ "rewards/accuracy_reward": 0.021875,
593
+ "rewards/format_reward": 0.775,
594
+ "step": 450
595
+ },
596
+ {
597
+ "completion_length": 49.778125,
598
+ "epoch": 0.49066666666666664,
599
+ "grad_norm": 0.5252532362937927,
600
+ "kl": 1.47607421875,
601
+ "learning_rate": 0.00029634797993278333,
602
+ "loss": 0.2026,
603
+ "reward": 0.89375,
604
+ "reward_std": 0.11293471753597259,
605
+ "rewards/accuracy_reward": 0.0125,
606
+ "rewards/format_reward": 0.88125,
607
+ "step": 460
608
+ },
609
+ {
610
+ "completion_length": 57.903125,
611
+ "epoch": 0.5013333333333333,
612
+ "grad_norm": 0.10718824714422226,
613
+ "kl": 1.683837890625,
614
+ "learning_rate": 0.000295928032807813,
615
+ "loss": 0.1887,
616
+ "reward": 0.859375,
617
+ "reward_std": 0.1423343911767006,
618
+ "rewards/accuracy_reward": 0.00625,
619
+ "rewards/format_reward": 0.853125,
620
+ "step": 470
621
+ },
622
+ {
623
+ "completion_length": 63.734375,
624
+ "epoch": 0.512,
625
+ "grad_norm": 0.32101932168006897,
626
+ "kl": 2.9671875,
627
+ "learning_rate": 0.00029548556740206994,
628
+ "loss": 0.3254,
629
+ "reward": 0.79375,
630
+ "reward_std": 0.29874250292778015,
631
+ "rewards/accuracy_reward": 0.009375,
632
+ "rewards/format_reward": 0.784375,
633
+ "step": 480
634
+ },
635
+ {
636
+ "completion_length": 66.253125,
637
+ "epoch": 0.5226666666666666,
638
+ "grad_norm": 0.7132259011268616,
639
+ "kl": 2.6101806640625,
640
+ "learning_rate": 0.0002950206519927731,
641
+ "loss": 0.2574,
642
+ "reward": 0.728125,
643
+ "reward_std": 0.3086773693561554,
644
+ "rewards/accuracy_reward": 0.01875,
645
+ "rewards/format_reward": 0.709375,
646
+ "step": 490
647
+ },
648
+ {
649
+ "completion_length": 64.765625,
650
+ "epoch": 0.5333333333333333,
651
+ "grad_norm": 0.7293491959571838,
652
+ "kl": 3.3251220703125,
653
+ "learning_rate": 0.00029453335832142075,
654
+ "loss": 0.3315,
655
+ "reward": 0.75625,
656
+ "reward_std": 0.27030970752239225,
657
+ "rewards/accuracy_reward": 0.009375,
658
+ "rewards/format_reward": 0.746875,
659
+ "step": 500
660
+ },
661
+ {
662
+ "completion_length": 61.140625,
663
+ "epoch": 0.544,
664
+ "grad_norm": 0.4900813400745392,
665
+ "kl": 1.6069091796875,
666
+ "learning_rate": 0.0002940237615827202,
667
+ "loss": 0.162,
668
+ "reward": 0.86875,
669
+ "reward_std": 0.21899680644273758,
670
+ "rewards/accuracy_reward": 0.053125,
671
+ "rewards/format_reward": 0.815625,
672
+ "step": 510
673
+ },
674
+ {
675
+ "completion_length": 59.621875,
676
+ "epoch": 0.5546666666666666,
677
+ "grad_norm": 0.4984245002269745,
678
+ "kl": 1.695751953125,
679
+ "learning_rate": 0.00029349194041298435,
680
+ "loss": 0.2075,
681
+ "reward": 0.903125,
682
+ "reward_std": 0.16081304997205734,
683
+ "rewards/accuracy_reward": 0.034375,
684
+ "rewards/format_reward": 0.86875,
685
+ "step": 520
686
+ },
687
+ {
688
+ "completion_length": 62.209375,
689
+ "epoch": 0.5653333333333334,
690
+ "grad_norm": 0.25215986371040344,
691
+ "kl": 1.5575439453125,
692
+ "learning_rate": 0.0002929379768779971,
693
+ "loss": 0.1648,
694
+ "reward": 0.890625,
695
+ "reward_std": 0.17983439117670058,
696
+ "rewards/accuracy_reward": 0.040625,
697
+ "rewards/format_reward": 0.85,
698
+ "step": 530
699
+ },
700
+ {
701
+ "completion_length": 65.865625,
702
+ "epoch": 0.576,
703
+ "grad_norm": 0.1489488184452057,
704
+ "kl": 2.0063720703125,
705
+ "learning_rate": 0.0002923619564603501,
706
+ "loss": 0.187,
707
+ "reward": 0.78125,
708
+ "reward_std": 0.2043856605887413,
709
+ "rewards/accuracy_reward": 0.03125,
710
+ "rewards/format_reward": 0.75,
711
+ "step": 540
712
+ },
713
+ {
714
+ "completion_length": 68.68125,
715
+ "epoch": 0.5866666666666667,
716
+ "grad_norm": 0.24991311132907867,
717
+ "kl": 1.0421142578125,
718
+ "learning_rate": 0.00029176396804625135,
719
+ "loss": 0.0977,
720
+ "reward": 0.909375,
721
+ "reward_std": 0.1441847175359726,
722
+ "rewards/accuracy_reward": 0.021875,
723
+ "rewards/format_reward": 0.8875,
724
+ "step": 550
725
+ },
726
+ {
727
+ "completion_length": 64.2,
728
+ "epoch": 0.5973333333333334,
729
+ "grad_norm": 0.7193971872329712,
730
+ "kl": 2.2302490234375,
731
+ "learning_rate": 0.00029114410391180946,
732
+ "loss": 0.2166,
733
+ "reward": 0.834375,
734
+ "reward_std": 0.2048343911767006,
735
+ "rewards/accuracy_reward": 0.015625,
736
+ "rewards/format_reward": 0.81875,
737
+ "step": 560
738
+ },
739
+ {
740
+ "completion_length": 66.01875,
741
+ "epoch": 0.608,
742
+ "grad_norm": 0.59996098279953,
743
+ "kl": 2.652294921875,
744
+ "learning_rate": 0.0002905024597087945,
745
+ "loss": 0.2907,
746
+ "reward": 0.815625,
747
+ "reward_std": 0.20596464574337006,
748
+ "rewards/accuracy_reward": 0.028125,
749
+ "rewards/format_reward": 0.7875,
750
+ "step": 570
751
+ },
752
+ {
753
+ "completion_length": 69.009375,
754
+ "epoch": 0.6186666666666667,
755
+ "grad_norm": 0.32363754510879517,
756
+ "kl": 0.75888671875,
757
+ "learning_rate": 0.0002898391344498775,
758
+ "loss": 0.112,
759
+ "reward": 0.896875,
760
+ "reward_std": 0.14761751294136047,
761
+ "rewards/accuracy_reward": 0.034375,
762
+ "rewards/format_reward": 0.8625,
763
+ "step": 580
764
+ },
765
+ {
766
+ "completion_length": 58.490625,
767
+ "epoch": 0.6293333333333333,
768
+ "grad_norm": 0.5817243456840515,
769
+ "kl": 3.2273193359375,
770
+ "learning_rate": 0.0002891542304933521,
771
+ "loss": 0.3775,
772
+ "reward": 0.796875,
773
+ "reward_std": 0.24620190411806106,
774
+ "rewards/accuracy_reward": 0.00625,
775
+ "rewards/format_reward": 0.790625,
776
+ "step": 590
777
+ },
778
+ {
779
+ "completion_length": 56.890625,
780
+ "epoch": 0.64,
781
+ "grad_norm": 0.561817467212677,
782
+ "kl": 1.345703125,
783
+ "learning_rate": 0.00028844785352733924,
784
+ "loss": 0.1409,
785
+ "reward": 0.884375,
786
+ "reward_std": 0.1315855011343956,
787
+ "rewards/accuracy_reward": 0.021875,
788
+ "rewards/format_reward": 0.8625,
789
+ "step": 600
790
+ },
791
+ {
792
+ "completion_length": 54.61875,
793
+ "epoch": 0.6506666666666666,
794
+ "grad_norm": 0.41451311111450195,
795
+ "kl": 1.3966552734375,
796
+ "learning_rate": 0.00028772011255347873,
797
+ "loss": 0.1476,
798
+ "reward": 0.890625,
799
+ "reward_std": 0.16838996410369872,
800
+ "rewards/accuracy_reward": 0.01875,
801
+ "rewards/format_reward": 0.871875,
802
+ "step": 610
803
+ },
804
+ {
805
+ "completion_length": 58.15,
806
+ "epoch": 0.6613333333333333,
807
+ "grad_norm": 0.38927924633026123,
808
+ "kl": 2.2388427734375,
809
+ "learning_rate": 0.00028697111987010865,
810
+ "loss": 0.2576,
811
+ "reward": 0.871875,
812
+ "reward_std": 0.1995512694120407,
813
+ "rewards/accuracy_reward": 0.028125,
814
+ "rewards/format_reward": 0.84375,
815
+ "step": 620
816
+ },
817
+ {
818
+ "completion_length": 57.20625,
819
+ "epoch": 0.672,
820
+ "grad_norm": 0.09751415997743607,
821
+ "kl": 0.793994140625,
822
+ "learning_rate": 0.0002862009910549369,
823
+ "loss": 0.0629,
824
+ "reward": 0.9375,
825
+ "reward_std": 0.10386751294136047,
826
+ "rewards/accuracy_reward": 0.01875,
827
+ "rewards/format_reward": 0.91875,
828
+ "step": 630
829
+ },
830
+ {
831
+ "completion_length": 65.846875,
832
+ "epoch": 0.6826666666666666,
833
+ "grad_norm": 0.2675510048866272,
834
+ "kl": 2.466015625,
835
+ "learning_rate": 0.0002854098449472061,
836
+ "loss": 0.2627,
837
+ "reward": 0.79375,
838
+ "reward_std": 0.2520918682217598,
839
+ "rewards/accuracy_reward": 0.015625,
840
+ "rewards/format_reward": 0.778125,
841
+ "step": 640
842
+ },
843
+ {
844
+ "completion_length": 62.98125,
845
+ "epoch": 0.6933333333333334,
846
+ "grad_norm": 0.15855202078819275,
847
+ "kl": 1.8398193359375,
848
+ "learning_rate": 0.00028459780362935527,
849
+ "loss": 0.177,
850
+ "reward": 0.91875,
851
+ "reward_std": 0.15879059880971907,
852
+ "rewards/accuracy_reward": 0.0125,
853
+ "rewards/format_reward": 0.90625,
854
+ "step": 650
855
+ },
856
+ {
857
+ "completion_length": 62.6,
858
+ "epoch": 0.704,
859
+ "grad_norm": 0.12087615579366684,
860
+ "kl": 2.0813720703125,
861
+ "learning_rate": 0.0002837649924081816,
862
+ "loss": 0.1866,
863
+ "reward": 0.90625,
864
+ "reward_std": 0.19479155987501146,
865
+ "rewards/accuracy_reward": 0.03125,
866
+ "rewards/format_reward": 0.875,
867
+ "step": 660
868
+ },
869
+ {
870
+ "completion_length": 65.925,
871
+ "epoch": 0.7146666666666667,
872
+ "grad_norm": 0.39411771297454834,
873
+ "kl": 1.392919921875,
874
+ "learning_rate": 0.00028291153979550387,
875
+ "loss": 0.2015,
876
+ "reward": 0.915625,
877
+ "reward_std": 0.1775405988097191,
878
+ "rewards/accuracy_reward": 0.028125,
879
+ "rewards/format_reward": 0.8875,
880
+ "step": 670
881
+ },
882
+ {
883
+ "completion_length": 64.378125,
884
+ "epoch": 0.7253333333333334,
885
+ "grad_norm": 1.1659783124923706,
886
+ "kl": 2.8165283203125,
887
+ "learning_rate": 0.00028203757748833174,
888
+ "loss": 0.3109,
889
+ "reward": 0.778125,
890
+ "reward_std": 0.21169123351573943,
891
+ "rewards/accuracy_reward": 0.003125,
892
+ "rewards/format_reward": 0.775,
893
+ "step": 680
894
+ },
895
+ {
896
+ "completion_length": 57.334375,
897
+ "epoch": 0.736,
898
+ "grad_norm": 0.27627384662628174,
899
+ "kl": 1.0085693359375,
900
+ "learning_rate": 0.0002811432403485437,
901
+ "loss": 0.1226,
902
+ "reward": 0.859375,
903
+ "reward_std": 0.11540063470602036,
904
+ "rewards/accuracy_reward": 0.009375,
905
+ "rewards/format_reward": 0.85,
906
+ "step": 690
907
+ },
908
+ {
909
+ "completion_length": 54.6,
910
+ "epoch": 0.7466666666666667,
911
+ "grad_norm": 0.4506663382053375,
912
+ "kl": 2.3274169921875,
913
+ "learning_rate": 0.00028022866638207624,
914
+ "loss": 0.2726,
915
+ "reward": 0.853125,
916
+ "reward_std": 0.2143363133072853,
917
+ "rewards/accuracy_reward": 0.025,
918
+ "rewards/format_reward": 0.828125,
919
+ "step": 700
920
+ },
921
+ {
922
+ "completion_length": 64.646875,
923
+ "epoch": 0.7573333333333333,
924
+ "grad_norm": 0.24161870777606964,
925
+ "kl": 1.10751953125,
926
+ "learning_rate": 0.00027929399671762793,
927
+ "loss": 0.1497,
928
+ "reward": 0.878125,
929
+ "reward_std": 0.18096464574337007,
930
+ "rewards/accuracy_reward": 0.04375,
931
+ "rewards/format_reward": 0.834375,
932
+ "step": 710
933
+ },
934
+ {
935
+ "completion_length": 65.128125,
936
+ "epoch": 0.768,
937
+ "grad_norm": 0.22652657330036163,
938
+ "kl": 1.7567138671875,
939
+ "learning_rate": 0.00027833937558488183,
940
+ "loss": 0.1692,
941
+ "reward": 0.865625,
942
+ "reward_std": 0.19575843811035157,
943
+ "rewards/accuracy_reward": 0.059375,
944
+ "rewards/format_reward": 0.80625,
945
+ "step": 720
946
+ },
947
+ {
948
+ "completion_length": 76.259375,
949
+ "epoch": 0.7786666666666666,
950
+ "grad_norm": 0.46417316794395447,
951
+ "kl": 3.4798828125,
952
+ "learning_rate": 0.0002773649502922495,
953
+ "loss": 0.3618,
954
+ "reward": 0.7125,
955
+ "reward_std": 0.31879488229751585,
956
+ "rewards/accuracy_reward": 0.0125,
957
+ "rewards/format_reward": 0.7,
958
+ "step": 730
959
+ },
960
+ {
961
+ "completion_length": 64.928125,
962
+ "epoch": 0.7893333333333333,
963
+ "grad_norm": 0.8743041753768921,
964
+ "kl": 2.166650390625,
965
+ "learning_rate": 0.00027637087120413933,
966
+ "loss": 0.2562,
967
+ "reward": 0.840625,
968
+ "reward_std": 0.2851921945810318,
969
+ "rewards/accuracy_reward": 0.0375,
970
+ "rewards/format_reward": 0.803125,
971
+ "step": 740
972
+ },
973
+ {
974
+ "completion_length": 58.028125,
975
+ "epoch": 0.8,
976
+ "grad_norm": 0.18655003607273102,
977
+ "kl": 1.73994140625,
978
+ "learning_rate": 0.000275357291717754,
979
+ "loss": 0.191,
980
+ "reward": 0.909375,
981
+ "reward_std": 0.19460364878177644,
982
+ "rewards/accuracy_reward": 0.053125,
983
+ "rewards/format_reward": 0.85625,
984
+ "step": 750
985
+ },
986
+ {
987
+ "completion_length": 60.803125,
988
+ "epoch": 0.8106666666666666,
989
+ "grad_norm": 0.04459076747298241,
990
+ "kl": 1.7782470703125,
991
+ "learning_rate": 0.0002743243682394195,
992
+ "loss": 0.2117,
993
+ "reward": 0.83125,
994
+ "reward_std": 0.16213996410369874,
995
+ "rewards/accuracy_reward": 0.025,
996
+ "rewards/format_reward": 0.80625,
997
+ "step": 760
998
+ },
999
+ {
1000
+ "completion_length": 56.203125,
1001
+ "epoch": 0.8213333333333334,
1002
+ "grad_norm": 0.04220689460635185,
1003
+ "kl": 1.7406494140625,
1004
+ "learning_rate": 0.00027327226016044963,
1005
+ "loss": 0.1999,
1006
+ "reward": 0.878125,
1007
+ "reward_std": 0.1423343911767006,
1008
+ "rewards/accuracy_reward": 0.034375,
1009
+ "rewards/format_reward": 0.84375,
1010
+ "step": 770
1011
+ },
1012
+ {
1013
+ "completion_length": 58.375,
1014
+ "epoch": 0.832,
1015
+ "grad_norm": 0.3807085156440735,
1016
+ "kl": 1.8222412109375,
1017
+ "learning_rate": 0.00027220112983255087,
1018
+ "loss": 0.2296,
1019
+ "reward": 0.903125,
1020
+ "reward_std": 0.20482564270496367,
1021
+ "rewards/accuracy_reward": 0.034375,
1022
+ "rewards/format_reward": 0.86875,
1023
+ "step": 780
1024
+ },
1025
+ {
1026
+ "completion_length": 63.378125,
1027
+ "epoch": 0.8426666666666667,
1028
+ "grad_norm": 0.01206011138856411,
1029
+ "kl": 2.458740234375,
1030
+ "learning_rate": 0.00027111114254276913,
1031
+ "loss": 0.3096,
1032
+ "reward": 0.84375,
1033
+ "reward_std": 0.2114198923110962,
1034
+ "rewards/accuracy_reward": 0.021875,
1035
+ "rewards/format_reward": 0.821875,
1036
+ "step": 790
1037
+ },
1038
+ {
1039
+ "completion_length": 59.7625,
1040
+ "epoch": 0.8533333333333334,
1041
+ "grad_norm": 0.40591439604759216,
1042
+ "kl": 1.378076171875,
1043
+ "learning_rate": 0.00027000246648798456,
1044
+ "loss": 0.1403,
1045
+ "reward": 0.934375,
1046
+ "reward_std": 0.14083535224199295,
1047
+ "rewards/accuracy_reward": 0.03125,
1048
+ "rewards/format_reward": 0.903125,
1049
+ "step": 800
1050
+ },
1051
+ {
1052
+ "completion_length": 62.284375,
1053
+ "epoch": 0.864,
1054
+ "grad_norm": 0.27511999011039734,
1055
+ "kl": 2.2107177734375,
1056
+ "learning_rate": 0.0002688752727489565,
1057
+ "loss": 0.2636,
1058
+ "reward": 0.8875,
1059
+ "reward_std": 0.21739855110645295,
1060
+ "rewards/accuracy_reward": 0.028125,
1061
+ "rewards/format_reward": 0.859375,
1062
+ "step": 810
1063
+ },
1064
+ {
1065
+ "completion_length": 65.265625,
1066
+ "epoch": 0.8746666666666667,
1067
+ "grad_norm": 0.2582601010799408,
1068
+ "kl": 2.3897705078125,
1069
+ "learning_rate": 0.00026772973526392453,
1070
+ "loss": 0.2965,
1071
+ "reward": 0.83125,
1072
+ "reward_std": 0.2494538262486458,
1073
+ "rewards/accuracy_reward": 0.028125,
1074
+ "rewards/format_reward": 0.803125,
1075
+ "step": 820
1076
+ },
1077
+ {
1078
+ "completion_length": 54.865625,
1079
+ "epoch": 0.8853333333333333,
1080
+ "grad_norm": 0.23494267463684082,
1081
+ "kl": 2.6015625,
1082
+ "learning_rate": 0.0002665660308017671,
1083
+ "loss": 0.252,
1084
+ "reward": 0.9,
1085
+ "reward_std": 0.23950843811035155,
1086
+ "rewards/accuracy_reward": 0.04375,
1087
+ "rewards/format_reward": 0.85625,
1088
+ "step": 830
1089
+ },
1090
+ {
1091
+ "completion_length": 55.503125,
1092
+ "epoch": 0.896,
1093
+ "grad_norm": 0.20798054337501526,
1094
+ "kl": 1.5889892578125,
1095
+ "learning_rate": 0.000265384338934725,
1096
+ "loss": 0.1996,
1097
+ "reward": 0.9375,
1098
+ "reward_std": 0.20120493620634078,
1099
+ "rewards/accuracy_reward": 0.0625,
1100
+ "rewards/format_reward": 0.875,
1101
+ "step": 840
1102
+ },
1103
+ {
1104
+ "completion_length": 59.909375,
1105
+ "epoch": 0.9066666666666666,
1106
+ "grad_norm": 0.23807695508003235,
1107
+ "kl": 1.656982421875,
1108
+ "learning_rate": 0.00026418484201069055,
1109
+ "loss": 0.194,
1110
+ "reward": 0.840625,
1111
+ "reward_std": 0.17524680644273757,
1112
+ "rewards/accuracy_reward": 0.034375,
1113
+ "rewards/format_reward": 0.80625,
1114
+ "step": 850
1115
+ },
1116
+ {
1117
+ "completion_length": 56.74375,
1118
+ "epoch": 0.9173333333333333,
1119
+ "grad_norm": 0.21559438109397888,
1120
+ "kl": 0.813427734375,
1121
+ "learning_rate": 0.00026296772512507025,
1122
+ "loss": 0.1054,
1123
+ "reward": 0.884375,
1124
+ "reward_std": 0.13916241526603698,
1125
+ "rewards/accuracy_reward": 0.021875,
1126
+ "rewards/format_reward": 0.8625,
1127
+ "step": 860
1128
+ },
1129
+ {
1130
+ "completion_length": 62.390625,
1131
+ "epoch": 0.928,
1132
+ "grad_norm": 0.1291944831609726,
1133
+ "kl": 1.9663330078125,
1134
+ "learning_rate": 0.0002617331760922218,
1135
+ "loss": 0.2316,
1136
+ "reward": 0.85625,
1137
+ "reward_std": 0.15685684233903885,
1138
+ "rewards/accuracy_reward": 0.0125,
1139
+ "rewards/format_reward": 0.84375,
1140
+ "step": 870
1141
+ },
1142
+ {
1143
+ "completion_length": 56.009375,
1144
+ "epoch": 0.9386666666666666,
1145
+ "grad_norm": 1.045857548713684,
1146
+ "kl": 1.652001953125,
1147
+ "learning_rate": 0.0002604813854164726,
1148
+ "loss": 0.1616,
1149
+ "reward": 0.9375,
1150
+ "reward_std": 0.16336943507194518,
1151
+ "rewards/accuracy_reward": 0.028125,
1152
+ "rewards/format_reward": 0.909375,
1153
+ "step": 880
1154
+ },
1155
+ {
1156
+ "completion_length": 64.446875,
1157
+ "epoch": 0.9493333333333334,
1158
+ "grad_norm": 0.33091413974761963,
1159
+ "kl": 3.235400390625,
1160
+ "learning_rate": 0.0002592125462627231,
1161
+ "loss": 0.3973,
1162
+ "reward": 0.796875,
1163
+ "reward_std": 0.2716366216540337,
1164
+ "rewards/accuracy_reward": 0.015625,
1165
+ "rewards/format_reward": 0.78125,
1166
+ "step": 890
1167
+ },
1168
+ {
1169
+ "completion_length": 59.0,
1170
+ "epoch": 0.96,
1171
+ "grad_norm": 0.25974419713020325,
1172
+ "kl": 1.752197265625,
1173
+ "learning_rate": 0.00025792685442663877,
1174
+ "loss": 0.1938,
1175
+ "reward": 0.89375,
1176
+ "reward_std": 0.1826515957713127,
1177
+ "rewards/accuracy_reward": 0.028125,
1178
+ "rewards/format_reward": 0.865625,
1179
+ "step": 900
1180
+ },
1181
+ {
1182
+ "completion_length": 57.4125,
1183
+ "epoch": 0.9706666666666667,
1184
+ "grad_norm": 0.2569887936115265,
1185
+ "kl": 2.5720703125,
1186
+ "learning_rate": 0.00025662450830443733,
1187
+ "loss": 0.3213,
1188
+ "reward": 0.846875,
1189
+ "reward_std": 0.22065922170877456,
1190
+ "rewards/accuracy_reward": 0.03125,
1191
+ "rewards/format_reward": 0.815625,
1192
+ "step": 910
1193
+ },
1194
+ {
1195
+ "completion_length": 51.140625,
1196
+ "epoch": 0.9813333333333333,
1197
+ "grad_norm": 0.18798935413360596,
1198
+ "kl": 1.6124267578125,
1199
+ "learning_rate": 0.0002553057088622736,
1200
+ "loss": 0.2214,
1201
+ "reward": 0.925,
1202
+ "reward_std": 0.1477062076330185,
1203
+ "rewards/accuracy_reward": 0.015625,
1204
+ "rewards/format_reward": 0.909375,
1205
+ "step": 920
1206
+ },
1207
+ {
1208
+ "completion_length": 53.921875,
1209
+ "epoch": 0.992,
1210
+ "grad_norm": 0.8309330940246582,
1211
+ "kl": 1.651806640625,
1212
+ "learning_rate": 0.0002539706596052286,
1213
+ "loss": 0.1893,
1214
+ "reward": 0.909375,
1215
+ "reward_std": 0.16504059880971908,
1216
+ "rewards/accuracy_reward": 0.03125,
1217
+ "rewards/format_reward": 0.878125,
1218
+ "step": 930
1219
+ },
1220
+ {
1221
+ "completion_length": 59.96052631578947,
1222
+ "epoch": 1.0021333333333333,
1223
+ "grad_norm": 0.3670661151409149,
1224
+ "kl": 3.8713250411184212,
1225
+ "learning_rate": 0.000252619566545906,
1226
+ "loss": 0.4192,
1227
+ "reward": 0.7796052631578947,
1228
+ "reward_std": 0.2917690135930714,
1229
+ "rewards/accuracy_reward": 0.01644736842105263,
1230
+ "rewards/format_reward": 0.7631578947368421,
1231
+ "step": 940
1232
+ },
1233
+ {
1234
+ "completion_length": 59.1125,
1235
+ "epoch": 1.0128,
1236
+ "grad_norm": 0.2787770926952362,
1237
+ "kl": 2.920068359375,
1238
+ "learning_rate": 0.0002512526381726427,
1239
+ "loss": 0.4194,
1240
+ "reward": 0.734375,
1241
+ "reward_std": 0.3439827933907509,
1242
+ "rewards/accuracy_reward": 0.01875,
1243
+ "rewards/format_reward": 0.715625,
1244
+ "step": 950
1245
+ },
1246
+ {
1247
+ "completion_length": 57.63125,
1248
+ "epoch": 1.0234666666666667,
1249
+ "grad_norm": 0.15397749841213226,
1250
+ "kl": 2.7442626953125,
1251
+ "learning_rate": 0.00024987008541733663,
1252
+ "loss": 0.3308,
1253
+ "reward": 0.81875,
1254
+ "reward_std": 0.24064744114875794,
1255
+ "rewards/accuracy_reward": 0.0125,
1256
+ "rewards/format_reward": 0.80625,
1257
+ "step": 960
1258
+ },
1259
+ {
1260
+ "completion_length": 51.134375,
1261
+ "epoch": 1.0341333333333333,
1262
+ "grad_norm": 0.41957736015319824,
1263
+ "kl": 1.035986328125,
1264
+ "learning_rate": 0.0002484721216228974,
1265
+ "loss": 0.1489,
1266
+ "reward": 0.9625,
1267
+ "reward_std": 0.10561862289905548,
1268
+ "rewards/accuracy_reward": 0.021875,
1269
+ "rewards/format_reward": 0.940625,
1270
+ "step": 970
1271
+ },
1272
+ {
1273
+ "completion_length": 56.68125,
1274
+ "epoch": 1.0448,
1275
+ "grad_norm": 0.11578945815563202,
1276
+ "kl": 1.0271240234375,
1277
+ "learning_rate": 0.0002470589625103255,
1278
+ "loss": 0.1162,
1279
+ "reward": 0.9,
1280
+ "reward_std": 0.13415063470602034,
1281
+ "rewards/accuracy_reward": 0.034375,
1282
+ "rewards/format_reward": 0.865625,
1283
+ "step": 980
1284
+ },
1285
+ {
1286
+ "completion_length": 55.071875,
1287
+ "epoch": 1.0554666666666668,
1288
+ "grad_norm": 0.006299301981925964,
1289
+ "kl": 1.3626953125,
1290
+ "learning_rate": 0.0002456308261454241,
1291
+ "loss": 0.1452,
1292
+ "reward": 0.909375,
1293
+ "reward_std": 0.1264015957713127,
1294
+ "rewards/accuracy_reward": 0.021875,
1295
+ "rewards/format_reward": 0.8875,
1296
+ "step": 990
1297
+ },
1298
+ {
1299
+ "completion_length": 58.0875,
1300
+ "epoch": 1.0661333333333334,
1301
+ "grad_norm": 0.007178621832281351,
1302
+ "kl": 1.4033203125,
1303
+ "learning_rate": 0.00024418793290514906,
1304
+ "loss": 0.1534,
1305
+ "reward": 0.86875,
1306
+ "reward_std": 0.13291241526603698,
1307
+ "rewards/accuracy_reward": 0.01875,
1308
+ "rewards/format_reward": 0.85,
1309
+ "step": 1000
1310
+ },
1311
+ {
1312
+ "completion_length": 58.828125,
1313
+ "epoch": 1.0768,
1314
+ "grad_norm": 0.1767469048500061,
1315
+ "kl": 1.26591796875,
1316
+ "learning_rate": 0.0002427305054436024,
1317
+ "loss": 0.1309,
1318
+ "reward": 0.915625,
1319
+ "reward_std": 0.15447435528039932,
1320
+ "rewards/accuracy_reward": 0.034375,
1321
+ "rewards/format_reward": 0.88125,
1322
+ "step": 1010
1323
+ },
1324
+ {
1325
+ "completion_length": 60.309375,
1326
+ "epoch": 1.0874666666666666,
1327
+ "grad_norm": 0.20854564011096954,
1328
+ "kl": 2.313330078125,
1329
+ "learning_rate": 0.00024125876865767438,
1330
+ "loss": 0.2191,
1331
+ "reward": 0.90625,
1332
+ "reward_std": 0.17595286518335343,
1333
+ "rewards/accuracy_reward": 0.053125,
1334
+ "rewards/format_reward": 0.853125,
1335
+ "step": 1020
1336
+ },
1337
+ {
1338
+ "completion_length": 60.540625,
1339
+ "epoch": 1.0981333333333334,
1340
+ "grad_norm": 0.13962095975875854,
1341
+ "kl": 2.0788818359375,
1342
+ "learning_rate": 0.0002397729496523396,
1343
+ "loss": 0.226,
1344
+ "reward": 0.890625,
1345
+ "reward_std": 0.18282372057437896,
1346
+ "rewards/accuracy_reward": 0.025,
1347
+ "rewards/format_reward": 0.865625,
1348
+ "step": 1030
1349
+ },
1350
+ {
1351
+ "completion_length": 58.89375,
1352
+ "epoch": 1.1088,
1353
+ "grad_norm": 0.25990164279937744,
1354
+ "kl": 1.064794921875,
1355
+ "learning_rate": 0.0002382732777056119,
1356
+ "loss": 0.1602,
1357
+ "reward": 0.915625,
1358
+ "reward_std": 0.16433631330728532,
1359
+ "rewards/accuracy_reward": 0.025,
1360
+ "rewards/format_reward": 0.890625,
1361
+ "step": 1040
1362
+ },
1363
+ {
1364
+ "completion_length": 58.478125,
1365
+ "epoch": 1.1194666666666666,
1366
+ "grad_norm": 0.21004174649715424,
1367
+ "kl": 1.47158203125,
1368
+ "learning_rate": 0.00023675998423316457,
1369
+ "loss": 0.1682,
1370
+ "reward": 0.953125,
1371
+ "reward_std": 0.18810684233903885,
1372
+ "rewards/accuracy_reward": 0.05,
1373
+ "rewards/format_reward": 0.903125,
1374
+ "step": 1050
1375
+ },
1376
+ {
1377
+ "completion_length": 59.39375,
1378
+ "epoch": 1.1301333333333332,
1379
+ "grad_norm": 0.08877279609441757,
1380
+ "kl": 1.388037109375,
1381
+ "learning_rate": 0.00023523330275262037,
1382
+ "loss": 0.1636,
1383
+ "reward": 0.9125,
1384
+ "reward_std": 0.1637136846780777,
1385
+ "rewards/accuracy_reward": 0.0375,
1386
+ "rewards/format_reward": 0.875,
1387
+ "step": 1060
1388
+ },
1389
+ {
1390
+ "completion_length": 63.709375,
1391
+ "epoch": 1.1408,
1392
+ "grad_norm": 0.218344584107399,
1393
+ "kl": 1.704052734375,
1394
+ "learning_rate": 0.00023369346884751706,
1395
+ "loss": 0.2163,
1396
+ "reward": 0.884375,
1397
+ "reward_std": 0.23439744114875793,
1398
+ "rewards/accuracy_reward": 0.046875,
1399
+ "rewards/format_reward": 0.8375,
1400
+ "step": 1070
1401
+ },
1402
+ {
1403
+ "completion_length": 60.93125,
1404
+ "epoch": 1.1514666666666666,
1405
+ "grad_norm": 0.24491117894649506,
1406
+ "kl": 1.15849609375,
1407
+ "learning_rate": 0.00023214072013095434,
1408
+ "loss": 0.1445,
1409
+ "reward": 0.925,
1410
+ "reward_std": 0.14206304997205735,
1411
+ "rewards/accuracy_reward": 0.034375,
1412
+ "rewards/format_reward": 0.890625,
1413
+ "step": 1080
1414
+ },
1415
+ {
1416
+ "completion_length": 55.40625,
1417
+ "epoch": 1.1621333333333332,
1418
+ "grad_norm": 1.4203561544418335,
1419
+ "kl": 1.4958740234375,
1420
+ "learning_rate": 0.00023057529620892773,
1421
+ "loss": 0.2111,
1422
+ "reward": 0.946875,
1423
+ "reward_std": 0.18555223047733307,
1424
+ "rewards/accuracy_reward": 0.040625,
1425
+ "rewards/format_reward": 0.90625,
1426
+ "step": 1090
1427
+ },
1428
+ {
1429
+ "completion_length": 61.6375,
1430
+ "epoch": 1.1728,
1431
+ "grad_norm": 0.07859649509191513,
1432
+ "kl": 2.87578125,
1433
+ "learning_rate": 0.00022899743864335462,
1434
+ "loss": 0.3232,
1435
+ "reward": 0.825,
1436
+ "reward_std": 0.22410253882408143,
1437
+ "rewards/accuracy_reward": 0.0125,
1438
+ "rewards/format_reward": 0.8125,
1439
+ "step": 1100
1440
+ },
1441
+ {
1442
+ "completion_length": 56.74375,
1443
+ "epoch": 1.1834666666666667,
1444
+ "grad_norm": 0.6798639893531799,
1445
+ "kl": 2.165380859375,
1446
+ "learning_rate": 0.0002274073909147986,
1447
+ "loss": 0.29,
1448
+ "reward": 0.86875,
1449
+ "reward_std": 0.17693375647068024,
1450
+ "rewards/accuracy_reward": 0.0125,
1451
+ "rewards/format_reward": 0.85625,
1452
+ "step": 1110
1453
+ },
1454
+ {
1455
+ "completion_length": 56.08125,
1456
+ "epoch": 1.1941333333333333,
1457
+ "grad_norm": 0.46122825145721436,
1458
+ "kl": 2.6029052734375,
1459
+ "learning_rate": 0.000225805398384898,
1460
+ "loss": 0.2877,
1461
+ "reward": 0.85625,
1462
+ "reward_std": 0.15879059880971907,
1463
+ "rewards/accuracy_reward": 0.01875,
1464
+ "rewards/format_reward": 0.8375,
1465
+ "step": 1120
1466
+ },
1467
+ {
1468
+ "completion_length": 53.4875,
1469
+ "epoch": 1.2048,
1470
+ "grad_norm": 0.0872046947479248,
1471
+ "kl": 1.120751953125,
1472
+ "learning_rate": 0.0002241917082585036,
1473
+ "loss": 0.1583,
1474
+ "reward": 0.959375,
1475
+ "reward_std": 0.16838996410369872,
1476
+ "rewards/accuracy_reward": 0.046875,
1477
+ "rewards/format_reward": 0.9125,
1478
+ "step": 1130
1479
+ },
1480
+ {
1481
+ "completion_length": 52.09375,
1482
+ "epoch": 1.2154666666666667,
1483
+ "grad_norm": 0.2135591208934784,
1484
+ "kl": 1.0230712890625,
1485
+ "learning_rate": 0.00022256656954553245,
1486
+ "loss": 0.1191,
1487
+ "reward": 0.9625,
1488
+ "reward_std": 0.14858439117670058,
1489
+ "rewards/accuracy_reward": 0.05625,
1490
+ "rewards/format_reward": 0.90625,
1491
+ "step": 1140
1492
+ },
1493
+ {
1494
+ "completion_length": 50.771875,
1495
+ "epoch": 1.2261333333333333,
1496
+ "grad_norm": 0.2628862261772156,
1497
+ "kl": 2.2327880859375,
1498
+ "learning_rate": 0.00022093023302254295,
1499
+ "loss": 0.2802,
1500
+ "reward": 0.953125,
1501
+ "reward_std": 0.19831304997205734,
1502
+ "rewards/accuracy_reward": 0.053125,
1503
+ "rewards/format_reward": 0.9,
1504
+ "step": 1150
1505
+ },
1506
+ {
1507
+ "completion_length": 55.653125,
1508
+ "epoch": 1.2368000000000001,
1509
+ "grad_norm": 0.01043323241174221,
1510
+ "kl": 1.43515625,
1511
+ "learning_rate": 0.0002192829511940371,
1512
+ "loss": 0.216,
1513
+ "reward": 0.871875,
1514
+ "reward_std": 0.17604155987501144,
1515
+ "rewards/accuracy_reward": 0.03125,
1516
+ "rewards/format_reward": 0.840625,
1517
+ "step": 1160
1518
+ },
1519
+ {
1520
+ "completion_length": 56.01875,
1521
+ "epoch": 1.2474666666666667,
1522
+ "grad_norm": 0.30781543254852295,
1523
+ "kl": 1.352490234375,
1524
+ "learning_rate": 0.00021762497825349663,
1525
+ "loss": 0.1604,
1526
+ "reward": 0.875,
1527
+ "reward_std": 0.18801814764738084,
1528
+ "rewards/accuracy_reward": 0.040625,
1529
+ "rewards/format_reward": 0.834375,
1530
+ "step": 1170
1531
+ },
1532
+ {
1533
+ "completion_length": 57.009375,
1534
+ "epoch": 1.2581333333333333,
1535
+ "grad_norm": 0.268877774477005,
1536
+ "kl": 1.6961669921875,
1537
+ "learning_rate": 0.00021595657004415777,
1538
+ "loss": 0.2207,
1539
+ "reward": 0.896875,
1540
+ "reward_std": 0.18351925760507584,
1541
+ "rewards/accuracy_reward": 0.03125,
1542
+ "rewards/format_reward": 0.865625,
1543
+ "step": 1180
1544
+ },
1545
+ {
1546
+ "completion_length": 53.909375,
1547
+ "epoch": 1.2688,
1548
+ "grad_norm": 0.10004394501447678,
1549
+ "kl": 1.055419921875,
1550
+ "learning_rate": 0.00021427798401953233,
1551
+ "loss": 0.1206,
1552
+ "reward": 0.915625,
1553
+ "reward_std": 0.10359617173671723,
1554
+ "rewards/accuracy_reward": 0.0375,
1555
+ "rewards/format_reward": 0.878125,
1556
+ "step": 1190
1557
+ },
1558
+ {
1559
+ "completion_length": 55.93125,
1560
+ "epoch": 1.2794666666666665,
1561
+ "grad_norm": 0.16535454988479614,
1562
+ "kl": 2.3431884765625,
1563
+ "learning_rate": 0.0002125894792036794,
1564
+ "loss": 0.3288,
1565
+ "reward": 0.903125,
1566
+ "reward_std": 0.22568152397871016,
1567
+ "rewards/accuracy_reward": 0.0375,
1568
+ "rewards/format_reward": 0.865625,
1569
+ "step": 1200
1570
+ },
1571
+ {
1572
+ "completion_length": 55.09375,
1573
+ "epoch": 1.2901333333333334,
1574
+ "grad_norm": 0.19274021685123444,
1575
+ "kl": 1.0988037109375,
1576
+ "learning_rate": 0.0002108913161512354,
1577
+ "loss": 0.1432,
1578
+ "reward": 0.9375,
1579
+ "reward_std": 0.12358439117670059,
1580
+ "rewards/accuracy_reward": 0.034375,
1581
+ "rewards/format_reward": 0.903125,
1582
+ "step": 1210
1583
+ },
1584
+ {
1585
+ "completion_length": 55.675,
1586
+ "epoch": 1.3008,
1587
+ "grad_norm": 0.15594810247421265,
1588
+ "kl": 1.42451171875,
1589
+ "learning_rate": 0.0002091837569072076,
1590
+ "loss": 0.1693,
1591
+ "reward": 0.94375,
1592
+ "reward_std": 0.16636751294136048,
1593
+ "rewards/accuracy_reward": 0.046875,
1594
+ "rewards/format_reward": 0.896875,
1595
+ "step": 1220
1596
+ },
1597
+ {
1598
+ "completion_length": 56.88125,
1599
+ "epoch": 1.3114666666666666,
1600
+ "grad_norm": 0.3196319341659546,
1601
+ "kl": 1.6291015625,
1602
+ "learning_rate": 0.00020746706496653765,
1603
+ "loss": 0.2144,
1604
+ "reward": 0.915625,
1605
+ "reward_std": 0.1927691087126732,
1606
+ "rewards/accuracy_reward": 0.046875,
1607
+ "rewards/format_reward": 0.86875,
1608
+ "step": 1230
1609
+ },
1610
+ {
1611
+ "completion_length": 58.7875,
1612
+ "epoch": 1.3221333333333334,
1613
+ "grad_norm": 0.13602705299854279,
1614
+ "kl": 1.5197509765625,
1615
+ "learning_rate": 0.00020574150523344152,
1616
+ "loss": 0.1651,
1617
+ "reward": 0.94375,
1618
+ "reward_std": 0.16670301407575608,
1619
+ "rewards/accuracy_reward": 0.0375,
1620
+ "rewards/format_reward": 0.90625,
1621
+ "step": 1240
1622
+ },
1623
+ {
1624
+ "completion_length": 62.996875,
1625
+ "epoch": 1.3328,
1626
+ "grad_norm": 0.05853046849370003,
1627
+ "kl": 1.815576171875,
1628
+ "learning_rate": 0.00020400734398053186,
1629
+ "loss": 0.1795,
1630
+ "reward": 0.86875,
1631
+ "reward_std": 0.19752006977796555,
1632
+ "rewards/accuracy_reward": 0.0375,
1633
+ "rewards/format_reward": 0.83125,
1634
+ "step": 1250
1635
+ },
1636
+ {
1637
+ "completion_length": 62.54375,
1638
+ "epoch": 1.3434666666666666,
1639
+ "grad_norm": 0.007495929021388292,
1640
+ "kl": 1.3890869140625,
1641
+ "learning_rate": 0.0002022648488077294,
1642
+ "loss": 0.1695,
1643
+ "reward": 0.884375,
1644
+ "reward_std": 0.1775405988097191,
1645
+ "rewards/accuracy_reward": 0.028125,
1646
+ "rewards/format_reward": 0.85625,
1647
+ "step": 1260
1648
+ },
1649
+ {
1650
+ "completion_length": 61.278125,
1651
+ "epoch": 1.3541333333333334,
1652
+ "grad_norm": 0.1842016726732254,
1653
+ "kl": 1.7712646484375,
1654
+ "learning_rate": 0.0002005142886009691,
1655
+ "loss": 0.2379,
1656
+ "reward": 0.875,
1657
+ "reward_std": 0.16706304997205734,
1658
+ "rewards/accuracy_reward": 0.021875,
1659
+ "rewards/format_reward": 0.853125,
1660
+ "step": 1270
1661
+ },
1662
+ {
1663
+ "completion_length": 62.025,
1664
+ "epoch": 1.3648,
1665
+ "grad_norm": 0.16039888560771942,
1666
+ "kl": 1.9799560546875,
1667
+ "learning_rate": 0.00019875593349070832,
1668
+ "loss": 0.2323,
1669
+ "reward": 0.91875,
1670
+ "reward_std": 0.20685684233903884,
1671
+ "rewards/accuracy_reward": 0.046875,
1672
+ "rewards/format_reward": 0.871875,
1673
+ "step": 1280
1674
+ },
1675
+ {
1676
+ "completion_length": 61.0375,
1677
+ "epoch": 1.3754666666666666,
1678
+ "grad_norm": 0.15333615243434906,
1679
+ "kl": 2.3615478515625,
1680
+ "learning_rate": 0.0001969900548102427,
1681
+ "loss": 0.2778,
1682
+ "reward": 0.84375,
1683
+ "reward_std": 0.20719234347343446,
1684
+ "rewards/accuracy_reward": 0.01875,
1685
+ "rewards/format_reward": 0.825,
1686
+ "step": 1290
1687
+ },
1688
+ {
1689
+ "completion_length": 58.328125,
1690
+ "epoch": 1.3861333333333334,
1691
+ "grad_norm": 0.07369455695152283,
1692
+ "kl": 2.516748046875,
1693
+ "learning_rate": 0.00019521692505383657,
1694
+ "loss": 0.3136,
1695
+ "reward": 0.85,
1696
+ "reward_std": 0.19249776750802994,
1697
+ "rewards/accuracy_reward": 0.025,
1698
+ "rewards/format_reward": 0.825,
1699
+ "step": 1300
1700
+ },
1701
+ {
1702
+ "completion_length": 55.35625,
1703
+ "epoch": 1.3968,
1704
+ "grad_norm": 0.2832612693309784,
1705
+ "kl": 1.73642578125,
1706
+ "learning_rate": 0.000193436817834674,
1707
+ "loss": 0.2319,
1708
+ "reward": 0.925,
1709
+ "reward_std": 0.21794123351573944,
1710
+ "rewards/accuracy_reward": 0.034375,
1711
+ "rewards/format_reward": 0.890625,
1712
+ "step": 1310
1713
+ },
1714
+ {
1715
+ "completion_length": 59.0875,
1716
+ "epoch": 1.4074666666666666,
1717
+ "grad_norm": 0.24120619893074036,
1718
+ "kl": 2.8723876953125,
1719
+ "learning_rate": 0.0001916500078426373,
1720
+ "loss": 0.3392,
1721
+ "reward": 0.8375,
1722
+ "reward_std": 0.22023502588272095,
1723
+ "rewards/accuracy_reward": 0.025,
1724
+ "rewards/format_reward": 0.8125,
1725
+ "step": 1320
1726
+ },
1727
+ {
1728
+ "completion_length": 55.025,
1729
+ "epoch": 1.4181333333333335,
1730
+ "grad_norm": 0.18106360733509064,
1731
+ "kl": 1.674365234375,
1732
+ "learning_rate": 0.0001898567708019196,
1733
+ "loss": 0.2313,
1734
+ "reward": 0.88125,
1735
+ "reward_std": 0.17693375647068024,
1736
+ "rewards/accuracy_reward": 0.01875,
1737
+ "rewards/format_reward": 0.8625,
1738
+ "step": 1330
1739
+ },
1740
+ {
1741
+ "completion_length": 55.15625,
1742
+ "epoch": 1.4288,
1743
+ "grad_norm": 0.1365566849708557,
1744
+ "kl": 2.2543212890625,
1745
+ "learning_rate": 0.00018805738342847727,
1746
+ "loss": 0.3175,
1747
+ "reward": 0.878125,
1748
+ "reward_std": 0.2048343911767006,
1749
+ "rewards/accuracy_reward": 0.028125,
1750
+ "rewards/format_reward": 0.85,
1751
+ "step": 1340
1752
+ },
1753
+ {
1754
+ "completion_length": 55.5625,
1755
+ "epoch": 1.4394666666666667,
1756
+ "grad_norm": 0.10590548813343048,
1757
+ "kl": 1.425244140625,
1758
+ "learning_rate": 0.00018625212338733,
1759
+ "loss": 0.1371,
1760
+ "reward": 0.8875,
1761
+ "reward_std": 0.10915063470602035,
1762
+ "rewards/accuracy_reward": 0.028125,
1763
+ "rewards/format_reward": 0.859375,
1764
+ "step": 1350
1765
+ },
1766
+ {
1767
+ "completion_length": 53.225,
1768
+ "epoch": 1.4501333333333333,
1769
+ "grad_norm": 0.15103192627429962,
1770
+ "kl": 1.1132080078125,
1771
+ "learning_rate": 0.00018444126924971387,
1772
+ "loss": 0.1228,
1773
+ "reward": 0.953125,
1774
+ "reward_std": 0.11838996410369873,
1775
+ "rewards/accuracy_reward": 0.0375,
1776
+ "rewards/format_reward": 0.915625,
1777
+ "step": 1360
1778
+ },
1779
+ {
1780
+ "completion_length": 57.265625,
1781
+ "epoch": 1.4607999999999999,
1782
+ "grad_norm": 0.05684982240200043,
1783
+ "kl": 1.13671875,
1784
+ "learning_rate": 0.0001826251004500947,
1785
+ "loss": 0.1437,
1786
+ "reward": 0.9125,
1787
+ "reward_std": 0.19223694801330565,
1788
+ "rewards/accuracy_reward": 0.065625,
1789
+ "rewards/format_reward": 0.846875,
1790
+ "step": 1370
1791
+ },
1792
+ {
1793
+ "completion_length": 59.325,
1794
+ "epoch": 1.4714666666666667,
1795
+ "grad_norm": 0.17307031154632568,
1796
+ "kl": 1.8568115234375,
1797
+ "learning_rate": 0.0001808038972430486,
1798
+ "loss": 0.2279,
1799
+ "reward": 0.871875,
1800
+ "reward_std": 0.16398502588272096,
1801
+ "rewards/accuracy_reward": 0.01875,
1802
+ "rewards/format_reward": 0.853125,
1803
+ "step": 1380
1804
+ },
1805
+ {
1806
+ "completion_length": 59.646875,
1807
+ "epoch": 1.4821333333333333,
1808
+ "grad_norm": 0.007796150632202625,
1809
+ "kl": 2.4506591796875,
1810
+ "learning_rate": 0.00017897794066001524,
1811
+ "loss": 0.2992,
1812
+ "reward": 0.84375,
1813
+ "reward_std": 0.20420301407575608,
1814
+ "rewards/accuracy_reward": 0.034375,
1815
+ "rewards/format_reward": 0.809375,
1816
+ "step": 1390
1817
+ },
1818
+ {
1819
+ "completion_length": 58.3,
1820
+ "epoch": 1.4928,
1821
+ "grad_norm": 0.1803148239850998,
1822
+ "kl": 1.1746826171875,
1823
+ "learning_rate": 0.00017714751246593197,
1824
+ "loss": 0.1374,
1825
+ "reward": 0.85625,
1826
+ "reward_std": 0.09665063470602035,
1827
+ "rewards/accuracy_reward": 0.0125,
1828
+ "rewards/format_reward": 0.84375,
1829
+ "step": 1400
1830
+ },
1831
+ {
1832
+ "completion_length": 51.30625,
1833
+ "epoch": 1.5034666666666667,
1834
+ "grad_norm": 0.12448029220104218,
1835
+ "kl": 1.7854248046875,
1836
+ "learning_rate": 0.00017531289511575425,
1837
+ "loss": 0.2174,
1838
+ "reward": 0.940625,
1839
+ "reward_std": 0.17032372057437897,
1840
+ "rewards/accuracy_reward": 0.04375,
1841
+ "rewards/format_reward": 0.896875,
1842
+ "step": 1410
1843
+ },
1844
+ {
1845
+ "completion_length": 50.446875,
1846
+ "epoch": 1.5141333333333333,
1847
+ "grad_norm": 0.1189781054854393,
1848
+ "kl": 1.4447998046875,
1849
+ "learning_rate": 0.0001734743717108699,
1850
+ "loss": 0.1672,
1851
+ "reward": 0.959375,
1852
+ "reward_std": 0.13282372057437897,
1853
+ "rewards/accuracy_reward": 0.0375,
1854
+ "rewards/format_reward": 0.921875,
1855
+ "step": 1420
1856
+ },
1857
+ {
1858
+ "completion_length": 55.278125,
1859
+ "epoch": 1.5248,
1860
+ "grad_norm": 0.10093328356742859,
1861
+ "kl": 1.5136474609375,
1862
+ "learning_rate": 0.0001716322259554132,
1863
+ "loss": 0.1768,
1864
+ "reward": 0.96875,
1865
+ "reward_std": 0.19523502588272096,
1866
+ "rewards/accuracy_reward": 0.065625,
1867
+ "rewards/format_reward": 0.903125,
1868
+ "step": 1430
1869
+ },
1870
+ {
1871
+ "completion_length": 59.934375,
1872
+ "epoch": 1.5354666666666668,
1873
+ "grad_norm": 0.2610551714897156,
1874
+ "kl": 1.778369140625,
1875
+ "learning_rate": 0.00016978674211248673,
1876
+ "loss": 0.2314,
1877
+ "reward": 0.84375,
1878
+ "reward_std": 0.20420301407575608,
1879
+ "rewards/accuracy_reward": 0.03125,
1880
+ "rewards/format_reward": 0.8125,
1881
+ "step": 1440
1882
+ },
1883
+ {
1884
+ "completion_length": 54.96875,
1885
+ "epoch": 1.5461333333333334,
1886
+ "grad_norm": 0.19814546406269073,
1887
+ "kl": 2.6193603515625,
1888
+ "learning_rate": 0.00016793820496029623,
1889
+ "loss": 0.3738,
1890
+ "reward": 0.88125,
1891
+ "reward_std": 0.2356409251689911,
1892
+ "rewards/accuracy_reward": 0.025,
1893
+ "rewards/format_reward": 0.85625,
1894
+ "step": 1450
1895
+ },
1896
+ {
1897
+ "completion_length": 51.703125,
1898
+ "epoch": 1.5568,
1899
+ "grad_norm": 0.1247173473238945,
1900
+ "kl": 2.2877197265625,
1901
+ "learning_rate": 0.000166086899748206,
1902
+ "loss": 0.236,
1903
+ "reward": 0.940625,
1904
+ "reward_std": 0.2197028651833534,
1905
+ "rewards/accuracy_reward": 0.053125,
1906
+ "rewards/format_reward": 0.8875,
1907
+ "step": 1460
1908
+ },
1909
+ {
1910
+ "completion_length": 51.096875,
1911
+ "epoch": 1.5674666666666668,
1912
+ "grad_norm": 0.16843904554843903,
1913
+ "kl": 1.2693115234375,
1914
+ "learning_rate": 0.0001642331121527223,
1915
+ "loss": 0.1801,
1916
+ "reward": 0.959375,
1917
+ "reward_std": 0.14867308586835862,
1918
+ "rewards/accuracy_reward": 0.04375,
1919
+ "rewards/format_reward": 0.915625,
1920
+ "step": 1470
1921
+ },
1922
+ {
1923
+ "completion_length": 55.465625,
1924
+ "epoch": 1.5781333333333334,
1925
+ "grad_norm": 0.12530925869941711,
1926
+ "kl": 1.403857421875,
1927
+ "learning_rate": 0.0001623771282334099,
1928
+ "loss": 0.1621,
1929
+ "reward": 0.921875,
1930
+ "reward_std": 0.17568152397871017,
1931
+ "rewards/accuracy_reward": 0.046875,
1932
+ "rewards/format_reward": 0.875,
1933
+ "step": 1480
1934
+ },
1935
+ {
1936
+ "completion_length": 58.23125,
1937
+ "epoch": 1.5888,
1938
+ "grad_norm": 0.016702894121408463,
1939
+ "kl": 0.8815673828125,
1940
+ "learning_rate": 0.00016051923438875035,
1941
+ "loss": 0.0918,
1942
+ "reward": 0.8875,
1943
+ "reward_std": 0.0879347175359726,
1944
+ "rewards/accuracy_reward": 0.034375,
1945
+ "rewards/format_reward": 0.853125,
1946
+ "step": 1490
1947
+ },
1948
+ {
1949
+ "completion_length": 53.565625,
1950
+ "epoch": 1.5994666666666668,
1951
+ "grad_norm": 0.09671846032142639,
1952
+ "kl": 0.7630126953125,
1953
+ "learning_rate": 0.00015865971731194738,
1954
+ "loss": 0.0861,
1955
+ "reward": 0.98125,
1956
+ "reward_std": 0.11971687823534012,
1957
+ "rewards/accuracy_reward": 0.040625,
1958
+ "rewards/format_reward": 0.940625,
1959
+ "step": 1500
1960
+ },
1961
+ {
1962
+ "completion_length": 57.234375,
1963
+ "epoch": 1.6101333333333332,
1964
+ "grad_norm": 0.16410210728645325,
1965
+ "kl": 0.922265625,
1966
+ "learning_rate": 0.00015679886394668707,
1967
+ "loss": 0.1231,
1968
+ "reward": 0.925,
1969
+ "reward_std": 0.10915063470602035,
1970
+ "rewards/accuracy_reward": 0.028125,
1971
+ "rewards/format_reward": 0.896875,
1972
+ "step": 1510
1973
+ },
1974
+ {
1975
+ "completion_length": 95.821875,
1976
+ "epoch": 1.6208,
1977
+ "grad_norm": 0.09543804824352264,
1978
+ "kl": 6.0548583984375,
1979
+ "learning_rate": 0.00015493696144285935,
1980
+ "loss": 0.4094,
1981
+ "reward": 0.284375,
1982
+ "reward_std": 0.2681046098470688,
1983
+ "rewards/accuracy_reward": 0.025,
1984
+ "rewards/format_reward": 0.259375,
1985
+ "step": 1520
1986
+ },
1987
+ {
1988
+ "completion_length": 113.446875,
1989
+ "epoch": 1.6314666666666666,
1990
+ "grad_norm": 0.006384687032550573,
1991
+ "kl": 0.34674072265625,
1992
+ "learning_rate": 0.00015307429711224754,
1993
+ "loss": 0.0323,
1994
+ "reward": 0.05,
1995
+ "reward_std": 0.08221687823534012,
1996
+ "rewards/accuracy_reward": 0.03125,
1997
+ "rewards/format_reward": 0.01875,
1998
+ "step": 1530
1999
+ },
2000
+ {
2001
+ "completion_length": 116.7,
2002
+ "epoch": 1.6421333333333332,
2003
+ "grad_norm": 0.03033365309238434,
2004
+ "kl": 0.18465576171875,
2005
+ "learning_rate": 0.0001512111583841933,
2006
+ "loss": 0.0367,
2007
+ "reward": 0.053125,
2008
+ "reward_std": 0.10625,
2009
+ "rewards/accuracy_reward": 0.028125,
2010
+ "rewards/format_reward": 0.025,
2011
+ "step": 1540
2012
+ },
2013
+ {
2014
+ "completion_length": 117.84375,
2015
+ "epoch": 1.6528,
2016
+ "grad_norm": 0.007193129975348711,
2017
+ "kl": 0.1654541015625,
2018
+ "learning_rate": 0.00014934783276124278,
2019
+ "loss": 0.0284,
2020
+ "reward": 0.059375,
2021
+ "reward_std": 0.08318375647068024,
2022
+ "rewards/accuracy_reward": 0.025,
2023
+ "rewards/format_reward": 0.034375,
2024
+ "step": 1550
2025
+ },
2026
+ {
2027
+ "completion_length": 111.459375,
2028
+ "epoch": 1.6634666666666666,
2029
+ "grad_norm": 0.024860132485628128,
2030
+ "kl": 0.17579345703125,
2031
+ "learning_rate": 0.00014748460777478208,
2032
+ "loss": 0.0751,
2033
+ "reward": 0.13125,
2034
+ "reward_std": 0.20580126941204072,
2035
+ "rewards/accuracy_reward": 0.009375,
2036
+ "rewards/format_reward": 0.121875,
2037
+ "step": 1560
2038
+ },
2039
+ {
2040
+ "completion_length": 91.90625,
2041
+ "epoch": 1.6741333333333333,
2042
+ "grad_norm": 0.07996781170368195,
2043
+ "kl": 0.19302978515625,
2044
+ "learning_rate": 0.00014562177094066812,
2045
+ "loss": 0.1666,
2046
+ "reward": 0.478125,
2047
+ "reward_std": 0.42759600281715393,
2048
+ "rewards/accuracy_reward": 0.021875,
2049
+ "rewards/format_reward": 0.45625,
2050
+ "step": 1570
2051
+ },
2052
+ {
2053
+ "completion_length": 73.571875,
2054
+ "epoch": 1.6848,
2055
+ "grad_norm": 0.07270823419094086,
2056
+ "kl": 0.2430419921875,
2057
+ "learning_rate": 0.0001437596097148615,
2058
+ "loss": 0.1744,
2059
+ "reward": 0.76875,
2060
+ "reward_std": 0.332449671626091,
2061
+ "rewards/accuracy_reward": 0.028125,
2062
+ "rewards/format_reward": 0.740625,
2063
+ "step": 1580
2064
+ },
2065
+ {
2066
+ "completion_length": 64.4875,
2067
+ "epoch": 1.6954666666666667,
2068
+ "grad_norm": 0.1185784786939621,
2069
+ "kl": 0.27333984375,
2070
+ "learning_rate": 0.00014189841144906926,
2071
+ "loss": 0.1684,
2072
+ "reward": 0.80625,
2073
+ "reward_std": 0.28343056291341784,
2074
+ "rewards/accuracy_reward": 0.028125,
2075
+ "rewards/format_reward": 0.778125,
2076
+ "step": 1590
2077
+ },
2078
+ {
2079
+ "completion_length": 59.8375,
2080
+ "epoch": 1.7061333333333333,
2081
+ "grad_norm": 0.30180656909942627,
2082
+ "kl": 0.5548828125,
2083
+ "learning_rate": 0.00014003846334640323,
2084
+ "loss": 0.2054,
2085
+ "reward": 0.740625,
2086
+ "reward_std": 0.29256718456745145,
2087
+ "rewards/accuracy_reward": 0.0125,
2088
+ "rewards/format_reward": 0.728125,
2089
+ "step": 1600
2090
+ },
2091
+ {
2092
+ "completion_length": 59.284375,
2093
+ "epoch": 1.7168,
2094
+ "grad_norm": 0.26433998346328735,
2095
+ "kl": 2.827392578125,
2096
+ "learning_rate": 0.00013818005241706145,
2097
+ "loss": 0.469,
2098
+ "reward": 0.70625,
2099
+ "reward_std": 0.33853629529476165,
2100
+ "rewards/accuracy_reward": 0.00625,
2101
+ "rewards/format_reward": 0.7,
2102
+ "step": 1610
2103
+ },
2104
+ {
2105
+ "completion_length": 53.78125,
2106
+ "epoch": 1.7274666666666667,
2107
+ "grad_norm": 0.2971569299697876,
2108
+ "kl": 3.8951171875,
2109
+ "learning_rate": 0.00013632346543403947,
2110
+ "loss": 0.451,
2111
+ "reward": 0.81875,
2112
+ "reward_std": 0.23273502588272094,
2113
+ "rewards/accuracy_reward": 0.0125,
2114
+ "rewards/format_reward": 0.80625,
2115
+ "step": 1620
2116
+ },
2117
+ {
2118
+ "completion_length": 53.903125,
2119
+ "epoch": 1.7381333333333333,
2120
+ "grad_norm": 0.0691404640674591,
2121
+ "kl": 1.3744140625,
2122
+ "learning_rate": 0.00013446898888887804,
2123
+ "loss": 0.1657,
2124
+ "reward": 0.93125,
2125
+ "reward_std": 0.13943375647068024,
2126
+ "rewards/accuracy_reward": 0.015625,
2127
+ "rewards/format_reward": 0.915625,
2128
+ "step": 1630
2129
+ },
2130
+ {
2131
+ "completion_length": 57.153125,
2132
+ "epoch": 1.7488000000000001,
2133
+ "grad_norm": 0.03758076950907707,
2134
+ "kl": 0.640966796875,
2135
+ "learning_rate": 0.00013261690894745442,
2136
+ "loss": 0.0775,
2137
+ "reward": 0.884375,
2138
+ "reward_std": 0.07596687823534012,
2139
+ "rewards/accuracy_reward": 0.0125,
2140
+ "rewards/format_reward": 0.871875,
2141
+ "step": 1640
2142
+ },
2143
+ {
2144
+ "completion_length": 58.0125,
2145
+ "epoch": 1.7594666666666665,
2146
+ "grad_norm": 0.009222053922712803,
2147
+ "kl": 1.1447265625,
2148
+ "learning_rate": 0.00013076751140582394,
2149
+ "loss": 0.1472,
2150
+ "reward": 0.88125,
2151
+ "reward_std": 0.14963996410369873,
2152
+ "rewards/accuracy_reward": 0.05,
2153
+ "rewards/format_reward": 0.83125,
2154
+ "step": 1650
2155
+ },
2156
+ {
2157
+ "completion_length": 55.746875,
2158
+ "epoch": 1.7701333333333333,
2159
+ "grad_norm": 0.029693789780139923,
2160
+ "kl": 1.351123046875,
2161
+ "learning_rate": 0.00012892108164611857,
2162
+ "loss": 0.161,
2163
+ "reward": 0.890625,
2164
+ "reward_std": 0.14867308586835862,
2165
+ "rewards/accuracy_reward": 0.021875,
2166
+ "rewards/format_reward": 0.86875,
2167
+ "step": 1660
2168
+ },
2169
+ {
2170
+ "completion_length": 53.184375,
2171
+ "epoch": 1.7808000000000002,
2172
+ "grad_norm": 0.23443636298179626,
2173
+ "kl": 1.46533203125,
2174
+ "learning_rate": 0.00012707790459250904,
2175
+ "loss": 0.1583,
2176
+ "reward": 0.9625,
2177
+ "reward_std": 0.14665063470602036,
2178
+ "rewards/accuracy_reward": 0.034375,
2179
+ "rewards/format_reward": 0.928125,
2180
+ "step": 1670
2181
+ },
2182
+ {
2183
+ "completion_length": 52.959375,
2184
+ "epoch": 1.7914666666666665,
2185
+ "grad_norm": 0.010363437235355377,
2186
+ "kl": 1.73623046875,
2187
+ "learning_rate": 0.0001252382646672384,
2188
+ "loss": 0.1757,
2189
+ "reward": 0.921875,
2190
+ "reward_std": 0.16504059880971908,
2191
+ "rewards/accuracy_reward": 0.021875,
2192
+ "rewards/format_reward": 0.9,
2193
+ "step": 1680
2194
+ },
2195
+ {
2196
+ "completion_length": 55.971875,
2197
+ "epoch": 1.8021333333333334,
2198
+ "grad_norm": 0.0745161771774292,
2199
+ "kl": 1.418701171875,
2200
+ "learning_rate": 0.00012340244574673238,
2201
+ "loss": 0.1882,
2202
+ "reward": 0.91875,
2203
+ "reward_std": 0.1851816728711128,
2204
+ "rewards/accuracy_reward": 0.046875,
2205
+ "rewards/format_reward": 0.871875,
2206
+ "step": 1690
2207
+ },
2208
+ {
2209
+ "completion_length": 57.903125,
2210
+ "epoch": 1.8128,
2211
+ "grad_norm": 0.09005508571863174,
2212
+ "kl": 1.6597900390625,
2213
+ "learning_rate": 0.000121570731117794,
2214
+ "loss": 0.1998,
2215
+ "reward": 0.8875,
2216
+ "reward_std": 0.1477808892726898,
2217
+ "rewards/accuracy_reward": 0.01875,
2218
+ "rewards/format_reward": 0.86875,
2219
+ "step": 1700
2220
+ },
2221
+ {
2222
+ "completion_length": 58.240625,
2223
+ "epoch": 1.8234666666666666,
2224
+ "grad_norm": 0.09877178072929382,
2225
+ "kl": 1.344873046875,
2226
+ "learning_rate": 0.00011974340343388972,
2227
+ "loss": 0.163,
2228
+ "reward": 0.921875,
2229
+ "reward_std": 0.18511751294136047,
2230
+ "rewards/accuracy_reward": 0.034375,
2231
+ "rewards/format_reward": 0.8875,
2232
+ "step": 1710
2233
+ },
2234
+ {
2235
+ "completion_length": 66.65,
2236
+ "epoch": 1.8341333333333334,
2237
+ "grad_norm": 0.09796544909477234,
2238
+ "kl": 2.161865234375,
2239
+ "learning_rate": 0.00011792074467153248,
2240
+ "loss": 0.2649,
2241
+ "reward": 0.81875,
2242
+ "reward_std": 0.20843056291341783,
2243
+ "rewards/accuracy_reward": 0.034375,
2244
+ "rewards/format_reward": 0.784375,
2245
+ "step": 1720
2246
+ },
2247
+ {
2248
+ "completion_length": 65.9,
2249
+ "epoch": 1.8448,
2250
+ "grad_norm": 0.12498176097869873,
2251
+ "kl": 2.451318359375,
2252
+ "learning_rate": 0.00011610303608677008,
2253
+ "loss": 0.3047,
2254
+ "reward": 0.809375,
2255
+ "reward_std": 0.268188039958477,
2256
+ "rewards/accuracy_reward": 0.040625,
2257
+ "rewards/format_reward": 0.76875,
2258
+ "step": 1730
2259
+ },
2260
+ {
2261
+ "completion_length": 61.346875,
2262
+ "epoch": 1.8554666666666666,
2263
+ "grad_norm": 0.40386486053466797,
2264
+ "kl": 3.258935546875,
2265
+ "learning_rate": 0.00011429055817178411,
2266
+ "loss": 0.3857,
2267
+ "reward": 0.846875,
2268
+ "reward_std": 0.28369315564632414,
2269
+ "rewards/accuracy_reward": 0.028125,
2270
+ "rewards/format_reward": 0.81875,
2271
+ "step": 1740
2272
+ },
2273
+ {
2274
+ "completion_length": 58.859375,
2275
+ "epoch": 1.8661333333333334,
2276
+ "grad_norm": 0.12184485048055649,
2277
+ "kl": 3.230029296875,
2278
+ "learning_rate": 0.00011248359061160698,
2279
+ "loss": 0.3751,
2280
+ "reward": 0.86875,
2281
+ "reward_std": 0.2617799282073975,
2282
+ "rewards/accuracy_reward": 0.03125,
2283
+ "rewards/format_reward": 0.8375,
2284
+ "step": 1750
2285
+ },
2286
+ {
2287
+ "completion_length": 57.69375,
2288
+ "epoch": 1.8768,
2289
+ "grad_norm": 0.1783817708492279,
2290
+ "kl": 2.4540283203125,
2291
+ "learning_rate": 0.00011068241224096347,
2292
+ "loss": 0.2785,
2293
+ "reward": 0.875,
2294
+ "reward_std": 0.20430223047733306,
2295
+ "rewards/accuracy_reward": 0.01875,
2296
+ "rewards/format_reward": 0.85625,
2297
+ "step": 1760
2298
+ },
2299
+ {
2300
+ "completion_length": 64.459375,
2301
+ "epoch": 1.8874666666666666,
2302
+ "grad_norm": 0.038270145654678345,
2303
+ "kl": 2.579345703125,
2304
+ "learning_rate": 0.00010888730100124353,
2305
+ "loss": 0.2952,
2306
+ "reward": 0.815625,
2307
+ "reward_std": 0.24689744114875795,
2308
+ "rewards/accuracy_reward": 0.025,
2309
+ "rewards/format_reward": 0.790625,
2310
+ "step": 1770
2311
+ },
2312
+ {
2313
+ "completion_length": 61.403125,
2314
+ "epoch": 1.8981333333333335,
2315
+ "grad_norm": 0.07670488953590393,
2316
+ "kl": 2.2768310546875,
2317
+ "learning_rate": 0.00010709853389761286,
2318
+ "loss": 0.3084,
2319
+ "reward": 0.884375,
2320
+ "reward_std": 0.23545301407575608,
2321
+ "rewards/accuracy_reward": 0.0375,
2322
+ "rewards/format_reward": 0.846875,
2323
+ "step": 1780
2324
+ },
2325
+ {
2326
+ "completion_length": 62.98125,
2327
+ "epoch": 1.9088,
2328
+ "grad_norm": 0.21110066771507263,
2329
+ "kl": 3.0247314453125,
2330
+ "learning_rate": 0.00010531638695626811,
2331
+ "loss": 0.3866,
2332
+ "reward": 0.8,
2333
+ "reward_std": 0.2520918682217598,
2334
+ "rewards/accuracy_reward": 0.03125,
2335
+ "rewards/format_reward": 0.76875,
2336
+ "step": 1790
2337
+ },
2338
+ {
2339
+ "completion_length": 64.06875,
2340
+ "epoch": 1.9194666666666667,
2341
+ "grad_norm": 0.19934044778347015,
2342
+ "kl": 4.1655029296875,
2343
+ "learning_rate": 0.00010354113518184303,
2344
+ "loss": 0.4661,
2345
+ "reward": 0.784375,
2346
+ "reward_std": 0.2908942475914955,
2347
+ "rewards/accuracy_reward": 0.028125,
2348
+ "rewards/format_reward": 0.75625,
2349
+ "step": 1800
2350
+ },
2351
+ {
2352
+ "completion_length": 57.765625,
2353
+ "epoch": 1.9301333333333335,
2354
+ "grad_norm": 0.07659115642309189,
2355
+ "kl": 1.541015625,
2356
+ "learning_rate": 0.000101773052514972,
2357
+ "loss": 0.1994,
2358
+ "reward": 0.865625,
2359
+ "reward_std": 0.17630237936973572,
2360
+ "rewards/accuracy_reward": 0.025,
2361
+ "rewards/format_reward": 0.840625,
2362
+ "step": 1810
2363
+ },
2364
+ {
2365
+ "completion_length": 58.328125,
2366
+ "epoch": 1.9407999999999999,
2367
+ "grad_norm": 0.053584493696689606,
2368
+ "kl": 1.616796875,
2369
+ "learning_rate": 0.00010001241179001836,
2370
+ "loss": 0.2072,
2371
+ "reward": 0.88125,
2372
+ "reward_std": 0.15386751294136047,
2373
+ "rewards/accuracy_reward": 0.0125,
2374
+ "rewards/format_reward": 0.86875,
2375
+ "step": 1820
2376
+ },
2377
+ {
2378
+ "completion_length": 59.36875,
2379
+ "epoch": 1.9514666666666667,
2380
+ "grad_norm": 0.06573835760354996,
2381
+ "kl": 1.567236328125,
2382
+ "learning_rate": 9.825948469297301e-05,
2383
+ "loss": 0.2048,
2384
+ "reward": 0.921875,
2385
+ "reward_std": 0.19004059880971907,
2386
+ "rewards/accuracy_reward": 0.053125,
2387
+ "rewards/format_reward": 0.86875,
2388
+ "step": 1830
2389
+ },
2390
+ {
2391
+ "completion_length": 55.890625,
2392
+ "epoch": 1.9621333333333333,
2393
+ "grad_norm": 0.18969739973545074,
2394
+ "kl": 1.7246337890625,
2395
+ "learning_rate": 9.651454171953012e-05,
2396
+ "loss": 0.2224,
2397
+ "reward": 0.946875,
2398
+ "reward_std": 0.1664562076330185,
2399
+ "rewards/accuracy_reward": 0.040625,
2400
+ "rewards/format_reward": 0.90625,
2401
+ "step": 1840
2402
+ },
2403
+ {
2404
+ "completion_length": 60.59375,
2405
+ "epoch": 1.9727999999999999,
2406
+ "grad_norm": 0.04010459780693054,
2407
+ "kl": 2.70771484375,
2408
+ "learning_rate": 9.477785213334706e-05,
2409
+ "loss": 0.3228,
2410
+ "reward": 0.834375,
2411
+ "reward_std": 0.22910557091236114,
2412
+ "rewards/accuracy_reward": 0.025,
2413
+ "rewards/format_reward": 0.809375,
2414
+ "step": 1850
2415
+ },
2416
+ {
2417
+ "completion_length": 63.846875,
2418
+ "epoch": 1.9834666666666667,
2419
+ "grad_norm": 0.2044885903596878,
2420
+ "kl": 3.81484375,
2421
+ "learning_rate": 9.30496839244936e-05,
2422
+ "loss": 0.4808,
2423
+ "reward": 0.753125,
2424
+ "reward_std": 0.307637582719326,
2425
+ "rewards/accuracy_reward": 0.028125,
2426
+ "rewards/format_reward": 0.725,
2427
+ "step": 1860
2428
+ },
2429
+ {
2430
+ "completion_length": 63.05,
2431
+ "epoch": 1.9941333333333333,
2432
+ "grad_norm": 0.32608747482299805,
2433
+ "kl": 2.995166015625,
2434
+ "learning_rate": 9.133030376809867e-05,
2435
+ "loss": 0.4066,
2436
+ "reward": 0.734375,
2437
+ "reward_std": 0.31027562469244,
2438
+ "rewards/accuracy_reward": 0.009375,
2439
+ "rewards/format_reward": 0.725,
2440
+ "step": 1870
2441
+ }
2442
+ ],
2443
+ "logging_steps": 10,
2444
+ "max_steps": 2811,
2445
+ "num_input_tokens_seen": 0,
2446
+ "num_train_epochs": 3,
2447
+ "save_steps": 500,
2448
+ "stateful_callbacks": {
2449
+ "TrainerControl": {
2450
+ "args": {
2451
+ "should_epoch_stop": false,
2452
+ "should_evaluate": false,
2453
+ "should_log": false,
2454
+ "should_save": true,
2455
+ "should_training_stop": false
2456
+ },
2457
+ "attributes": {}
2458
+ }
2459
+ },
2460
+ "total_flos": 0.0,
2461
+ "train_batch_size": 8,
2462
+ "trial_name": null,
2463
+ "trial_params": null
2464
+ }
checkpoint-1876/training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7e596a9f480193cbd2a4bf4d6fda1f9ed131e1dbe00394449bfdaab1880a0f79
3
+ size 7544
checkpoint-1876/vocab.json ADDED
The diff for this file is too large to render. See raw diff
 
checkpoint-1876/zero_to_fp32.py ADDED
@@ -0,0 +1,674 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env python
2
+
3
+ # Copyright (c) Microsoft Corporation.
4
+ # SPDX-License-Identifier: Apache-2.0
5
+
6
+ # DeepSpeed Team
7
+
8
+ # This script extracts fp32 consolidated weights from a zero 1, 2 and 3 DeepSpeed checkpoints. It gets
9
+ # copied into the top level checkpoint dir, so the user can easily do the conversion at any point in
10
+ # the future. Once extracted, the weights don't require DeepSpeed and can be used in any
11
+ # application.
12
+ #
13
+ # example:
14
+ # python zero_to_fp32.py . output_dir/
15
+ # or
16
+ # python zero_to_fp32.py . output_dir/ --safe_serialization
17
+
18
+ import argparse
19
+ import torch
20
+ import glob
21
+ import math
22
+ import os
23
+ import re
24
+ import json
25
+ from tqdm import tqdm
26
+ from collections import OrderedDict
27
+ from dataclasses import dataclass
28
+
29
+ # while this script doesn't use deepspeed to recover data, since the checkpoints are pickled with
30
+ # DeepSpeed data structures it has to be available in the current python environment.
31
+ from deepspeed.utils import logger
32
+ from deepspeed.checkpoint.constants import (DS_VERSION, OPTIMIZER_STATE_DICT, SINGLE_PARTITION_OF_FP32_GROUPS,
33
+ FP32_FLAT_GROUPS, ZERO_STAGE, PARTITION_COUNT, PARAM_SHAPES, BUFFER_NAMES,
34
+ FROZEN_PARAM_SHAPES, FROZEN_PARAM_FRAGMENTS)
35
+
36
+
37
+ @dataclass
38
+ class zero_model_state:
39
+ buffers: dict()
40
+ param_shapes: dict()
41
+ shared_params: list
42
+ ds_version: int
43
+ frozen_param_shapes: dict()
44
+ frozen_param_fragments: dict()
45
+
46
+
47
+ debug = 0
48
+
49
+ # load to cpu
50
+ device = torch.device('cpu')
51
+
52
+
53
+ def atoi(text):
54
+ return int(text) if text.isdigit() else text
55
+
56
+
57
+ def natural_keys(text):
58
+ '''
59
+ alist.sort(key=natural_keys) sorts in human order
60
+ http://nedbatchelder.com/blog/200712/human_sorting.html
61
+ (See Toothy's implementation in the comments)
62
+ '''
63
+ return [atoi(c) for c in re.split(r'(\d+)', text)]
64
+
65
+
66
+ def get_model_state_file(checkpoint_dir, zero_stage):
67
+ if not os.path.isdir(checkpoint_dir):
68
+ raise FileNotFoundError(f"Directory '{checkpoint_dir}' doesn't exist")
69
+
70
+ # there should be only one file
71
+ if zero_stage <= 2:
72
+ file = os.path.join(checkpoint_dir, "mp_rank_00_model_states.pt")
73
+ elif zero_stage == 3:
74
+ file = os.path.join(checkpoint_dir, "zero_pp_rank_0_mp_rank_00_model_states.pt")
75
+
76
+ if not os.path.exists(file):
77
+ raise FileNotFoundError(f"can't find model states file at '{file}'")
78
+
79
+ return file
80
+
81
+
82
+ def get_checkpoint_files(checkpoint_dir, glob_pattern):
83
+ # XXX: need to test that this simple glob rule works for multi-node setup too
84
+ ckpt_files = sorted(glob.glob(os.path.join(checkpoint_dir, glob_pattern)), key=natural_keys)
85
+
86
+ if len(ckpt_files) == 0:
87
+ raise FileNotFoundError(f"can't find {glob_pattern} files in directory '{checkpoint_dir}'")
88
+
89
+ return ckpt_files
90
+
91
+
92
+ def get_optim_files(checkpoint_dir):
93
+ return get_checkpoint_files(checkpoint_dir, "*_optim_states.pt")
94
+
95
+
96
+ def get_model_state_files(checkpoint_dir):
97
+ return get_checkpoint_files(checkpoint_dir, "*_model_states.pt")
98
+
99
+
100
+ def parse_model_states(files):
101
+ zero_model_states = []
102
+ for file in files:
103
+ state_dict = torch.load(file, map_location=device)
104
+
105
+ if BUFFER_NAMES not in state_dict:
106
+ raise ValueError(f"{file} is not a model state checkpoint")
107
+ buffer_names = state_dict[BUFFER_NAMES]
108
+ if debug:
109
+ print("Found buffers:", buffer_names)
110
+
111
+ # recover just the buffers while restoring them to fp32 if they were saved in fp16
112
+ buffers = {k: v.float() for k, v in state_dict["module"].items() if k in buffer_names}
113
+ param_shapes = state_dict[PARAM_SHAPES]
114
+
115
+ # collect parameters that are included in param_shapes
116
+ param_names = []
117
+ for s in param_shapes:
118
+ for name in s.keys():
119
+ param_names.append(name)
120
+
121
+ # update with frozen parameters
122
+ frozen_param_shapes = state_dict.get(FROZEN_PARAM_SHAPES, None)
123
+ if frozen_param_shapes is not None:
124
+ if debug:
125
+ print(f"Found frozen_param_shapes: {frozen_param_shapes}")
126
+ param_names += list(frozen_param_shapes.keys())
127
+
128
+ # handle shared params
129
+ shared_params = [[k, v] for k, v in state_dict["shared_params"].items()]
130
+
131
+ ds_version = state_dict.get(DS_VERSION, None)
132
+
133
+ frozen_param_fragments = state_dict.get(FROZEN_PARAM_FRAGMENTS, None)
134
+
135
+ z_model_state = zero_model_state(buffers=buffers,
136
+ param_shapes=param_shapes,
137
+ shared_params=shared_params,
138
+ ds_version=ds_version,
139
+ frozen_param_shapes=frozen_param_shapes,
140
+ frozen_param_fragments=frozen_param_fragments)
141
+ zero_model_states.append(z_model_state)
142
+
143
+ return zero_model_states
144
+
145
+
146
+ def parse_optim_states(files, ds_checkpoint_dir):
147
+ total_files = len(files)
148
+ state_dicts = []
149
+ for f in files:
150
+ state_dict = torch.load(f, map_location=device)
151
+ # immediately discard the potentially huge 2 optimizer states as we only care for fp32 master weights
152
+ # and also handle the case where it was already removed by another helper script
153
+ state_dict["optimizer_state_dict"].pop("optimizer_state_dict", None)
154
+ state_dicts.append(state_dict)
155
+
156
+ if not ZERO_STAGE in state_dicts[0][OPTIMIZER_STATE_DICT]:
157
+ raise ValueError(f"{files[0]} is not a zero checkpoint")
158
+ zero_stage = state_dicts[0][OPTIMIZER_STATE_DICT][ZERO_STAGE]
159
+ world_size = state_dicts[0][OPTIMIZER_STATE_DICT][PARTITION_COUNT]
160
+
161
+ # For ZeRO-2 each param group can have different partition_count as data parallelism for expert
162
+ # parameters can be different from data parallelism for non-expert parameters. So we can just
163
+ # use the max of the partition_count to get the dp world_size.
164
+
165
+ if type(world_size) is list:
166
+ world_size = max(world_size)
167
+
168
+ if world_size != total_files:
169
+ raise ValueError(
170
+ f"Expected {world_size} of '*_optim_states.pt' under '{ds_checkpoint_dir}' but found {total_files} files. "
171
+ "Possibly due to an overwrite of an old checkpoint, or a checkpoint didn't get saved by one or more processes."
172
+ )
173
+
174
+ # the groups are named differently in each stage
175
+ if zero_stage <= 2:
176
+ fp32_groups_key = SINGLE_PARTITION_OF_FP32_GROUPS
177
+ elif zero_stage == 3:
178
+ fp32_groups_key = FP32_FLAT_GROUPS
179
+ else:
180
+ raise ValueError(f"unknown zero stage {zero_stage}")
181
+
182
+ if zero_stage <= 2:
183
+ fp32_flat_groups = [state_dicts[i][OPTIMIZER_STATE_DICT][fp32_groups_key] for i in range(len(state_dicts))]
184
+ elif zero_stage == 3:
185
+ # if there is more than one param group, there will be multiple flattened tensors - one
186
+ # flattened tensor per group - for simplicity merge them into a single tensor
187
+ #
188
+ # XXX: could make the script more memory efficient for when there are multiple groups - it
189
+ # will require matching the sub-lists of param_shapes for each param group flattened tensor
190
+
191
+ fp32_flat_groups = [
192
+ torch.cat(state_dicts[i][OPTIMIZER_STATE_DICT][fp32_groups_key], 0) for i in range(len(state_dicts))
193
+ ]
194
+
195
+ return zero_stage, world_size, fp32_flat_groups
196
+
197
+
198
+ def _get_fp32_state_dict_from_zero_checkpoint(ds_checkpoint_dir, exclude_frozen_parameters):
199
+ """
200
+ Returns fp32 state_dict reconstructed from ds checkpoint
201
+
202
+ Args:
203
+ - ``ds_checkpoint_dir``: path to the deepspeed checkpoint folder (where the optimizer files are)
204
+
205
+ """
206
+ print(f"Processing zero checkpoint '{ds_checkpoint_dir}'")
207
+
208
+ optim_files = get_optim_files(ds_checkpoint_dir)
209
+ zero_stage, world_size, fp32_flat_groups = parse_optim_states(optim_files, ds_checkpoint_dir)
210
+ print(f"Detected checkpoint of type zero stage {zero_stage}, world_size: {world_size}")
211
+
212
+ model_files = get_model_state_files(ds_checkpoint_dir)
213
+
214
+ zero_model_states = parse_model_states(model_files)
215
+ print(f'Parsing checkpoint created by deepspeed=={zero_model_states[0].ds_version}')
216
+
217
+ if zero_stage <= 2:
218
+ return _get_fp32_state_dict_from_zero2_checkpoint(world_size, fp32_flat_groups, zero_model_states,
219
+ exclude_frozen_parameters)
220
+ elif zero_stage == 3:
221
+ return _get_fp32_state_dict_from_zero3_checkpoint(world_size, fp32_flat_groups, zero_model_states,
222
+ exclude_frozen_parameters)
223
+
224
+
225
+ def _zero2_merge_frozen_params(state_dict, zero_model_states):
226
+ if zero_model_states[0].frozen_param_shapes is None or len(zero_model_states[0].frozen_param_shapes) == 0:
227
+ return
228
+
229
+ frozen_param_shapes = zero_model_states[0].frozen_param_shapes
230
+ frozen_param_fragments = zero_model_states[0].frozen_param_fragments
231
+
232
+ if debug:
233
+ num_elem = sum(s.numel() for s in frozen_param_shapes.values())
234
+ print(f'rank 0: {FROZEN_PARAM_SHAPES}.numel = {num_elem}')
235
+
236
+ wanted_params = len(frozen_param_shapes)
237
+ wanted_numel = sum(s.numel() for s in frozen_param_shapes.values())
238
+ avail_numel = sum([p.numel() for p in frozen_param_fragments.values()])
239
+ print(f'Frozen params: Have {avail_numel} numels to process.')
240
+ print(f'Frozen params: Need {wanted_numel} numels in {wanted_params} params')
241
+
242
+ total_params = 0
243
+ total_numel = 0
244
+ for name, shape in frozen_param_shapes.items():
245
+ total_params += 1
246
+ unpartitioned_numel = shape.numel()
247
+ total_numel += unpartitioned_numel
248
+
249
+ state_dict[name] = frozen_param_fragments[name]
250
+
251
+ if debug:
252
+ print(f"{name} full shape: {shape} unpartitioned numel {unpartitioned_numel} ")
253
+
254
+ print(f"Reconstructed Frozen fp32 state dict with {total_params} params {total_numel} elements")
255
+
256
+
257
+ def _has_callable(obj, fn):
258
+ attr = getattr(obj, fn, None)
259
+ return callable(attr)
260
+
261
+
262
+ def _zero2_merge_trainable_params(state_dict, world_size, fp32_flat_groups, zero_model_states):
263
+ param_shapes = zero_model_states[0].param_shapes
264
+
265
+ # Reconstruction protocol:
266
+ #
267
+ # XXX: document this
268
+
269
+ if debug:
270
+ for i in range(world_size):
271
+ for j in range(len(fp32_flat_groups[0])):
272
+ print(f"{FP32_FLAT_GROUPS}[{i}][{j}].shape={fp32_flat_groups[i][j].shape}")
273
+
274
+ # XXX: memory usage doubles here (zero2)
275
+ num_param_groups = len(fp32_flat_groups[0])
276
+ merged_single_partition_of_fp32_groups = []
277
+ for i in range(num_param_groups):
278
+ merged_partitions = [sd[i] for sd in fp32_flat_groups]
279
+ full_single_fp32_vector = torch.cat(merged_partitions, 0)
280
+ merged_single_partition_of_fp32_groups.append(full_single_fp32_vector)
281
+ avail_numel = sum(
282
+ [full_single_fp32_vector.numel() for full_single_fp32_vector in merged_single_partition_of_fp32_groups])
283
+
284
+ if debug:
285
+ wanted_params = sum([len(shapes) for shapes in param_shapes])
286
+ wanted_numel = sum([sum(shape.numel() for shape in shapes.values()) for shapes in param_shapes])
287
+ # not asserting if there is a mismatch due to possible padding
288
+ print(f"Have {avail_numel} numels to process.")
289
+ print(f"Need {wanted_numel} numels in {wanted_params} params.")
290
+
291
+ # params
292
+ # XXX: for huge models that can't fit into the host's RAM we will have to recode this to support
293
+ # out-of-core computing solution
294
+ total_numel = 0
295
+ total_params = 0
296
+ for shapes, full_single_fp32_vector in zip(param_shapes, merged_single_partition_of_fp32_groups):
297
+ offset = 0
298
+ avail_numel = full_single_fp32_vector.numel()
299
+ for name, shape in shapes.items():
300
+
301
+ unpartitioned_numel = shape.numel() if _has_callable(shape, 'numel') else math.prod(shape)
302
+ total_numel += unpartitioned_numel
303
+ total_params += 1
304
+
305
+ if debug:
306
+ print(f"{name} full shape: {shape} unpartitioned numel {unpartitioned_numel} ")
307
+ state_dict[name] = full_single_fp32_vector.narrow(0, offset, unpartitioned_numel).view(shape)
308
+ offset += unpartitioned_numel
309
+
310
+ # Z2 started to align to 2*world_size to improve nccl performance. Therefore both offset and
311
+ # avail_numel can differ by anywhere between 0..2*world_size. Due to two unrelated complex
312
+ # paddings performed in the code it's almost impossible to predict the exact numbers w/o the
313
+ # live optimizer object, so we are checking that the numbers are within the right range
314
+ align_to = 2 * world_size
315
+
316
+ def zero2_align(x):
317
+ return align_to * math.ceil(x / align_to)
318
+
319
+ if debug:
320
+ print(f"original offset={offset}, avail_numel={avail_numel}")
321
+
322
+ offset = zero2_align(offset)
323
+ avail_numel = zero2_align(avail_numel)
324
+
325
+ if debug:
326
+ print(f"aligned offset={offset}, avail_numel={avail_numel}")
327
+
328
+ # Sanity check
329
+ if offset != avail_numel:
330
+ raise ValueError(f"consumed {offset} numels out of {avail_numel} - something is wrong")
331
+
332
+ print(f"Reconstructed fp32 state dict with {total_params} params {total_numel} elements")
333
+
334
+
335
+ def _get_fp32_state_dict_from_zero2_checkpoint(world_size, fp32_flat_groups, zero_model_states,
336
+ exclude_frozen_parameters):
337
+ state_dict = OrderedDict()
338
+
339
+ # buffers
340
+ buffers = zero_model_states[0].buffers
341
+ state_dict.update(buffers)
342
+ if debug:
343
+ print(f"added {len(buffers)} buffers")
344
+
345
+ if not exclude_frozen_parameters:
346
+ _zero2_merge_frozen_params(state_dict, zero_model_states)
347
+
348
+ _zero2_merge_trainable_params(state_dict, world_size, fp32_flat_groups, zero_model_states)
349
+
350
+ # recover shared parameters
351
+ for pair in zero_model_states[0].shared_params:
352
+ if pair[1] in state_dict:
353
+ state_dict[pair[0]] = state_dict[pair[1]]
354
+
355
+ return state_dict
356
+
357
+
358
+ def zero3_partitioned_param_info(unpartitioned_numel, world_size):
359
+ remainder = unpartitioned_numel % world_size
360
+ padding_numel = (world_size - remainder) if remainder else 0
361
+ partitioned_numel = math.ceil(unpartitioned_numel / world_size)
362
+ return partitioned_numel, padding_numel
363
+
364
+
365
+ def _zero3_merge_frozen_params(state_dict, world_size, zero_model_states):
366
+ if zero_model_states[0].frozen_param_shapes is None or len(zero_model_states[0].frozen_param_shapes) == 0:
367
+ return
368
+
369
+ if debug:
370
+ for i in range(world_size):
371
+ num_elem = sum(s.numel() for s in zero_model_states[i].frozen_param_fragments.values())
372
+ print(f'rank {i}: {FROZEN_PARAM_SHAPES}.numel = {num_elem}')
373
+
374
+ frozen_param_shapes = zero_model_states[0].frozen_param_shapes
375
+ wanted_params = len(frozen_param_shapes)
376
+ wanted_numel = sum(s.numel() for s in frozen_param_shapes.values())
377
+ avail_numel = sum([p.numel() for p in zero_model_states[0].frozen_param_fragments.values()]) * world_size
378
+ print(f'Frozen params: Have {avail_numel} numels to process.')
379
+ print(f'Frozen params: Need {wanted_numel} numels in {wanted_params} params')
380
+
381
+ total_params = 0
382
+ total_numel = 0
383
+ for name, shape in zero_model_states[0].frozen_param_shapes.items():
384
+ total_params += 1
385
+ unpartitioned_numel = shape.numel()
386
+ total_numel += unpartitioned_numel
387
+
388
+ param_frags = tuple(model_state.frozen_param_fragments[name] for model_state in zero_model_states)
389
+ state_dict[name] = torch.cat(param_frags, 0).narrow(0, 0, unpartitioned_numel).view(shape)
390
+
391
+ partitioned_numel, partitioned_padding_numel = zero3_partitioned_param_info(unpartitioned_numel, world_size)
392
+
393
+ if debug:
394
+ print(
395
+ f"Frozen params: {total_params} {name} full shape: {shape} partition0 numel={partitioned_numel} partitioned_padding_numel={partitioned_padding_numel}"
396
+ )
397
+
398
+ print(f"Reconstructed Frozen fp32 state dict with {total_params} params {total_numel} elements")
399
+
400
+
401
+ def _zero3_merge_trainable_params(state_dict, world_size, fp32_flat_groups, zero_model_states):
402
+ param_shapes = zero_model_states[0].param_shapes
403
+ avail_numel = fp32_flat_groups[0].numel() * world_size
404
+ # Reconstruction protocol: For zero3 we need to zip the partitions together at boundary of each
405
+ # param, re-consolidating each param, while dealing with padding if any
406
+
407
+ # merge list of dicts, preserving order
408
+ param_shapes = {k: v for d in param_shapes for k, v in d.items()}
409
+
410
+ if debug:
411
+ for i in range(world_size):
412
+ print(f"{FP32_FLAT_GROUPS}[{i}].shape={fp32_flat_groups[i].shape}")
413
+
414
+ wanted_params = len(param_shapes)
415
+ wanted_numel = sum(shape.numel() for shape in param_shapes.values())
416
+ # not asserting if there is a mismatch due to possible padding
417
+ avail_numel = fp32_flat_groups[0].numel() * world_size
418
+ print(f"Trainable params: Have {avail_numel} numels to process.")
419
+ print(f"Trainable params: Need {wanted_numel} numels in {wanted_params} params.")
420
+
421
+ # params
422
+ # XXX: for huge models that can't fit into the host's RAM we will have to recode this to support
423
+ # out-of-core computing solution
424
+ offset = 0
425
+ total_numel = 0
426
+ total_params = 0
427
+ for name, shape in tqdm(param_shapes.items(), desc='Gathering Sharded Weights'):
428
+ unpartitioned_numel = shape.numel()
429
+ total_numel += unpartitioned_numel
430
+ total_params += 1
431
+ partitioned_numel, partitioned_padding_numel = zero3_partitioned_param_info(unpartitioned_numel, world_size)
432
+
433
+ if debug:
434
+ print(
435
+ f"Trainable params: {total_params} {name} full shape: {shape} partition0 numel={partitioned_numel} partitioned_padding_numel={partitioned_padding_numel}"
436
+ )
437
+
438
+ # XXX: memory usage doubles here
439
+ state_dict[name] = torch.cat(
440
+ tuple(fp32_flat_groups[i].narrow(0, offset, partitioned_numel) for i in range(world_size)),
441
+ 0).narrow(0, 0, unpartitioned_numel).view(shape)
442
+ offset += partitioned_numel
443
+
444
+ offset *= world_size
445
+
446
+ # Sanity check
447
+ if offset != avail_numel:
448
+ raise ValueError(f"consumed {offset} numels out of {avail_numel} - something is wrong")
449
+
450
+ print(f"Reconstructed Trainable fp32 state dict with {total_params} params {total_numel} elements")
451
+
452
+
453
+ def _get_fp32_state_dict_from_zero3_checkpoint(world_size, fp32_flat_groups, zero_model_states,
454
+ exclude_frozen_parameters):
455
+ state_dict = OrderedDict()
456
+
457
+ # buffers
458
+ buffers = zero_model_states[0].buffers
459
+ state_dict.update(buffers)
460
+ if debug:
461
+ print(f"added {len(buffers)} buffers")
462
+
463
+ if not exclude_frozen_parameters:
464
+ _zero3_merge_frozen_params(state_dict, world_size, zero_model_states)
465
+
466
+ _zero3_merge_trainable_params(state_dict, world_size, fp32_flat_groups, zero_model_states)
467
+
468
+ # recover shared parameters
469
+ for pair in zero_model_states[0].shared_params:
470
+ if pair[1] in state_dict:
471
+ state_dict[pair[0]] = state_dict[pair[1]]
472
+
473
+ return state_dict
474
+
475
+
476
+ def get_fp32_state_dict_from_zero_checkpoint(checkpoint_dir, tag=None, exclude_frozen_parameters=False):
477
+ """
478
+ Convert ZeRO 2 or 3 checkpoint into a single fp32 consolidated state_dict that can be loaded with
479
+ ``load_state_dict()`` and used for training without DeepSpeed or shared with others, for example
480
+ via a model hub.
481
+
482
+ Args:
483
+ - ``checkpoint_dir``: path to the desired checkpoint folder
484
+ - ``tag``: checkpoint tag used as a unique identifier for checkpoint. If not provided will attempt to load tag in 'latest' file. e.g., ``global_step14``
485
+ - ``exclude_frozen_parameters``: exclude frozen parameters
486
+
487
+ Returns:
488
+ - pytorch ``state_dict``
489
+
490
+ Note: this approach may not work if your application doesn't have sufficient free CPU memory and
491
+ you may need to use the offline approach using the ``zero_to_fp32.py`` script that is saved with
492
+ the checkpoint.
493
+
494
+ A typical usage might be ::
495
+
496
+ from deepspeed.utils.zero_to_fp32 import get_fp32_state_dict_from_zero_checkpoint
497
+ # do the training and checkpoint saving
498
+ state_dict = get_fp32_state_dict_from_zero_checkpoint(checkpoint_dir) # already on cpu
499
+ model = model.cpu() # move to cpu
500
+ model.load_state_dict(state_dict)
501
+ # submit to model hub or save the model to share with others
502
+
503
+ In this example the ``model`` will no longer be usable in the deepspeed context of the same
504
+ application. i.e. you will need to re-initialize the deepspeed engine, since
505
+ ``model.load_state_dict(state_dict)`` will remove all the deepspeed magic from it.
506
+
507
+ If you want it all done for you, use ``load_state_dict_from_zero_checkpoint`` instead.
508
+
509
+ """
510
+ if tag is None:
511
+ latest_path = os.path.join(checkpoint_dir, 'latest')
512
+ if os.path.isfile(latest_path):
513
+ with open(latest_path, 'r') as fd:
514
+ tag = fd.read().strip()
515
+ else:
516
+ raise ValueError(f"Unable to find 'latest' file at {latest_path}")
517
+
518
+ ds_checkpoint_dir = os.path.join(checkpoint_dir, tag)
519
+
520
+ if not os.path.isdir(ds_checkpoint_dir):
521
+ raise FileNotFoundError(f"Directory '{ds_checkpoint_dir}' doesn't exist")
522
+
523
+ return _get_fp32_state_dict_from_zero_checkpoint(ds_checkpoint_dir, exclude_frozen_parameters)
524
+
525
+
526
+ def convert_zero_checkpoint_to_fp32_state_dict(checkpoint_dir,
527
+ output_dir,
528
+ max_shard_size="5GB",
529
+ safe_serialization=False,
530
+ tag=None,
531
+ exclude_frozen_parameters=False):
532
+ """
533
+ Convert ZeRO 2 or 3 checkpoint into a single fp32 consolidated ``state_dict`` file that can be
534
+ loaded with ``torch.load(file)`` + ``load_state_dict()`` and used for training without DeepSpeed.
535
+
536
+ Args:
537
+ - ``checkpoint_dir``: path to the desired checkpoint folder. (one that contains the tag-folder, like ``global_step14``)
538
+ - ``output_dir``: directory to the pytorch fp32 state_dict output files
539
+ - ``max_shard_size``: the maximum size for a checkpoint before being sharded, default value is 5GB
540
+ - ``safe_serialization``: whether to save the model using `safetensors` or the traditional PyTorch way (that uses `pickle`).
541
+ - ``tag``: checkpoint tag used as a unique identifier for checkpoint. If not provided will attempt to load tag in the file named ``latest`` in the checkpoint folder, e.g., ``global_step14``
542
+ - ``exclude_frozen_parameters``: exclude frozen parameters
543
+ """
544
+ # Dependency pre-check
545
+ if safe_serialization:
546
+ try:
547
+ from safetensors.torch import save_file
548
+ except ImportError:
549
+ print('If you want to use `safe_serialization`, please `pip install safetensors`')
550
+ raise
551
+ if max_shard_size is not None:
552
+ try:
553
+ from huggingface_hub import split_torch_state_dict_into_shards
554
+ except ImportError:
555
+ print('If you want to use `max_shard_size`, please `pip install huggingface_hub`')
556
+ raise
557
+
558
+ # Convert zero checkpoint to state_dict
559
+ state_dict = get_fp32_state_dict_from_zero_checkpoint(checkpoint_dir, tag, exclude_frozen_parameters)
560
+
561
+ # Shard the model if it is too big.
562
+ weights_name = "model.safetensors" if safe_serialization else "pytorch_model.bin"
563
+ if max_shard_size is not None:
564
+ filename_pattern = weights_name.replace(".bin", "{suffix}.bin").replace(".safetensors", "{suffix}.safetensors")
565
+ state_dict_split = split_torch_state_dict_into_shards(state_dict,
566
+ filename_pattern=filename_pattern,
567
+ max_shard_size=max_shard_size)
568
+ else:
569
+ from collections import namedtuple
570
+ StateDictSplit = namedtuple("StateDictSplit", ["is_sharded", "filename_to_tensors"])
571
+ state_dict_split = StateDictSplit(is_sharded=False,
572
+ filename_to_tensors={weights_name: list(state_dict.keys())})
573
+
574
+ # Save the model
575
+ filename_to_tensors = state_dict_split.filename_to_tensors.items()
576
+ for shard_file, tensors in tqdm(filename_to_tensors, desc="Saving checkpoint shards"):
577
+ shard = {tensor: state_dict[tensor].contiguous() for tensor in tensors}
578
+ output_path = os.path.join(output_dir, shard_file)
579
+ if safe_serialization:
580
+ save_file(shard, output_path, metadata={"format": "pt"})
581
+ else:
582
+ torch.save(shard, output_path)
583
+
584
+ # Save index if sharded
585
+ if state_dict_split.is_sharded:
586
+ index = {
587
+ "metadata": state_dict_split.metadata,
588
+ "weight_map": state_dict_split.tensor_to_filename,
589
+ }
590
+ save_index_file = "model.safetensors.index.json" if safe_serialization else "pytorch_model.bin.index.json"
591
+ save_index_file = os.path.join(output_dir, save_index_file)
592
+ with open(save_index_file, "w", encoding="utf-8") as f:
593
+ content = json.dumps(index, indent=2, sort_keys=True) + "\n"
594
+ f.write(content)
595
+
596
+
597
+ def load_state_dict_from_zero_checkpoint(model, checkpoint_dir, tag=None):
598
+ """
599
+ 1. Put the provided model to cpu
600
+ 2. Convert ZeRO 2 or 3 checkpoint into a single fp32 consolidated ``state_dict``
601
+ 3. Load it into the provided model
602
+
603
+ Args:
604
+ - ``model``: the model object to update
605
+ - ``checkpoint_dir``: path to the desired checkpoint folder. (one that contains the tag-folder, like ``global_step14``)
606
+ - ``tag``: checkpoint tag used as a unique identifier for checkpoint. If not provided will attempt to load tag in the file named ``latest`` in the checkpoint folder, e.g., ``global_step14``
607
+
608
+ Returns:
609
+ - ``model`: modified model
610
+
611
+ Make sure you have plenty of CPU memory available before you call this function. If you don't
612
+ have enough use the ``zero_to_fp32.py`` utility to do the conversion. You will find it
613
+ conveniently placed for you in the checkpoint folder.
614
+
615
+ A typical usage might be ::
616
+
617
+ from deepspeed.utils.zero_to_fp32 import load_state_dict_from_zero_checkpoint
618
+ model = load_state_dict_from_zero_checkpoint(trainer.model, checkpoint_dir)
619
+ # submit to model hub or save the model to share with others
620
+
621
+ Note, that once this was run, the ``model`` will no longer be usable in the deepspeed context
622
+ of the same application. i.e. you will need to re-initialize the deepspeed engine, since
623
+ ``model.load_state_dict(state_dict)`` will remove all the deepspeed magic from it.
624
+
625
+ """
626
+ logger.info(f"Extracting fp32 weights")
627
+ state_dict = get_fp32_state_dict_from_zero_checkpoint(checkpoint_dir, tag)
628
+
629
+ logger.info(f"Overwriting model with fp32 weights")
630
+ model = model.cpu()
631
+ model.load_state_dict(state_dict, strict=False)
632
+
633
+ return model
634
+
635
+
636
+ if __name__ == "__main__":
637
+ parser = argparse.ArgumentParser()
638
+ parser.add_argument("checkpoint_dir",
639
+ type=str,
640
+ help="path to the desired checkpoint folder, e.g., path/checkpoint-12")
641
+ parser.add_argument("output_dir",
642
+ type=str,
643
+ help="directory to the pytorch fp32 state_dict output files"
644
+ "(e.g. path/checkpoint-12-output/)")
645
+ parser.add_argument(
646
+ "--max_shard_size",
647
+ type=str,
648
+ default="5GB",
649
+ help="The maximum size for a checkpoint before being sharded. Checkpoints shard will then be each of size"
650
+ "lower than this size. If expressed as a string, needs to be digits followed by a unit (like `5MB`"
651
+ "We default it to 5GB in order for models to be able to run easily on free-tier google colab instances"
652
+ "without CPU OOM issues.")
653
+ parser.add_argument(
654
+ "--safe_serialization",
655
+ default=False,
656
+ action='store_true',
657
+ help="Whether to save the model using `safetensors` or the traditional PyTorch way (that uses `pickle`).")
658
+ parser.add_argument("-t",
659
+ "--tag",
660
+ type=str,
661
+ default=None,
662
+ help="checkpoint tag used as a unique identifier for checkpoint. e.g., global_step1")
663
+ parser.add_argument("--exclude_frozen_parameters", action='store_true', help="exclude frozen parameters")
664
+ parser.add_argument("-d", "--debug", action='store_true', help="enable debug")
665
+ args = parser.parse_args()
666
+
667
+ debug = args.debug
668
+
669
+ convert_zero_checkpoint_to_fp32_state_dict(args.checkpoint_dir,
670
+ args.output_dir,
671
+ max_shard_size=args.max_shard_size,
672
+ safe_serialization=args.safe_serialization,
673
+ tag=args.tag,
674
+ exclude_frozen_parameters=args.exclude_frozen_parameters)
checkpoint-2811/README.md ADDED
@@ -0,0 +1,202 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: Qwen/Qwen2.5-1.5B
3
+ library_name: peft
4
+ ---
5
+
6
+ # Model Card for Model ID
7
+
8
+ <!-- Provide a quick summary of what the model is/does. -->
9
+
10
+
11
+
12
+ ## Model Details
13
+
14
+ ### Model Description
15
+
16
+ <!-- Provide a longer summary of what this model is. -->
17
+
18
+
19
+
20
+ - **Developed by:** [More Information Needed]
21
+ - **Funded by [optional]:** [More Information Needed]
22
+ - **Shared by [optional]:** [More Information Needed]
23
+ - **Model type:** [More Information Needed]
24
+ - **Language(s) (NLP):** [More Information Needed]
25
+ - **License:** [More Information Needed]
26
+ - **Finetuned from model [optional]:** [More Information Needed]
27
+
28
+ ### Model Sources [optional]
29
+
30
+ <!-- Provide the basic links for the model. -->
31
+
32
+ - **Repository:** [More Information Needed]
33
+ - **Paper [optional]:** [More Information Needed]
34
+ - **Demo [optional]:** [More Information Needed]
35
+
36
+ ## Uses
37
+
38
+ <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
39
+
40
+ ### Direct Use
41
+
42
+ <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
43
+
44
+ [More Information Needed]
45
+
46
+ ### Downstream Use [optional]
47
+
48
+ <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
49
+
50
+ [More Information Needed]
51
+
52
+ ### Out-of-Scope Use
53
+
54
+ <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
55
+
56
+ [More Information Needed]
57
+
58
+ ## Bias, Risks, and Limitations
59
+
60
+ <!-- This section is meant to convey both technical and sociotechnical limitations. -->
61
+
62
+ [More Information Needed]
63
+
64
+ ### Recommendations
65
+
66
+ <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
67
+
68
+ Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
69
+
70
+ ## How to Get Started with the Model
71
+
72
+ Use the code below to get started with the model.
73
+
74
+ [More Information Needed]
75
+
76
+ ## Training Details
77
+
78
+ ### Training Data
79
+
80
+ <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
81
+
82
+ [More Information Needed]
83
+
84
+ ### Training Procedure
85
+
86
+ <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
87
+
88
+ #### Preprocessing [optional]
89
+
90
+ [More Information Needed]
91
+
92
+
93
+ #### Training Hyperparameters
94
+
95
+ - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
96
+
97
+ #### Speeds, Sizes, Times [optional]
98
+
99
+ <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
100
+
101
+ [More Information Needed]
102
+
103
+ ## Evaluation
104
+
105
+ <!-- This section describes the evaluation protocols and provides the results. -->
106
+
107
+ ### Testing Data, Factors & Metrics
108
+
109
+ #### Testing Data
110
+
111
+ <!-- This should link to a Dataset Card if possible. -->
112
+
113
+ [More Information Needed]
114
+
115
+ #### Factors
116
+
117
+ <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
118
+
119
+ [More Information Needed]
120
+
121
+ #### Metrics
122
+
123
+ <!-- These are the evaluation metrics being used, ideally with a description of why. -->
124
+
125
+ [More Information Needed]
126
+
127
+ ### Results
128
+
129
+ [More Information Needed]
130
+
131
+ #### Summary
132
+
133
+
134
+
135
+ ## Model Examination [optional]
136
+
137
+ <!-- Relevant interpretability work for the model goes here -->
138
+
139
+ [More Information Needed]
140
+
141
+ ## Environmental Impact
142
+
143
+ <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
144
+
145
+ Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
146
+
147
+ - **Hardware Type:** [More Information Needed]
148
+ - **Hours used:** [More Information Needed]
149
+ - **Cloud Provider:** [More Information Needed]
150
+ - **Compute Region:** [More Information Needed]
151
+ - **Carbon Emitted:** [More Information Needed]
152
+
153
+ ## Technical Specifications [optional]
154
+
155
+ ### Model Architecture and Objective
156
+
157
+ [More Information Needed]
158
+
159
+ ### Compute Infrastructure
160
+
161
+ [More Information Needed]
162
+
163
+ #### Hardware
164
+
165
+ [More Information Needed]
166
+
167
+ #### Software
168
+
169
+ [More Information Needed]
170
+
171
+ ## Citation [optional]
172
+
173
+ <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
174
+
175
+ **BibTeX:**
176
+
177
+ [More Information Needed]
178
+
179
+ **APA:**
180
+
181
+ [More Information Needed]
182
+
183
+ ## Glossary [optional]
184
+
185
+ <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
186
+
187
+ [More Information Needed]
188
+
189
+ ## More Information [optional]
190
+
191
+ [More Information Needed]
192
+
193
+ ## Model Card Authors [optional]
194
+
195
+ [More Information Needed]
196
+
197
+ ## Model Card Contact
198
+
199
+ [More Information Needed]
200
+ ### Framework versions
201
+
202
+ - PEFT 0.14.0
checkpoint-2811/adapter_config.json ADDED
@@ -0,0 +1,34 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alpha_pattern": {},
3
+ "auto_mapping": null,
4
+ "base_model_name_or_path": "Qwen/Qwen2.5-1.5B",
5
+ "bias": "none",
6
+ "eva_config": null,
7
+ "exclude_modules": null,
8
+ "fan_in_fan_out": false,
9
+ "inference_mode": true,
10
+ "init_lora_weights": true,
11
+ "layer_replication": null,
12
+ "layers_pattern": null,
13
+ "layers_to_transform": null,
14
+ "loftq_config": {},
15
+ "lora_alpha": 8,
16
+ "lora_bias": false,
17
+ "lora_dropout": 0.0,
18
+ "megatron_config": null,
19
+ "megatron_core": "megatron.core",
20
+ "modules_to_save": null,
21
+ "peft_type": "LORA",
22
+ "r": 32,
23
+ "rank_pattern": {},
24
+ "revision": null,
25
+ "target_modules": [
26
+ "q_proj",
27
+ "v_proj",
28
+ "embed_tokens",
29
+ "k_proj"
30
+ ],
31
+ "task_type": "CAUSAL_LM",
32
+ "use_dora": false,
33
+ "use_rslora": false
34
+ }
checkpoint-2811/adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d0e2d12773cde23612f66f956756bfff79b5088a590085701d068e152e8b9f0d
3
+ size 488520640
checkpoint-2811/added_tokens.json ADDED
@@ -0,0 +1,24 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "</tool_call>": 151658,
3
+ "<tool_call>": 151657,
4
+ "<|box_end|>": 151649,
5
+ "<|box_start|>": 151648,
6
+ "<|endoftext|>": 151643,
7
+ "<|file_sep|>": 151664,
8
+ "<|fim_middle|>": 151660,
9
+ "<|fim_pad|>": 151662,
10
+ "<|fim_prefix|>": 151659,
11
+ "<|fim_suffix|>": 151661,
12
+ "<|im_end|>": 151645,
13
+ "<|im_start|>": 151644,
14
+ "<|image_pad|>": 151655,
15
+ "<|object_ref_end|>": 151647,
16
+ "<|object_ref_start|>": 151646,
17
+ "<|quad_end|>": 151651,
18
+ "<|quad_start|>": 151650,
19
+ "<|repo_name|>": 151663,
20
+ "<|video_pad|>": 151656,
21
+ "<|vision_end|>": 151653,
22
+ "<|vision_pad|>": 151654,
23
+ "<|vision_start|>": 151652
24
+ }
checkpoint-2811/global_step2810/bf16_zero_pp_rank_0_mp_rank_00_optim_states.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6dd7034c580e6acd4b359e2cea8d943b8921122f1d8c1cc9cb50dc1a4196b681
3
+ size 130520624
checkpoint-2811/global_step2810/mp_rank_00_model_states.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ded2b4ee29d3c0bb6b96ec7a9d4e62d8b04f5fdea19018fb908b35ad044528d1
3
+ size 488645432
checkpoint-2811/latest ADDED
@@ -0,0 +1 @@
 
 
1
+ global_step2810
checkpoint-2811/merges.txt ADDED
The diff for this file is too large to render. See raw diff
 
checkpoint-2811/rng_state.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b6647a496131c714e6f25c6ed38080948283f93dbdaa708df34a03fc09a51826
3
+ size 14244
checkpoint-2811/scheduler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5a702cef1bbbaf4b23cf4944bfd88935ecce43dafe80ceae33d9d07e5c46a0fa
3
+ size 1064
checkpoint-2811/special_tokens_map.json ADDED
@@ -0,0 +1,31 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "additional_special_tokens": [
3
+ "<|im_start|>",
4
+ "<|im_end|>",
5
+ "<|object_ref_start|>",
6
+ "<|object_ref_end|>",
7
+ "<|box_start|>",
8
+ "<|box_end|>",
9
+ "<|quad_start|>",
10
+ "<|quad_end|>",
11
+ "<|vision_start|>",
12
+ "<|vision_end|>",
13
+ "<|vision_pad|>",
14
+ "<|image_pad|>",
15
+ "<|video_pad|>"
16
+ ],
17
+ "eos_token": {
18
+ "content": "<|endoftext|>",
19
+ "lstrip": false,
20
+ "normalized": false,
21
+ "rstrip": false,
22
+ "single_word": false
23
+ },
24
+ "pad_token": {
25
+ "content": "<|endoftext|>",
26
+ "lstrip": false,
27
+ "normalized": false,
28
+ "rstrip": false,
29
+ "single_word": false
30
+ }
31
+ }
checkpoint-2811/tokenizer.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5eee858c5123a4279c3e1f7b81247343f356ac767940b2692a928ad929543214
3
+ size 11422063
checkpoint-2811/tokenizer_config.json ADDED
@@ -0,0 +1,209 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_bos_token": false,
3
+ "add_prefix_space": false,
4
+ "added_tokens_decoder": {
5
+ "151643": {
6
+ "content": "<|endoftext|>",
7
+ "lstrip": false,
8
+ "normalized": false,
9
+ "rstrip": false,
10
+ "single_word": false,
11
+ "special": true
12
+ },
13
+ "151644": {
14
+ "content": "<|im_start|>",
15
+ "lstrip": false,
16
+ "normalized": false,
17
+ "rstrip": false,
18
+ "single_word": false,
19
+ "special": true
20
+ },
21
+ "151645": {
22
+ "content": "<|im_end|>",
23
+ "lstrip": false,
24
+ "normalized": false,
25
+ "rstrip": false,
26
+ "single_word": false,
27
+ "special": true
28
+ },
29
+ "151646": {
30
+ "content": "<|object_ref_start|>",
31
+ "lstrip": false,
32
+ "normalized": false,
33
+ "rstrip": false,
34
+ "single_word": false,
35
+ "special": true
36
+ },
37
+ "151647": {
38
+ "content": "<|object_ref_end|>",
39
+ "lstrip": false,
40
+ "normalized": false,
41
+ "rstrip": false,
42
+ "single_word": false,
43
+ "special": true
44
+ },
45
+ "151648": {
46
+ "content": "<|box_start|>",
47
+ "lstrip": false,
48
+ "normalized": false,
49
+ "rstrip": false,
50
+ "single_word": false,
51
+ "special": true
52
+ },
53
+ "151649": {
54
+ "content": "<|box_end|>",
55
+ "lstrip": false,
56
+ "normalized": false,
57
+ "rstrip": false,
58
+ "single_word": false,
59
+ "special": true
60
+ },
61
+ "151650": {
62
+ "content": "<|quad_start|>",
63
+ "lstrip": false,
64
+ "normalized": false,
65
+ "rstrip": false,
66
+ "single_word": false,
67
+ "special": true
68
+ },
69
+ "151651": {
70
+ "content": "<|quad_end|>",
71
+ "lstrip": false,
72
+ "normalized": false,
73
+ "rstrip": false,
74
+ "single_word": false,
75
+ "special": true
76
+ },
77
+ "151652": {
78
+ "content": "<|vision_start|>",
79
+ "lstrip": false,
80
+ "normalized": false,
81
+ "rstrip": false,
82
+ "single_word": false,
83
+ "special": true
84
+ },
85
+ "151653": {
86
+ "content": "<|vision_end|>",
87
+ "lstrip": false,
88
+ "normalized": false,
89
+ "rstrip": false,
90
+ "single_word": false,
91
+ "special": true
92
+ },
93
+ "151654": {
94
+ "content": "<|vision_pad|>",
95
+ "lstrip": false,
96
+ "normalized": false,
97
+ "rstrip": false,
98
+ "single_word": false,
99
+ "special": true
100
+ },
101
+ "151655": {
102
+ "content": "<|image_pad|>",
103
+ "lstrip": false,
104
+ "normalized": false,
105
+ "rstrip": false,
106
+ "single_word": false,
107
+ "special": true
108
+ },
109
+ "151656": {
110
+ "content": "<|video_pad|>",
111
+ "lstrip": false,
112
+ "normalized": false,
113
+ "rstrip": false,
114
+ "single_word": false,
115
+ "special": true
116
+ },
117
+ "151657": {
118
+ "content": "<tool_call>",
119
+ "lstrip": false,
120
+ "normalized": false,
121
+ "rstrip": false,
122
+ "single_word": false,
123
+ "special": false
124
+ },
125
+ "151658": {
126
+ "content": "</tool_call>",
127
+ "lstrip": false,
128
+ "normalized": false,
129
+ "rstrip": false,
130
+ "single_word": false,
131
+ "special": false
132
+ },
133
+ "151659": {
134
+ "content": "<|fim_prefix|>",
135
+ "lstrip": false,
136
+ "normalized": false,
137
+ "rstrip": false,
138
+ "single_word": false,
139
+ "special": false
140
+ },
141
+ "151660": {
142
+ "content": "<|fim_middle|>",
143
+ "lstrip": false,
144
+ "normalized": false,
145
+ "rstrip": false,
146
+ "single_word": false,
147
+ "special": false
148
+ },
149
+ "151661": {
150
+ "content": "<|fim_suffix|>",
151
+ "lstrip": false,
152
+ "normalized": false,
153
+ "rstrip": false,
154
+ "single_word": false,
155
+ "special": false
156
+ },
157
+ "151662": {
158
+ "content": "<|fim_pad|>",
159
+ "lstrip": false,
160
+ "normalized": false,
161
+ "rstrip": false,
162
+ "single_word": false,
163
+ "special": false
164
+ },
165
+ "151663": {
166
+ "content": "<|repo_name|>",
167
+ "lstrip": false,
168
+ "normalized": false,
169
+ "rstrip": false,
170
+ "single_word": false,
171
+ "special": false
172
+ },
173
+ "151664": {
174
+ "content": "<|file_sep|>",
175
+ "lstrip": false,
176
+ "normalized": false,
177
+ "rstrip": false,
178
+ "single_word": false,
179
+ "special": false
180
+ }
181
+ },
182
+ "additional_special_tokens": [
183
+ "<|im_start|>",
184
+ "<|im_end|>",
185
+ "<|object_ref_start|>",
186
+ "<|object_ref_end|>",
187
+ "<|box_start|>",
188
+ "<|box_end|>",
189
+ "<|quad_start|>",
190
+ "<|quad_end|>",
191
+ "<|vision_start|>",
192
+ "<|vision_end|>",
193
+ "<|vision_pad|>",
194
+ "<|image_pad|>",
195
+ "<|video_pad|>"
196
+ ],
197
+ "bos_token": null,
198
+ "chat_template": "{%- if tools %}\n {{- '<|im_start|>system\\n' }}\n {%- if messages[0]['role'] == 'system' %}\n {{- messages[0]['content'] }}\n {%- else %}\n {{- 'You are a helpful assistant.' }}\n {%- endif %}\n {{- \"\\n\\n# Tools\\n\\nYou may call one or more functions to assist with the user query.\\n\\nYou are provided with function signatures within <tools></tools> XML tags:\\n<tools>\" }}\n {%- for tool in tools %}\n {{- \"\\n\" }}\n {{- tool | tojson }}\n {%- endfor %}\n {{- \"\\n</tools>\\n\\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\\n<tool_call>\\n{\\\"name\\\": <function-name>, \\\"arguments\\\": <args-json-object>}\\n</tool_call><|im_end|>\\n\" }}\n{%- else %}\n {%- if messages[0]['role'] == 'system' %}\n {{- '<|im_start|>system\\n' + messages[0]['content'] + '<|im_end|>\\n' }}\n {%- else %}\n {{- '<|im_start|>system\\nYou are a helpful assistant.<|im_end|>\\n' }}\n {%- endif %}\n{%- endif %}\n{%- for message in messages %}\n {%- if (message.role == \"user\") or (message.role == \"system\" and not loop.first) or (message.role == \"assistant\" and not message.tool_calls) %}\n {{- '<|im_start|>' + message.role + '\\n' + message.content + '<|im_end|>' + '\\n' }}\n {%- elif message.role == \"assistant\" %}\n {{- '<|im_start|>' + message.role }}\n {%- if message.content %}\n {{- '\\n' + message.content }}\n {%- endif %}\n {%- for tool_call in message.tool_calls %}\n {%- if tool_call.function is defined %}\n {%- set tool_call = tool_call.function %}\n {%- endif %}\n {{- '\\n<tool_call>\\n{\"name\": \"' }}\n {{- tool_call.name }}\n {{- '\", \"arguments\": ' }}\n {{- tool_call.arguments | tojson }}\n {{- '}\\n</tool_call>' }}\n {%- endfor %}\n {{- '<|im_end|>\\n' }}\n {%- elif message.role == \"tool\" %}\n {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != \"tool\") %}\n {{- '<|im_start|>user' }}\n {%- endif %}\n {{- '\\n<tool_response>\\n' }}\n {{- message.content }}\n {{- '\\n</tool_response>' }}\n {%- if loop.last or (messages[loop.index0 + 1].role != \"tool\") %}\n {{- '<|im_end|>\\n' }}\n {%- endif %}\n {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n {{- '<|im_start|>assistant\\n' }}\n{%- endif %}\n",
199
+ "clean_up_tokenization_spaces": false,
200
+ "eos_token": "<|endoftext|>",
201
+ "errors": "replace",
202
+ "extra_special_tokens": {},
203
+ "model_max_length": 131072,
204
+ "pad_token": "<|endoftext|>",
205
+ "padding_side": "left",
206
+ "split_special_tokens": false,
207
+ "tokenizer_class": "Qwen2Tokenizer",
208
+ "unk_token": null
209
+ }
checkpoint-2811/trainer_state.json ADDED
The diff for this file is too large to render. See raw diff
 
checkpoint-2811/training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7e596a9f480193cbd2a4bf4d6fda1f9ed131e1dbe00394449bfdaab1880a0f79
3
+ size 7544
checkpoint-2811/vocab.json ADDED
The diff for this file is too large to render. See raw diff
 
checkpoint-2811/zero_to_fp32.py ADDED
@@ -0,0 +1,674 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env python
2
+
3
+ # Copyright (c) Microsoft Corporation.
4
+ # SPDX-License-Identifier: Apache-2.0
5
+
6
+ # DeepSpeed Team
7
+
8
+ # This script extracts fp32 consolidated weights from a zero 1, 2 and 3 DeepSpeed checkpoints. It gets
9
+ # copied into the top level checkpoint dir, so the user can easily do the conversion at any point in
10
+ # the future. Once extracted, the weights don't require DeepSpeed and can be used in any
11
+ # application.
12
+ #
13
+ # example:
14
+ # python zero_to_fp32.py . output_dir/
15
+ # or
16
+ # python zero_to_fp32.py . output_dir/ --safe_serialization
17
+
18
+ import argparse
19
+ import torch
20
+ import glob
21
+ import math
22
+ import os
23
+ import re
24
+ import json
25
+ from tqdm import tqdm
26
+ from collections import OrderedDict
27
+ from dataclasses import dataclass
28
+
29
+ # while this script doesn't use deepspeed to recover data, since the checkpoints are pickled with
30
+ # DeepSpeed data structures it has to be available in the current python environment.
31
+ from deepspeed.utils import logger
32
+ from deepspeed.checkpoint.constants import (DS_VERSION, OPTIMIZER_STATE_DICT, SINGLE_PARTITION_OF_FP32_GROUPS,
33
+ FP32_FLAT_GROUPS, ZERO_STAGE, PARTITION_COUNT, PARAM_SHAPES, BUFFER_NAMES,
34
+ FROZEN_PARAM_SHAPES, FROZEN_PARAM_FRAGMENTS)
35
+
36
+
37
+ @dataclass
38
+ class zero_model_state:
39
+ buffers: dict()
40
+ param_shapes: dict()
41
+ shared_params: list
42
+ ds_version: int
43
+ frozen_param_shapes: dict()
44
+ frozen_param_fragments: dict()
45
+
46
+
47
+ debug = 0
48
+
49
+ # load to cpu
50
+ device = torch.device('cpu')
51
+
52
+
53
+ def atoi(text):
54
+ return int(text) if text.isdigit() else text
55
+
56
+
57
+ def natural_keys(text):
58
+ '''
59
+ alist.sort(key=natural_keys) sorts in human order
60
+ http://nedbatchelder.com/blog/200712/human_sorting.html
61
+ (See Toothy's implementation in the comments)
62
+ '''
63
+ return [atoi(c) for c in re.split(r'(\d+)', text)]
64
+
65
+
66
+ def get_model_state_file(checkpoint_dir, zero_stage):
67
+ if not os.path.isdir(checkpoint_dir):
68
+ raise FileNotFoundError(f"Directory '{checkpoint_dir}' doesn't exist")
69
+
70
+ # there should be only one file
71
+ if zero_stage <= 2:
72
+ file = os.path.join(checkpoint_dir, "mp_rank_00_model_states.pt")
73
+ elif zero_stage == 3:
74
+ file = os.path.join(checkpoint_dir, "zero_pp_rank_0_mp_rank_00_model_states.pt")
75
+
76
+ if not os.path.exists(file):
77
+ raise FileNotFoundError(f"can't find model states file at '{file}'")
78
+
79
+ return file
80
+
81
+
82
+ def get_checkpoint_files(checkpoint_dir, glob_pattern):
83
+ # XXX: need to test that this simple glob rule works for multi-node setup too
84
+ ckpt_files = sorted(glob.glob(os.path.join(checkpoint_dir, glob_pattern)), key=natural_keys)
85
+
86
+ if len(ckpt_files) == 0:
87
+ raise FileNotFoundError(f"can't find {glob_pattern} files in directory '{checkpoint_dir}'")
88
+
89
+ return ckpt_files
90
+
91
+
92
+ def get_optim_files(checkpoint_dir):
93
+ return get_checkpoint_files(checkpoint_dir, "*_optim_states.pt")
94
+
95
+
96
+ def get_model_state_files(checkpoint_dir):
97
+ return get_checkpoint_files(checkpoint_dir, "*_model_states.pt")
98
+
99
+
100
+ def parse_model_states(files):
101
+ zero_model_states = []
102
+ for file in files:
103
+ state_dict = torch.load(file, map_location=device)
104
+
105
+ if BUFFER_NAMES not in state_dict:
106
+ raise ValueError(f"{file} is not a model state checkpoint")
107
+ buffer_names = state_dict[BUFFER_NAMES]
108
+ if debug:
109
+ print("Found buffers:", buffer_names)
110
+
111
+ # recover just the buffers while restoring them to fp32 if they were saved in fp16
112
+ buffers = {k: v.float() for k, v in state_dict["module"].items() if k in buffer_names}
113
+ param_shapes = state_dict[PARAM_SHAPES]
114
+
115
+ # collect parameters that are included in param_shapes
116
+ param_names = []
117
+ for s in param_shapes:
118
+ for name in s.keys():
119
+ param_names.append(name)
120
+
121
+ # update with frozen parameters
122
+ frozen_param_shapes = state_dict.get(FROZEN_PARAM_SHAPES, None)
123
+ if frozen_param_shapes is not None:
124
+ if debug:
125
+ print(f"Found frozen_param_shapes: {frozen_param_shapes}")
126
+ param_names += list(frozen_param_shapes.keys())
127
+
128
+ # handle shared params
129
+ shared_params = [[k, v] for k, v in state_dict["shared_params"].items()]
130
+
131
+ ds_version = state_dict.get(DS_VERSION, None)
132
+
133
+ frozen_param_fragments = state_dict.get(FROZEN_PARAM_FRAGMENTS, None)
134
+
135
+ z_model_state = zero_model_state(buffers=buffers,
136
+ param_shapes=param_shapes,
137
+ shared_params=shared_params,
138
+ ds_version=ds_version,
139
+ frozen_param_shapes=frozen_param_shapes,
140
+ frozen_param_fragments=frozen_param_fragments)
141
+ zero_model_states.append(z_model_state)
142
+
143
+ return zero_model_states
144
+
145
+
146
+ def parse_optim_states(files, ds_checkpoint_dir):
147
+ total_files = len(files)
148
+ state_dicts = []
149
+ for f in files:
150
+ state_dict = torch.load(f, map_location=device)
151
+ # immediately discard the potentially huge 2 optimizer states as we only care for fp32 master weights
152
+ # and also handle the case where it was already removed by another helper script
153
+ state_dict["optimizer_state_dict"].pop("optimizer_state_dict", None)
154
+ state_dicts.append(state_dict)
155
+
156
+ if not ZERO_STAGE in state_dicts[0][OPTIMIZER_STATE_DICT]:
157
+ raise ValueError(f"{files[0]} is not a zero checkpoint")
158
+ zero_stage = state_dicts[0][OPTIMIZER_STATE_DICT][ZERO_STAGE]
159
+ world_size = state_dicts[0][OPTIMIZER_STATE_DICT][PARTITION_COUNT]
160
+
161
+ # For ZeRO-2 each param group can have different partition_count as data parallelism for expert
162
+ # parameters can be different from data parallelism for non-expert parameters. So we can just
163
+ # use the max of the partition_count to get the dp world_size.
164
+
165
+ if type(world_size) is list:
166
+ world_size = max(world_size)
167
+
168
+ if world_size != total_files:
169
+ raise ValueError(
170
+ f"Expected {world_size} of '*_optim_states.pt' under '{ds_checkpoint_dir}' but found {total_files} files. "
171
+ "Possibly due to an overwrite of an old checkpoint, or a checkpoint didn't get saved by one or more processes."
172
+ )
173
+
174
+ # the groups are named differently in each stage
175
+ if zero_stage <= 2:
176
+ fp32_groups_key = SINGLE_PARTITION_OF_FP32_GROUPS
177
+ elif zero_stage == 3:
178
+ fp32_groups_key = FP32_FLAT_GROUPS
179
+ else:
180
+ raise ValueError(f"unknown zero stage {zero_stage}")
181
+
182
+ if zero_stage <= 2:
183
+ fp32_flat_groups = [state_dicts[i][OPTIMIZER_STATE_DICT][fp32_groups_key] for i in range(len(state_dicts))]
184
+ elif zero_stage == 3:
185
+ # if there is more than one param group, there will be multiple flattened tensors - one
186
+ # flattened tensor per group - for simplicity merge them into a single tensor
187
+ #
188
+ # XXX: could make the script more memory efficient for when there are multiple groups - it
189
+ # will require matching the sub-lists of param_shapes for each param group flattened tensor
190
+
191
+ fp32_flat_groups = [
192
+ torch.cat(state_dicts[i][OPTIMIZER_STATE_DICT][fp32_groups_key], 0) for i in range(len(state_dicts))
193
+ ]
194
+
195
+ return zero_stage, world_size, fp32_flat_groups
196
+
197
+
198
+ def _get_fp32_state_dict_from_zero_checkpoint(ds_checkpoint_dir, exclude_frozen_parameters):
199
+ """
200
+ Returns fp32 state_dict reconstructed from ds checkpoint
201
+
202
+ Args:
203
+ - ``ds_checkpoint_dir``: path to the deepspeed checkpoint folder (where the optimizer files are)
204
+
205
+ """
206
+ print(f"Processing zero checkpoint '{ds_checkpoint_dir}'")
207
+
208
+ optim_files = get_optim_files(ds_checkpoint_dir)
209
+ zero_stage, world_size, fp32_flat_groups = parse_optim_states(optim_files, ds_checkpoint_dir)
210
+ print(f"Detected checkpoint of type zero stage {zero_stage}, world_size: {world_size}")
211
+
212
+ model_files = get_model_state_files(ds_checkpoint_dir)
213
+
214
+ zero_model_states = parse_model_states(model_files)
215
+ print(f'Parsing checkpoint created by deepspeed=={zero_model_states[0].ds_version}')
216
+
217
+ if zero_stage <= 2:
218
+ return _get_fp32_state_dict_from_zero2_checkpoint(world_size, fp32_flat_groups, zero_model_states,
219
+ exclude_frozen_parameters)
220
+ elif zero_stage == 3:
221
+ return _get_fp32_state_dict_from_zero3_checkpoint(world_size, fp32_flat_groups, zero_model_states,
222
+ exclude_frozen_parameters)
223
+
224
+
225
+ def _zero2_merge_frozen_params(state_dict, zero_model_states):
226
+ if zero_model_states[0].frozen_param_shapes is None or len(zero_model_states[0].frozen_param_shapes) == 0:
227
+ return
228
+
229
+ frozen_param_shapes = zero_model_states[0].frozen_param_shapes
230
+ frozen_param_fragments = zero_model_states[0].frozen_param_fragments
231
+
232
+ if debug:
233
+ num_elem = sum(s.numel() for s in frozen_param_shapes.values())
234
+ print(f'rank 0: {FROZEN_PARAM_SHAPES}.numel = {num_elem}')
235
+
236
+ wanted_params = len(frozen_param_shapes)
237
+ wanted_numel = sum(s.numel() for s in frozen_param_shapes.values())
238
+ avail_numel = sum([p.numel() for p in frozen_param_fragments.values()])
239
+ print(f'Frozen params: Have {avail_numel} numels to process.')
240
+ print(f'Frozen params: Need {wanted_numel} numels in {wanted_params} params')
241
+
242
+ total_params = 0
243
+ total_numel = 0
244
+ for name, shape in frozen_param_shapes.items():
245
+ total_params += 1
246
+ unpartitioned_numel = shape.numel()
247
+ total_numel += unpartitioned_numel
248
+
249
+ state_dict[name] = frozen_param_fragments[name]
250
+
251
+ if debug:
252
+ print(f"{name} full shape: {shape} unpartitioned numel {unpartitioned_numel} ")
253
+
254
+ print(f"Reconstructed Frozen fp32 state dict with {total_params} params {total_numel} elements")
255
+
256
+
257
+ def _has_callable(obj, fn):
258
+ attr = getattr(obj, fn, None)
259
+ return callable(attr)
260
+
261
+
262
+ def _zero2_merge_trainable_params(state_dict, world_size, fp32_flat_groups, zero_model_states):
263
+ param_shapes = zero_model_states[0].param_shapes
264
+
265
+ # Reconstruction protocol:
266
+ #
267
+ # XXX: document this
268
+
269
+ if debug:
270
+ for i in range(world_size):
271
+ for j in range(len(fp32_flat_groups[0])):
272
+ print(f"{FP32_FLAT_GROUPS}[{i}][{j}].shape={fp32_flat_groups[i][j].shape}")
273
+
274
+ # XXX: memory usage doubles here (zero2)
275
+ num_param_groups = len(fp32_flat_groups[0])
276
+ merged_single_partition_of_fp32_groups = []
277
+ for i in range(num_param_groups):
278
+ merged_partitions = [sd[i] for sd in fp32_flat_groups]
279
+ full_single_fp32_vector = torch.cat(merged_partitions, 0)
280
+ merged_single_partition_of_fp32_groups.append(full_single_fp32_vector)
281
+ avail_numel = sum(
282
+ [full_single_fp32_vector.numel() for full_single_fp32_vector in merged_single_partition_of_fp32_groups])
283
+
284
+ if debug:
285
+ wanted_params = sum([len(shapes) for shapes in param_shapes])
286
+ wanted_numel = sum([sum(shape.numel() for shape in shapes.values()) for shapes in param_shapes])
287
+ # not asserting if there is a mismatch due to possible padding
288
+ print(f"Have {avail_numel} numels to process.")
289
+ print(f"Need {wanted_numel} numels in {wanted_params} params.")
290
+
291
+ # params
292
+ # XXX: for huge models that can't fit into the host's RAM we will have to recode this to support
293
+ # out-of-core computing solution
294
+ total_numel = 0
295
+ total_params = 0
296
+ for shapes, full_single_fp32_vector in zip(param_shapes, merged_single_partition_of_fp32_groups):
297
+ offset = 0
298
+ avail_numel = full_single_fp32_vector.numel()
299
+ for name, shape in shapes.items():
300
+
301
+ unpartitioned_numel = shape.numel() if _has_callable(shape, 'numel') else math.prod(shape)
302
+ total_numel += unpartitioned_numel
303
+ total_params += 1
304
+
305
+ if debug:
306
+ print(f"{name} full shape: {shape} unpartitioned numel {unpartitioned_numel} ")
307
+ state_dict[name] = full_single_fp32_vector.narrow(0, offset, unpartitioned_numel).view(shape)
308
+ offset += unpartitioned_numel
309
+
310
+ # Z2 started to align to 2*world_size to improve nccl performance. Therefore both offset and
311
+ # avail_numel can differ by anywhere between 0..2*world_size. Due to two unrelated complex
312
+ # paddings performed in the code it's almost impossible to predict the exact numbers w/o the
313
+ # live optimizer object, so we are checking that the numbers are within the right range
314
+ align_to = 2 * world_size
315
+
316
+ def zero2_align(x):
317
+ return align_to * math.ceil(x / align_to)
318
+
319
+ if debug:
320
+ print(f"original offset={offset}, avail_numel={avail_numel}")
321
+
322
+ offset = zero2_align(offset)
323
+ avail_numel = zero2_align(avail_numel)
324
+
325
+ if debug:
326
+ print(f"aligned offset={offset}, avail_numel={avail_numel}")
327
+
328
+ # Sanity check
329
+ if offset != avail_numel:
330
+ raise ValueError(f"consumed {offset} numels out of {avail_numel} - something is wrong")
331
+
332
+ print(f"Reconstructed fp32 state dict with {total_params} params {total_numel} elements")
333
+
334
+
335
+ def _get_fp32_state_dict_from_zero2_checkpoint(world_size, fp32_flat_groups, zero_model_states,
336
+ exclude_frozen_parameters):
337
+ state_dict = OrderedDict()
338
+
339
+ # buffers
340
+ buffers = zero_model_states[0].buffers
341
+ state_dict.update(buffers)
342
+ if debug:
343
+ print(f"added {len(buffers)} buffers")
344
+
345
+ if not exclude_frozen_parameters:
346
+ _zero2_merge_frozen_params(state_dict, zero_model_states)
347
+
348
+ _zero2_merge_trainable_params(state_dict, world_size, fp32_flat_groups, zero_model_states)
349
+
350
+ # recover shared parameters
351
+ for pair in zero_model_states[0].shared_params:
352
+ if pair[1] in state_dict:
353
+ state_dict[pair[0]] = state_dict[pair[1]]
354
+
355
+ return state_dict
356
+
357
+
358
+ def zero3_partitioned_param_info(unpartitioned_numel, world_size):
359
+ remainder = unpartitioned_numel % world_size
360
+ padding_numel = (world_size - remainder) if remainder else 0
361
+ partitioned_numel = math.ceil(unpartitioned_numel / world_size)
362
+ return partitioned_numel, padding_numel
363
+
364
+
365
+ def _zero3_merge_frozen_params(state_dict, world_size, zero_model_states):
366
+ if zero_model_states[0].frozen_param_shapes is None or len(zero_model_states[0].frozen_param_shapes) == 0:
367
+ return
368
+
369
+ if debug:
370
+ for i in range(world_size):
371
+ num_elem = sum(s.numel() for s in zero_model_states[i].frozen_param_fragments.values())
372
+ print(f'rank {i}: {FROZEN_PARAM_SHAPES}.numel = {num_elem}')
373
+
374
+ frozen_param_shapes = zero_model_states[0].frozen_param_shapes
375
+ wanted_params = len(frozen_param_shapes)
376
+ wanted_numel = sum(s.numel() for s in frozen_param_shapes.values())
377
+ avail_numel = sum([p.numel() for p in zero_model_states[0].frozen_param_fragments.values()]) * world_size
378
+ print(f'Frozen params: Have {avail_numel} numels to process.')
379
+ print(f'Frozen params: Need {wanted_numel} numels in {wanted_params} params')
380
+
381
+ total_params = 0
382
+ total_numel = 0
383
+ for name, shape in zero_model_states[0].frozen_param_shapes.items():
384
+ total_params += 1
385
+ unpartitioned_numel = shape.numel()
386
+ total_numel += unpartitioned_numel
387
+
388
+ param_frags = tuple(model_state.frozen_param_fragments[name] for model_state in zero_model_states)
389
+ state_dict[name] = torch.cat(param_frags, 0).narrow(0, 0, unpartitioned_numel).view(shape)
390
+
391
+ partitioned_numel, partitioned_padding_numel = zero3_partitioned_param_info(unpartitioned_numel, world_size)
392
+
393
+ if debug:
394
+ print(
395
+ f"Frozen params: {total_params} {name} full shape: {shape} partition0 numel={partitioned_numel} partitioned_padding_numel={partitioned_padding_numel}"
396
+ )
397
+
398
+ print(f"Reconstructed Frozen fp32 state dict with {total_params} params {total_numel} elements")
399
+
400
+
401
+ def _zero3_merge_trainable_params(state_dict, world_size, fp32_flat_groups, zero_model_states):
402
+ param_shapes = zero_model_states[0].param_shapes
403
+ avail_numel = fp32_flat_groups[0].numel() * world_size
404
+ # Reconstruction protocol: For zero3 we need to zip the partitions together at boundary of each
405
+ # param, re-consolidating each param, while dealing with padding if any
406
+
407
+ # merge list of dicts, preserving order
408
+ param_shapes = {k: v for d in param_shapes for k, v in d.items()}
409
+
410
+ if debug:
411
+ for i in range(world_size):
412
+ print(f"{FP32_FLAT_GROUPS}[{i}].shape={fp32_flat_groups[i].shape}")
413
+
414
+ wanted_params = len(param_shapes)
415
+ wanted_numel = sum(shape.numel() for shape in param_shapes.values())
416
+ # not asserting if there is a mismatch due to possible padding
417
+ avail_numel = fp32_flat_groups[0].numel() * world_size
418
+ print(f"Trainable params: Have {avail_numel} numels to process.")
419
+ print(f"Trainable params: Need {wanted_numel} numels in {wanted_params} params.")
420
+
421
+ # params
422
+ # XXX: for huge models that can't fit into the host's RAM we will have to recode this to support
423
+ # out-of-core computing solution
424
+ offset = 0
425
+ total_numel = 0
426
+ total_params = 0
427
+ for name, shape in tqdm(param_shapes.items(), desc='Gathering Sharded Weights'):
428
+ unpartitioned_numel = shape.numel()
429
+ total_numel += unpartitioned_numel
430
+ total_params += 1
431
+ partitioned_numel, partitioned_padding_numel = zero3_partitioned_param_info(unpartitioned_numel, world_size)
432
+
433
+ if debug:
434
+ print(
435
+ f"Trainable params: {total_params} {name} full shape: {shape} partition0 numel={partitioned_numel} partitioned_padding_numel={partitioned_padding_numel}"
436
+ )
437
+
438
+ # XXX: memory usage doubles here
439
+ state_dict[name] = torch.cat(
440
+ tuple(fp32_flat_groups[i].narrow(0, offset, partitioned_numel) for i in range(world_size)),
441
+ 0).narrow(0, 0, unpartitioned_numel).view(shape)
442
+ offset += partitioned_numel
443
+
444
+ offset *= world_size
445
+
446
+ # Sanity check
447
+ if offset != avail_numel:
448
+ raise ValueError(f"consumed {offset} numels out of {avail_numel} - something is wrong")
449
+
450
+ print(f"Reconstructed Trainable fp32 state dict with {total_params} params {total_numel} elements")
451
+
452
+
453
+ def _get_fp32_state_dict_from_zero3_checkpoint(world_size, fp32_flat_groups, zero_model_states,
454
+ exclude_frozen_parameters):
455
+ state_dict = OrderedDict()
456
+
457
+ # buffers
458
+ buffers = zero_model_states[0].buffers
459
+ state_dict.update(buffers)
460
+ if debug:
461
+ print(f"added {len(buffers)} buffers")
462
+
463
+ if not exclude_frozen_parameters:
464
+ _zero3_merge_frozen_params(state_dict, world_size, zero_model_states)
465
+
466
+ _zero3_merge_trainable_params(state_dict, world_size, fp32_flat_groups, zero_model_states)
467
+
468
+ # recover shared parameters
469
+ for pair in zero_model_states[0].shared_params:
470
+ if pair[1] in state_dict:
471
+ state_dict[pair[0]] = state_dict[pair[1]]
472
+
473
+ return state_dict
474
+
475
+
476
+ def get_fp32_state_dict_from_zero_checkpoint(checkpoint_dir, tag=None, exclude_frozen_parameters=False):
477
+ """
478
+ Convert ZeRO 2 or 3 checkpoint into a single fp32 consolidated state_dict that can be loaded with
479
+ ``load_state_dict()`` and used for training without DeepSpeed or shared with others, for example
480
+ via a model hub.
481
+
482
+ Args:
483
+ - ``checkpoint_dir``: path to the desired checkpoint folder
484
+ - ``tag``: checkpoint tag used as a unique identifier for checkpoint. If not provided will attempt to load tag in 'latest' file. e.g., ``global_step14``
485
+ - ``exclude_frozen_parameters``: exclude frozen parameters
486
+
487
+ Returns:
488
+ - pytorch ``state_dict``
489
+
490
+ Note: this approach may not work if your application doesn't have sufficient free CPU memory and
491
+ you may need to use the offline approach using the ``zero_to_fp32.py`` script that is saved with
492
+ the checkpoint.
493
+
494
+ A typical usage might be ::
495
+
496
+ from deepspeed.utils.zero_to_fp32 import get_fp32_state_dict_from_zero_checkpoint
497
+ # do the training and checkpoint saving
498
+ state_dict = get_fp32_state_dict_from_zero_checkpoint(checkpoint_dir) # already on cpu
499
+ model = model.cpu() # move to cpu
500
+ model.load_state_dict(state_dict)
501
+ # submit to model hub or save the model to share with others
502
+
503
+ In this example the ``model`` will no longer be usable in the deepspeed context of the same
504
+ application. i.e. you will need to re-initialize the deepspeed engine, since
505
+ ``model.load_state_dict(state_dict)`` will remove all the deepspeed magic from it.
506
+
507
+ If you want it all done for you, use ``load_state_dict_from_zero_checkpoint`` instead.
508
+
509
+ """
510
+ if tag is None:
511
+ latest_path = os.path.join(checkpoint_dir, 'latest')
512
+ if os.path.isfile(latest_path):
513
+ with open(latest_path, 'r') as fd:
514
+ tag = fd.read().strip()
515
+ else:
516
+ raise ValueError(f"Unable to find 'latest' file at {latest_path}")
517
+
518
+ ds_checkpoint_dir = os.path.join(checkpoint_dir, tag)
519
+
520
+ if not os.path.isdir(ds_checkpoint_dir):
521
+ raise FileNotFoundError(f"Directory '{ds_checkpoint_dir}' doesn't exist")
522
+
523
+ return _get_fp32_state_dict_from_zero_checkpoint(ds_checkpoint_dir, exclude_frozen_parameters)
524
+
525
+
526
+ def convert_zero_checkpoint_to_fp32_state_dict(checkpoint_dir,
527
+ output_dir,
528
+ max_shard_size="5GB",
529
+ safe_serialization=False,
530
+ tag=None,
531
+ exclude_frozen_parameters=False):
532
+ """
533
+ Convert ZeRO 2 or 3 checkpoint into a single fp32 consolidated ``state_dict`` file that can be
534
+ loaded with ``torch.load(file)`` + ``load_state_dict()`` and used for training without DeepSpeed.
535
+
536
+ Args:
537
+ - ``checkpoint_dir``: path to the desired checkpoint folder. (one that contains the tag-folder, like ``global_step14``)
538
+ - ``output_dir``: directory to the pytorch fp32 state_dict output files
539
+ - ``max_shard_size``: the maximum size for a checkpoint before being sharded, default value is 5GB
540
+ - ``safe_serialization``: whether to save the model using `safetensors` or the traditional PyTorch way (that uses `pickle`).
541
+ - ``tag``: checkpoint tag used as a unique identifier for checkpoint. If not provided will attempt to load tag in the file named ``latest`` in the checkpoint folder, e.g., ``global_step14``
542
+ - ``exclude_frozen_parameters``: exclude frozen parameters
543
+ """
544
+ # Dependency pre-check
545
+ if safe_serialization:
546
+ try:
547
+ from safetensors.torch import save_file
548
+ except ImportError:
549
+ print('If you want to use `safe_serialization`, please `pip install safetensors`')
550
+ raise
551
+ if max_shard_size is not None:
552
+ try:
553
+ from huggingface_hub import split_torch_state_dict_into_shards
554
+ except ImportError:
555
+ print('If you want to use `max_shard_size`, please `pip install huggingface_hub`')
556
+ raise
557
+
558
+ # Convert zero checkpoint to state_dict
559
+ state_dict = get_fp32_state_dict_from_zero_checkpoint(checkpoint_dir, tag, exclude_frozen_parameters)
560
+
561
+ # Shard the model if it is too big.
562
+ weights_name = "model.safetensors" if safe_serialization else "pytorch_model.bin"
563
+ if max_shard_size is not None:
564
+ filename_pattern = weights_name.replace(".bin", "{suffix}.bin").replace(".safetensors", "{suffix}.safetensors")
565
+ state_dict_split = split_torch_state_dict_into_shards(state_dict,
566
+ filename_pattern=filename_pattern,
567
+ max_shard_size=max_shard_size)
568
+ else:
569
+ from collections import namedtuple
570
+ StateDictSplit = namedtuple("StateDictSplit", ["is_sharded", "filename_to_tensors"])
571
+ state_dict_split = StateDictSplit(is_sharded=False,
572
+ filename_to_tensors={weights_name: list(state_dict.keys())})
573
+
574
+ # Save the model
575
+ filename_to_tensors = state_dict_split.filename_to_tensors.items()
576
+ for shard_file, tensors in tqdm(filename_to_tensors, desc="Saving checkpoint shards"):
577
+ shard = {tensor: state_dict[tensor].contiguous() for tensor in tensors}
578
+ output_path = os.path.join(output_dir, shard_file)
579
+ if safe_serialization:
580
+ save_file(shard, output_path, metadata={"format": "pt"})
581
+ else:
582
+ torch.save(shard, output_path)
583
+
584
+ # Save index if sharded
585
+ if state_dict_split.is_sharded:
586
+ index = {
587
+ "metadata": state_dict_split.metadata,
588
+ "weight_map": state_dict_split.tensor_to_filename,
589
+ }
590
+ save_index_file = "model.safetensors.index.json" if safe_serialization else "pytorch_model.bin.index.json"
591
+ save_index_file = os.path.join(output_dir, save_index_file)
592
+ with open(save_index_file, "w", encoding="utf-8") as f:
593
+ content = json.dumps(index, indent=2, sort_keys=True) + "\n"
594
+ f.write(content)
595
+
596
+
597
+ def load_state_dict_from_zero_checkpoint(model, checkpoint_dir, tag=None):
598
+ """
599
+ 1. Put the provided model to cpu
600
+ 2. Convert ZeRO 2 or 3 checkpoint into a single fp32 consolidated ``state_dict``
601
+ 3. Load it into the provided model
602
+
603
+ Args:
604
+ - ``model``: the model object to update
605
+ - ``checkpoint_dir``: path to the desired checkpoint folder. (one that contains the tag-folder, like ``global_step14``)
606
+ - ``tag``: checkpoint tag used as a unique identifier for checkpoint. If not provided will attempt to load tag in the file named ``latest`` in the checkpoint folder, e.g., ``global_step14``
607
+
608
+ Returns:
609
+ - ``model`: modified model
610
+
611
+ Make sure you have plenty of CPU memory available before you call this function. If you don't
612
+ have enough use the ``zero_to_fp32.py`` utility to do the conversion. You will find it
613
+ conveniently placed for you in the checkpoint folder.
614
+
615
+ A typical usage might be ::
616
+
617
+ from deepspeed.utils.zero_to_fp32 import load_state_dict_from_zero_checkpoint
618
+ model = load_state_dict_from_zero_checkpoint(trainer.model, checkpoint_dir)
619
+ # submit to model hub or save the model to share with others
620
+
621
+ Note, that once this was run, the ``model`` will no longer be usable in the deepspeed context
622
+ of the same application. i.e. you will need to re-initialize the deepspeed engine, since
623
+ ``model.load_state_dict(state_dict)`` will remove all the deepspeed magic from it.
624
+
625
+ """
626
+ logger.info(f"Extracting fp32 weights")
627
+ state_dict = get_fp32_state_dict_from_zero_checkpoint(checkpoint_dir, tag)
628
+
629
+ logger.info(f"Overwriting model with fp32 weights")
630
+ model = model.cpu()
631
+ model.load_state_dict(state_dict, strict=False)
632
+
633
+ return model
634
+
635
+
636
+ if __name__ == "__main__":
637
+ parser = argparse.ArgumentParser()
638
+ parser.add_argument("checkpoint_dir",
639
+ type=str,
640
+ help="path to the desired checkpoint folder, e.g., path/checkpoint-12")
641
+ parser.add_argument("output_dir",
642
+ type=str,
643
+ help="directory to the pytorch fp32 state_dict output files"
644
+ "(e.g. path/checkpoint-12-output/)")
645
+ parser.add_argument(
646
+ "--max_shard_size",
647
+ type=str,
648
+ default="5GB",
649
+ help="The maximum size for a checkpoint before being sharded. Checkpoints shard will then be each of size"
650
+ "lower than this size. If expressed as a string, needs to be digits followed by a unit (like `5MB`"
651
+ "We default it to 5GB in order for models to be able to run easily on free-tier google colab instances"
652
+ "without CPU OOM issues.")
653
+ parser.add_argument(
654
+ "--safe_serialization",
655
+ default=False,
656
+ action='store_true',
657
+ help="Whether to save the model using `safetensors` or the traditional PyTorch way (that uses `pickle`).")
658
+ parser.add_argument("-t",
659
+ "--tag",
660
+ type=str,
661
+ default=None,
662
+ help="checkpoint tag used as a unique identifier for checkpoint. e.g., global_step1")
663
+ parser.add_argument("--exclude_frozen_parameters", action='store_true', help="exclude frozen parameters")
664
+ parser.add_argument("-d", "--debug", action='store_true', help="enable debug")
665
+ args = parser.parse_args()
666
+
667
+ debug = args.debug
668
+
669
+ convert_zero_checkpoint_to_fp32_state_dict(args.checkpoint_dir,
670
+ args.output_dir,
671
+ max_shard_size=args.max_shard_size,
672
+ safe_serialization=args.safe_serialization,
673
+ tag=args.tag,
674
+ exclude_frozen_parameters=args.exclude_frozen_parameters)
checkpoint-938/README.md ADDED
@@ -0,0 +1,202 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: Qwen/Qwen2.5-1.5B
3
+ library_name: peft
4
+ ---
5
+
6
+ # Model Card for Model ID
7
+
8
+ <!-- Provide a quick summary of what the model is/does. -->
9
+
10
+
11
+
12
+ ## Model Details
13
+
14
+ ### Model Description
15
+
16
+ <!-- Provide a longer summary of what this model is. -->
17
+
18
+
19
+
20
+ - **Developed by:** [More Information Needed]
21
+ - **Funded by [optional]:** [More Information Needed]
22
+ - **Shared by [optional]:** [More Information Needed]
23
+ - **Model type:** [More Information Needed]
24
+ - **Language(s) (NLP):** [More Information Needed]
25
+ - **License:** [More Information Needed]
26
+ - **Finetuned from model [optional]:** [More Information Needed]
27
+
28
+ ### Model Sources [optional]
29
+
30
+ <!-- Provide the basic links for the model. -->
31
+
32
+ - **Repository:** [More Information Needed]
33
+ - **Paper [optional]:** [More Information Needed]
34
+ - **Demo [optional]:** [More Information Needed]
35
+
36
+ ## Uses
37
+
38
+ <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
39
+
40
+ ### Direct Use
41
+
42
+ <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
43
+
44
+ [More Information Needed]
45
+
46
+ ### Downstream Use [optional]
47
+
48
+ <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
49
+
50
+ [More Information Needed]
51
+
52
+ ### Out-of-Scope Use
53
+
54
+ <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
55
+
56
+ [More Information Needed]
57
+
58
+ ## Bias, Risks, and Limitations
59
+
60
+ <!-- This section is meant to convey both technical and sociotechnical limitations. -->
61
+
62
+ [More Information Needed]
63
+
64
+ ### Recommendations
65
+
66
+ <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
67
+
68
+ Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
69
+
70
+ ## How to Get Started with the Model
71
+
72
+ Use the code below to get started with the model.
73
+
74
+ [More Information Needed]
75
+
76
+ ## Training Details
77
+
78
+ ### Training Data
79
+
80
+ <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
81
+
82
+ [More Information Needed]
83
+
84
+ ### Training Procedure
85
+
86
+ <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
87
+
88
+ #### Preprocessing [optional]
89
+
90
+ [More Information Needed]
91
+
92
+
93
+ #### Training Hyperparameters
94
+
95
+ - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
96
+
97
+ #### Speeds, Sizes, Times [optional]
98
+
99
+ <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
100
+
101
+ [More Information Needed]
102
+
103
+ ## Evaluation
104
+
105
+ <!-- This section describes the evaluation protocols and provides the results. -->
106
+
107
+ ### Testing Data, Factors & Metrics
108
+
109
+ #### Testing Data
110
+
111
+ <!-- This should link to a Dataset Card if possible. -->
112
+
113
+ [More Information Needed]
114
+
115
+ #### Factors
116
+
117
+ <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
118
+
119
+ [More Information Needed]
120
+
121
+ #### Metrics
122
+
123
+ <!-- These are the evaluation metrics being used, ideally with a description of why. -->
124
+
125
+ [More Information Needed]
126
+
127
+ ### Results
128
+
129
+ [More Information Needed]
130
+
131
+ #### Summary
132
+
133
+
134
+
135
+ ## Model Examination [optional]
136
+
137
+ <!-- Relevant interpretability work for the model goes here -->
138
+
139
+ [More Information Needed]
140
+
141
+ ## Environmental Impact
142
+
143
+ <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
144
+
145
+ Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
146
+
147
+ - **Hardware Type:** [More Information Needed]
148
+ - **Hours used:** [More Information Needed]
149
+ - **Cloud Provider:** [More Information Needed]
150
+ - **Compute Region:** [More Information Needed]
151
+ - **Carbon Emitted:** [More Information Needed]
152
+
153
+ ## Technical Specifications [optional]
154
+
155
+ ### Model Architecture and Objective
156
+
157
+ [More Information Needed]
158
+
159
+ ### Compute Infrastructure
160
+
161
+ [More Information Needed]
162
+
163
+ #### Hardware
164
+
165
+ [More Information Needed]
166
+
167
+ #### Software
168
+
169
+ [More Information Needed]
170
+
171
+ ## Citation [optional]
172
+
173
+ <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
174
+
175
+ **BibTeX:**
176
+
177
+ [More Information Needed]
178
+
179
+ **APA:**
180
+
181
+ [More Information Needed]
182
+
183
+ ## Glossary [optional]
184
+
185
+ <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
186
+
187
+ [More Information Needed]
188
+
189
+ ## More Information [optional]
190
+
191
+ [More Information Needed]
192
+
193
+ ## Model Card Authors [optional]
194
+
195
+ [More Information Needed]
196
+
197
+ ## Model Card Contact
198
+
199
+ [More Information Needed]
200
+ ### Framework versions
201
+
202
+ - PEFT 0.14.0
checkpoint-938/adapter_config.json ADDED
@@ -0,0 +1,34 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alpha_pattern": {},
3
+ "auto_mapping": null,
4
+ "base_model_name_or_path": "Qwen/Qwen2.5-1.5B",
5
+ "bias": "none",
6
+ "eva_config": null,
7
+ "exclude_modules": null,
8
+ "fan_in_fan_out": false,
9
+ "inference_mode": true,
10
+ "init_lora_weights": true,
11
+ "layer_replication": null,
12
+ "layers_pattern": null,
13
+ "layers_to_transform": null,
14
+ "loftq_config": {},
15
+ "lora_alpha": 8,
16
+ "lora_bias": false,
17
+ "lora_dropout": 0.0,
18
+ "megatron_config": null,
19
+ "megatron_core": "megatron.core",
20
+ "modules_to_save": null,
21
+ "peft_type": "LORA",
22
+ "r": 32,
23
+ "rank_pattern": {},
24
+ "revision": null,
25
+ "target_modules": [
26
+ "q_proj",
27
+ "v_proj",
28
+ "embed_tokens",
29
+ "k_proj"
30
+ ],
31
+ "task_type": "CAUSAL_LM",
32
+ "use_dora": false,
33
+ "use_rslora": false
34
+ }
checkpoint-938/adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c32b4744a5f1e25fab4fcef69341d68015e77a0722895c2178ac5d0909e2dd89
3
+ size 488520640
checkpoint-938/added_tokens.json ADDED
@@ -0,0 +1,24 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "</tool_call>": 151658,
3
+ "<tool_call>": 151657,
4
+ "<|box_end|>": 151649,
5
+ "<|box_start|>": 151648,
6
+ "<|endoftext|>": 151643,
7
+ "<|file_sep|>": 151664,
8
+ "<|fim_middle|>": 151660,
9
+ "<|fim_pad|>": 151662,
10
+ "<|fim_prefix|>": 151659,
11
+ "<|fim_suffix|>": 151661,
12
+ "<|im_end|>": 151645,
13
+ "<|im_start|>": 151644,
14
+ "<|image_pad|>": 151655,
15
+ "<|object_ref_end|>": 151647,
16
+ "<|object_ref_start|>": 151646,
17
+ "<|quad_end|>": 151651,
18
+ "<|quad_start|>": 151650,
19
+ "<|repo_name|>": 151663,
20
+ "<|video_pad|>": 151656,
21
+ "<|vision_end|>": 151653,
22
+ "<|vision_pad|>": 151654,
23
+ "<|vision_start|>": 151652
24
+ }
checkpoint-938/global_step937/bf16_zero_pp_rank_0_mp_rank_00_optim_states.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6c69eca94feda5cb826eb693257e9258b78632b4b266e40c3212f1d5c7800fe2
3
+ size 130520624
checkpoint-938/global_step937/mp_rank_00_model_states.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5112b9399d6c69ae4dd09f6f8f74af27784d1b9795cde3e95246128f650ad458
3
+ size 488645432
checkpoint-938/latest ADDED
@@ -0,0 +1 @@
 
 
1
+ global_step937
checkpoint-938/merges.txt ADDED
The diff for this file is too large to render. See raw diff
 
checkpoint-938/rng_state.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:85ff83c9a62e5d58853a2656865d280c0e257ccdcd65dad6bd3060f966059592
3
+ size 14244
checkpoint-938/scheduler.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7328f6df045aa7953e963c7065b6c08cd9c4b4a17f305d9186fb192f49d75a3f
3
+ size 1064