diff --git a/notebooks/en/_toctree.yml b/notebooks/en/_toctree.yml index 1619b9f8..1ccf658c 100644 --- a/notebooks/en/_toctree.yml +++ b/notebooks/en/_toctree.yml @@ -88,8 +88,9 @@ title: Hyperparameter Optimization with Optuna and Transformers - local: function_calling_fine_tuning_llms_on_xlam title: Fine-tuning LLMs for Function Calling with the xLAM Dataset - - + - local: grpo_vllm_online_training + title: Efficient Online Training with GRPO and vLLM in TRL + - title: Computer Vision Recipes isExpanded: false diff --git a/notebooks/en/grpo_vllm_online_training.ipynb b/notebooks/en/grpo_vllm_online_training.ipynb new file mode 100644 index 00000000..30bdf3c9 --- /dev/null +++ b/notebooks/en/grpo_vllm_online_training.ipynb @@ -0,0 +1,17209 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "id": "vKadZFQ2IdJb" + }, + "source": [ + "# Efficient Online Training with GRPO and vLLM in TRL\n", + "\n", + "_Authored by: [Sergio Paniego](https://github.com/sergiopaniego)_\n", + "\n", + "In this notebook, we'll walk through how to post-train a Large Language Model (LLM) using **Group Relative Policy Optimization (GRPO)** with TRL, enhanced by **vLLM**. We already have a [previous recipe](https://huggingface.co/learn/cookbook/fine_tuning_llm_grpo_trl) focused on GRPO itself. In contrast, this notebook emphasizes **efficient online training with vLLM**.\n", + "\n", + "Although we focus on GRPO here, the same setup applies to **any online training method in TRL** that requires generating completions during training, such as DPO. The key idea is to **leverage vLLM** to remove the generation bottleneck and significantly accelerate training.\n", + "\n", + "Why does this matter? Online training methods rely on the model generating outputs in real time, which often becomes a critical speed and memory bottleneck. **vLLM** solves this with a high-throughput, low-latency inference engine built on **PagedAttention**, enabling faster generation and more efficient memory usage—making large-scale online training far more practical.\n", + "\n", + "This notebook is designed for Colab, where we only have access to a single GPU. We'll demonstrate how to run both TRL and vLLM on the same GPU efficiently. With more GPUs, this approach can scale seamlessly: TRL allows you to dedicate separate GPUs for training and vLLM, further boosting performance.\n", + "\n", + "As you'll see, with only a few adjustments, we can directly observe a measurable gain in training efficiency.\n", + "\n", + "The diagram below illustrates the overall training workflow and highlights where vLLM (blue box) and TRL (pink box) fit into the process:" + ] + }, + { + "cell_type": "markdown", + "source": [ + "" + ], + "metadata": { + "id": "8Kq4TBG3_Um_" + } + }, + { + "cell_type": "markdown", + "metadata": { + "id": "gSHmDKNFoqjC" + }, + "source": [ + "## 1. Install Dependencies\n", + "\n", + "First, let's install the essential libraries required for fine-tuning.\n", + "The important highlight here is **TRL with vLLM support**, which enables **high-throughput, low-latency generation** during online training, removing the common bottleneck in completion generation." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": { + "id": "GCMhPmFdIGSb", + "colab": { + "base_uri": "https://localhost:8080/" + }, + "outputId": "09d81e07-5612-45a0-9270-5fa05bb0d361" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "\u001b[?25l \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m0.0/207.9 kB\u001b[0m \u001b[31m?\u001b[0m eta \u001b[36m-:--:--\u001b[0m\r\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m207.9/207.9 kB\u001b[0m \u001b[31m12.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m864.4/864.4 kB\u001b[0m \u001b[31m48.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m63.5/63.5 MB\u001b[0m \u001b[31m41.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m325.4/325.4 kB\u001b[0m \u001b[31m28.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m436.4/436.4 MB\u001b[0m \u001b[31m5.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m180.0/180.0 kB\u001b[0m \u001b[31m15.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m45.5/45.5 kB\u001b[0m \u001b[31m4.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m45.4/45.4 kB\u001b[0m \u001b[31m3.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m3.9/3.9 MB\u001b[0m \u001b[31m78.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2.3/2.3 MB\u001b[0m \u001b[31m80.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m117.2/117.2 MB\u001b[0m \u001b[31m20.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m7.9/7.9 MB\u001b[0m \u001b[31m104.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m564.6/564.6 kB\u001b[0m \u001b[31m33.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m96.2/96.2 kB\u001b[0m \u001b[31m8.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m15.0/15.0 MB\u001b[0m \u001b[31m126.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m6.5/6.5 MB\u001b[0m \u001b[31m48.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m3.0/3.0 MB\u001b[0m \u001b[31m73.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m70.1/70.1 MB\u001b[0m \u001b[31m33.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.3/1.3 MB\u001b[0m \u001b[31m66.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m387.9/387.9 kB\u001b[0m \u001b[31m31.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m284.9/284.9 kB\u001b[0m \u001b[31m25.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m213.6/213.6 kB\u001b[0m \u001b[31m20.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m180.7/180.7 kB\u001b[0m \u001b[31m16.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m71.6/71.6 kB\u001b[0m \u001b[31m7.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m452.2/452.2 kB\u001b[0m \u001b[31m34.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m42.4/42.4 MB\u001b[0m \u001b[31m54.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m331.1/331.1 kB\u001b[0m \u001b[31m31.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m510.8/510.8 kB\u001b[0m \u001b[31m43.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m6.3/6.3 MB\u001b[0m \u001b[31m111.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m4.7/4.7 MB\u001b[0m \u001b[31m46.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m951.1/951.1 kB\u001b[0m \u001b[31m45.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", + "\u001b[?25h\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n", + "ipython 7.34.0 requires jedi>=0.16, which is not installed.\u001b[0m\u001b[31m\n", + "\u001b[0m" + ] + } + ], + "source": [ + "!pip install -U -q trl[vllm] peft math_verify trackio\n", + "\n", + "# Tested with trl==0.23.0, peft==0.17.1, math_verify==0.8.0, vllm==0.10.2, trackio==0.5.0" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "V0-2Lso6wkIh" + }, + "source": [ + "Authenticate with your Hugging Face account to save and share your model directly from this notebook 🗝️." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "id": "xcL4-bwGIoaR", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 17, + "referenced_widgets": [ + "24c88fcfb4db4c25a12826dc3225c3ce", + "bfe60ff41ea945a7a98174359cc50704", + "df8408f963284264821b753f5fb94bac", + "284112b4ebdb49f6afca757ca89c84fd", + "7a07f2e093b44306af017f6301b5f026", + "ba37714941b44b54b53df82090678a59", + "549b9b8e58024ba0a8960e046327bac9", + "c6c6a578564d4eda9bacf8e7bd983124", + "bfa2d7bb96d246659a79cb240927617d", + "eca1d61071a848cfbde43b1fe92080f3", + "435e6cb8b924472f93afb7c2f0e849b6", + "3bcd66944c334a728c300a9420d686cd", + "6e33bc3052524de88d7314ecf4597590", + "6e8cb8d286ff40f0bda822a87df5052d", + "de38e04fa3864af195cd0942670a57fa", + "8c8ddef66a844c4d978d69ca00af1c65", + "97f4110a52a54cbc8e9b479e71f3078c", + "6ec98dbdb99d4312a3e7e2d91fe32053", + "4a15dde2e22b4c589fa84324d702799f", + "4676cd8cbcd345b0a43520a1c105520d" + ] + }, + "outputId": "d646dfaa-df9e-4540-ad97-314a266ed3e2" + }, + "outputs": [ + { + "output_type": "display_data", + "data": { + "text/plain": [ + "VBox(children=(HTML(value='
Step | \n", + "Training Loss | \n", + "
---|---|
10 | \n", + "0.080900 | \n", + "
20 | \n", + "0.128500 | \n", + "
30 | \n", + "0.209800 | \n", + "
40 | \n", + "0.281500 | \n", + "
50 | \n", + "0.395200 | \n", + "
60 | \n", + "0.297500 | \n", + "
70 | \n", + "0.294200 | \n", + "
80 | \n", + "0.246800 | \n", + "
90 | \n", + "0.205800 | \n", + "
100 | \n", + "0.171900 | \n", + "
110 | \n", + "0.098800 | \n", + "
120 | \n", + "0.111600 | \n", + "
130 | \n", + "0.086400 | \n", + "
140 | \n", + "0.055100 | \n", + "
150 | \n", + "-0.014400 | \n", + "
160 | \n", + "0.089900 | \n", + "
170 | \n", + "0.051200 | \n", + "
180 | \n", + "0.081700 | \n", + "
190 | \n", + "0.097800 | \n", + "
200 | \n", + "0.039200 | \n", + "
210 | \n", + "0.073300 | \n", + "
220 | \n", + "0.220300 | \n", + "
230 | \n", + "0.040700 | \n", + "
240 | \n", + "0.165100 | \n", + "
250 | \n", + "0.026300 | \n", + "
260 | \n", + "0.096200 | \n", + "
270 | \n", + "0.063000 | \n", + "
280 | \n", + "0.024200 | \n", + "
290 | \n", + "0.054100 | \n", + "
300 | \n", + "0.051000 | \n", + "
310 | \n", + "0.124900 | \n", + "
320 | \n", + "0.047900 | \n", + "
330 | \n", + "-0.000400 | \n", + "
340 | \n", + "0.031200 | \n", + "
350 | \n", + "0.006600 | \n", + "
360 | \n", + "0.023100 | \n", + "
370 | \n", + "0.014700 | \n", + "
380 | \n", + "0.007000 | \n", + "
390 | \n", + "0.063100 | \n", + "
400 | \n", + "0.020700 | \n", + "
410 | \n", + "0.098100 | \n", + "
420 | \n", + "0.070300 | \n", + "
430 | \n", + "0.002400 | \n", + "
440 | \n", + "0.072800 | \n", + "
450 | \n", + "0.013100 | \n", + "
"
+ ]
+ },
+ "metadata": {}
+ },
+ {
+ "output_type": "stream",
+ "name": "stdout",
+ "text": [
+ "INFO 10-03 11:32:44 [block_pool.py:292] Successfully reset prefix cache\n",
+ "INFO 10-03 11:32:58 [block_pool.py:292] Successfully reset prefix cache\n",
+ "INFO 10-03 11:33:12 [block_pool.py:292] Successfully reset prefix cache\n"
+ ]
+ },
+ {
+ "output_type": "stream",
+ "name": "stderr",
+ "text": [
+ "WARNING:math_verify.grader:Timeout during comparison\n"
+ ]
+ },
+ {
+ "output_type": "stream",
+ "name": "stdout",
+ "text": [
+ "INFO 10-03 11:33:33 [block_pool.py:292] Successfully reset prefix cache\n",
+ "INFO 10-03 11:33:47 [block_pool.py:292] Successfully reset prefix cache\n",
+ "INFO 10-03 11:34:00 [block_pool.py:292] Successfully reset prefix cache\n",
+ "INFO 10-03 11:34:13 [block_pool.py:292] Successfully reset prefix cache\n",
+ "INFO 10-03 11:34:41 [block_pool.py:292] Successfully reset prefix cache\n",
+ "INFO 10-03 11:34:54 [block_pool.py:292] Successfully reset prefix cache\n",
+ "INFO 10-03 11:35:08 [block_pool.py:292] Successfully reset prefix cache\n"
+ ]
+ },
+ {
+ "output_type": "stream",
+ "name": "stderr",
+ "text": [
+ "/usr/local/lib/python3.12/dist-packages/torch/utils/checkpoint.py:85: UserWarning: None of the inputs have requires_grad=True. Gradients will be None\n",
+ " warnings.warn(\n",
+ "/usr/local/lib/python3.12/dist-packages/torch/utils/checkpoint.py:85: UserWarning: None of the inputs have requires_grad=True. Gradients will be None\n",
+ " warnings.warn(\n",
+ "/usr/local/lib/python3.12/dist-packages/torch/utils/checkpoint.py:85: UserWarning: None of the inputs have requires_grad=True. Gradients will be None\n",
+ " warnings.warn(\n",
+ "/usr/local/lib/python3.12/dist-packages/torch/utils/checkpoint.py:85: UserWarning: None of the inputs have requires_grad=True. Gradients will be None\n",
+ " warnings.warn(\n"
+ ]
+ },
+ {
+ "output_type": "stream",
+ "name": "stdout",
+ "text": [
+ "INFO 10-03 11:35:22 [block_pool.py:292] Successfully reset prefix cache\n"
+ ]
+ },
+ {
+ "output_type": "stream",
+ "name": "stderr",
+ "text": [
+ "/usr/local/lib/python3.12/dist-packages/torch/utils/checkpoint.py:85: UserWarning: None of the inputs have requires_grad=True. Gradients will be None\n",
+ " warnings.warn(\n",
+ "/usr/local/lib/python3.12/dist-packages/torch/utils/checkpoint.py:85: UserWarning: None of the inputs have requires_grad=True. Gradients will be None\n",
+ " warnings.warn(\n",
+ "/usr/local/lib/python3.12/dist-packages/torch/utils/checkpoint.py:85: UserWarning: None of the inputs have requires_grad=True. Gradients will be None\n",
+ " warnings.warn(\n",
+ "/usr/local/lib/python3.12/dist-packages/torch/utils/checkpoint.py:85: UserWarning: None of the inputs have requires_grad=True. Gradients will be None\n",
+ " warnings.warn(\n"
+ ]
+ },
+ {
+ "output_type": "stream",
+ "name": "stdout",
+ "text": [
+ "INFO 10-03 11:35:38 [block_pool.py:292] Successfully reset prefix cache\n"
+ ]
+ },
+ {
+ "output_type": "stream",
+ "name": "stderr",
+ "text": [
+ "/usr/local/lib/python3.12/dist-packages/torch/utils/checkpoint.py:85: UserWarning: None of the inputs have requires_grad=True. Gradients will be None\n",
+ " warnings.warn(\n",
+ "/usr/local/lib/python3.12/dist-packages/torch/utils/checkpoint.py:85: UserWarning: None of the inputs have requires_grad=True. Gradients will be None\n",
+ " warnings.warn(\n",
+ "/usr/local/lib/python3.12/dist-packages/torch/utils/checkpoint.py:85: UserWarning: None of the inputs have requires_grad=True. Gradients will be None\n",
+ " warnings.warn(\n",
+ "/usr/local/lib/python3.12/dist-packages/torch/utils/checkpoint.py:85: UserWarning: None of the inputs have requires_grad=True. Gradients will be None\n",
+ " warnings.warn(\n",
+ "/usr/local/lib/python3.12/dist-packages/torch/utils/checkpoint.py:85: UserWarning: None of the inputs have requires_grad=True. Gradients will be None\n",
+ " warnings.warn(\n"
+ ]
+ },
+ {
+ "output_type": "stream",
+ "name": "stdout",
+ "text": [
+ "INFO 10-03 11:35:50 [block_pool.py:292] Successfully reset prefix cache\n"
+ ]
+ },
+ {
+ "output_type": "stream",
+ "name": "stderr",
+ "text": [
+ "/usr/local/lib/python3.12/dist-packages/torch/utils/checkpoint.py:85: UserWarning: None of the inputs have requires_grad=True. Gradients will be None\n",
+ " warnings.warn(\n",
+ "/usr/local/lib/python3.12/dist-packages/torch/utils/checkpoint.py:85: UserWarning: None of the inputs have requires_grad=True. Gradients will be None\n",
+ " warnings.warn(\n",
+ "/usr/local/lib/python3.12/dist-packages/torch/utils/checkpoint.py:85: UserWarning: None of the inputs have requires_grad=True. Gradients will be None\n",
+ " warnings.warn(\n",
+ "/usr/local/lib/python3.12/dist-packages/torch/utils/checkpoint.py:85: UserWarning: None of the inputs have requires_grad=True. Gradients will be None\n",
+ " warnings.warn(\n"
+ ]
+ },
+ {
+ "output_type": "stream",
+ "name": "stdout",
+ "text": [
+ "INFO 10-03 11:36:04 [block_pool.py:292] Successfully reset prefix cache\n",
+ "INFO 10-03 11:36:17 [block_pool.py:292] Successfully reset prefix cache\n",
+ "INFO 10-03 11:36:30 [block_pool.py:292] Successfully reset prefix cache\n",
+ "INFO 10-03 11:36:43 [block_pool.py:292] Successfully reset prefix cache\n",
+ "INFO 10-03 11:36:57 [block_pool.py:292] Successfully reset prefix cache\n",
+ "INFO 10-03 11:37:09 [block_pool.py:292] Successfully reset prefix cache\n",
+ "INFO 10-03 11:37:23 [block_pool.py:292] Successfully reset prefix cache\n"
+ ]
+ },
+ {
+ "output_type": "stream",
+ "name": "stderr",
+ "text": [
+ "/usr/local/lib/python3.12/dist-packages/torch/utils/checkpoint.py:85: UserWarning: None of the inputs have requires_grad=True. Gradients will be None\n",
+ " warnings.warn(\n"
+ ]
+ },
+ {
+ "output_type": "stream",
+ "name": "stdout",
+ "text": [
+ "INFO 10-03 11:37:35 [block_pool.py:292] Successfully reset prefix cache\n",
+ "INFO 10-03 11:37:48 [block_pool.py:292] Successfully reset prefix cache\n",
+ "INFO 10-03 11:38:01 [block_pool.py:292] Successfully reset prefix cache\n",
+ "INFO 10-03 11:38:14 [block_pool.py:292] Successfully reset prefix cache\n",
+ "INFO 10-03 11:38:28 [block_pool.py:292] Successfully reset prefix cache\n",
+ "INFO 10-03 11:38:42 [block_pool.py:292] Successfully reset prefix cache\n",
+ "INFO 10-03 11:38:56 [block_pool.py:292] Successfully reset prefix cache\n",
+ "INFO 10-03 11:39:09 [block_pool.py:292] Successfully reset prefix cache\n",
+ "INFO 10-03 11:39:22 [block_pool.py:292] Successfully reset prefix cache\n",
+ "INFO 10-03 11:39:35 [block_pool.py:292] Successfully reset prefix cache\n"
+ ]
+ },
+ {
+ "output_type": "stream",
+ "name": "stderr",
+ "text": [
+ "/usr/local/lib/python3.12/dist-packages/torch/utils/checkpoint.py:85: UserWarning: None of the inputs have requires_grad=True. Gradients will be None\n",
+ " warnings.warn(\n"
+ ]
+ },
+ {
+ "output_type": "stream",
+ "name": "stdout",
+ "text": [
+ "INFO 10-03 11:39:48 [block_pool.py:292] Successfully reset prefix cache\n",
+ "INFO 10-03 11:40:03 [block_pool.py:292] Successfully reset prefix cache\n",
+ "INFO 10-03 11:40:15 [block_pool.py:292] Successfully reset prefix cache\n"
+ ]
+ },
+ {
+ "output_type": "stream",
+ "name": "stderr",
+ "text": [
+ "/usr/local/lib/python3.12/dist-packages/torch/utils/checkpoint.py:85: UserWarning: None of the inputs have requires_grad=True. Gradients will be None\n",
+ " warnings.warn(\n"
+ ]
+ },
+ {
+ "output_type": "stream",
+ "name": "stdout",
+ "text": [
+ "INFO 10-03 11:40:28 [block_pool.py:292] Successfully reset prefix cache\n",
+ "INFO 10-03 11:40:40 [block_pool.py:292] Successfully reset prefix cache\n",
+ "INFO 10-03 11:40:53 [block_pool.py:292] Successfully reset prefix cache\n",
+ "INFO 10-03 11:41:06 [block_pool.py:292] Successfully reset prefix cache\n",
+ "INFO 10-03 11:41:19 [block_pool.py:292] Successfully reset prefix cache\n",
+ "INFO 10-03 11:41:31 [block_pool.py:292] Successfully reset prefix cache\n",
+ "INFO 10-03 11:41:44 [block_pool.py:292] Successfully reset prefix cache\n"
+ ]
+ },
+ {
+ "output_type": "stream",
+ "name": "stderr",
+ "text": [
+ "/usr/local/lib/python3.12/dist-packages/torch/utils/checkpoint.py:85: UserWarning: None of the inputs have requires_grad=True. Gradients will be None\n",
+ " warnings.warn(\n"
+ ]
+ },
+ {
+ "output_type": "stream",
+ "name": "stdout",
+ "text": [
+ "INFO 10-03 11:41:57 [block_pool.py:292] Successfully reset prefix cache\n",
+ "INFO 10-03 11:42:10 [block_pool.py:292] Successfully reset prefix cache\n",
+ "INFO 10-03 11:42:22 [block_pool.py:292] Successfully reset prefix cache\n",
+ "INFO 10-03 11:42:34 [block_pool.py:292] Successfully reset prefix cache\n",
+ "INFO 10-03 11:42:47 [block_pool.py:292] Successfully reset prefix cache\n",
+ "INFO 10-03 11:42:59 [block_pool.py:292] Successfully reset prefix cache\n",
+ "INFO 10-03 11:43:11 [block_pool.py:292] Successfully reset prefix cache\n",
+ "INFO 10-03 11:43:24 [block_pool.py:292] Successfully reset prefix cache\n",
+ "INFO 10-03 11:43:36 [block_pool.py:292] Successfully reset prefix cache\n",
+ "INFO 10-03 11:43:48 [block_pool.py:292] Successfully reset prefix cache\n"
+ ]
+ },
+ {
+ "output_type": "stream",
+ "name": "stderr",
+ "text": [
+ "/usr/local/lib/python3.12/dist-packages/torch/utils/checkpoint.py:85: UserWarning: None of the inputs have requires_grad=True. Gradients will be None\n",
+ " warnings.warn(\n",
+ "WARNING:math_verify.grader:Timeout during comparison\n",
+ "WARNING:math_verify.parser:Timeout during parsing: