I want to disable thinking for thinking models in load testing

[psydok]

How do I do this when testing the load through guidellm?
In other words, I need to add a parameter to the request body: "chat_template_kwargs":{"enable_thinking":False}

[sjmonson]

You can specify extra request parameters using the extra_body backend arg: --backend-args='{"extra_body":{"chat_template_kwargs":{"enable_thinking":false}}}'

[psydok]

@sjmonson Thanks for answer!

I want to run test through docker. Can you tell me how it would be correct to pass these values through environment variables?

Would it be right to do it this way? And how can this be done through env?
When guidellm config is called, all variables that can be redefined are not displayed.

sudo docker run \
  --rm -it \
  -v "./data/guidellm:/results:rw" \
  -e GUIDELLM_TARGET=http://localhost:8000 \
  -e GUIDELLM_MODEL=Qwen/Qwen3-30B-A3B \
  -e GUIDELLM_PROCESSOR=Qwen/Qwen3-30B-A3B  \
  -e GUIDELLM_RANDOM_SEED=2025  \
  -e GUIDELLM_RATE_TYPE=concurrent -e GUIDELLM_RATE=1,3,5,8 \
  -e GUIDELLM_MAX_SECONDS=300 \
  -e GUIDELLM_DATA="prompt_tokens=4096,output_tokens=512" \
  -e GUIDELLM__PREFERRED_ROUTE="chat_completions" \
  -e GUIDELLM__REQUEST_HTTP2=0 \
  ghcr.io/vllm-project/guidellm:latest -- --backend-args='{"extra_body":{"chat_template_kwargs":{"enable_thinking":false}}}'

[sjmonson]

For the container image the default CMD is benchmark run. In the example above you override benchmark run by specifying arguments. To fix this just reintroduce benchmark run:

sudo docker run \
  --rm -it \
  -v "./data/guidellm:/results:rw" \
  -e GUIDELLM_TARGET=http://localhost:8000 \
  -e GUIDELLM_MODEL=Qwen/Qwen3-30B-A3B \
  -e GUIDELLM_PROCESSOR=Qwen/Qwen3-30B-A3B  \
  -e GUIDELLM_RANDOM_SEED=2025  \
  -e GUIDELLM_RATE_TYPE=concurrent -e GUIDELLM_RATE=1,3,5,8 \
  -e GUIDELLM_MAX_SECONDS=300 \
  -e GUIDELLM_DATA="prompt_tokens=4096,output_tokens=512" \
  -e GUIDELLM__PREFERRED_ROUTE="chat_completions" \
  -e GUIDELLM__REQUEST_HTTP2=0 \
  ghcr.io/vllm-project/guidellm:latest -- benchmark run --backend-args='{"extra_body":{"chat_template_kwargs":{"enable_thinking":false}}}'

[psydok]

The above did not work. I tried passing it like this: -e GUIDELLM_BACKEND_ARGS='{"extra_body":{"chat_template_kwargs":{"enable_thinking":false}}}'. It seems to work this way for rate-type=concurrent.

Thank you very much!