README: Pin exact transformers for LLaVA-Next to avoid “Got USER: <image>” error #1294
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Refs DrishtiShrrrma#2
Summary
This PR updates the Transformers Version Recommendation so that LLaVA-Next models don’t default to “latest”. We pin to tested, working versions. This prevents a reproducible crash where the image processor receives the literal
placeholder.
What changed
transformers==latestfor LLaVA-Next” with:transformers==4.48.0(recommended) ortransformers==4.46.0for LLaVA-Next series (e.g.,llava-hf/llava-v1.6-vicuna-7b-hf).transformers==latest” for the other model families listed in the README.Why
Newer
transformersbuilds change image handling and, in our testing, cause LLaVA-Next evaluation to pass the literal placeholder to the processor, leading to:Observations (tested)
transformersValueError: Incorrect image source. Must be a valid URL starting withhttp://orhttps://, a valid path to an image file, or a base64 encoded string. Got USER: <image>ValueError: Incorrect image source. Must be a valid URL starting withhttp://orhttps://, a valid path to an image file, or a base64 encoded string. Got USER: <image>Benchmarks tested
CountBenchQA,MMBench_DEV_EN,MME,SEEDBench_IMGModel tested
llava-hf/llava-v1.6-vicuna-7b-hf(key:llava_next_vicuna_7b)Minimal repro
Environment