Add hipho physics dataset #1318
Open
+2,078
−1
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Remove custom LogBuffer class and thread-safe logging - Replace safe_print with standard print statements - Remove threading and datetime imports - Simplify build_prompt function by removing verbose debug output - Update dataset URL from haiyuanwan/HiPhO to HY-Wan/HiPhO - Reduce code from 899 to 803 lines (10.7% reduction) - Maintain all core functionality: evaluation logic, prompt building, hipho_verifier integration
- Remove complex parallel evaluation using track_progress_rich - Simplify to sequential evaluation for better stability and debugging - Remove multiprocessing and parallel task management dependencies - Rename functions to remove '_with_buffer' suffix and log_buffer parameters - Remove nproc parameter handling and temporary file management - Reduce code from 803 to 774 lines (additional 3.6% reduction) - Maintain all core evaluation logic: fine/coarse-grained scoring, hipho_verifier integration - Sequential evaluation is sufficient for physics olympiad problem counts
Major improvements: - Remove 6 unnecessary try-except blocks that were hiding errors - Standardize judge model initialization to follow VLMEvalKit conventions - Move all prompt templates to utils/prompt_inference.py for better organization - Remove redundant count statistics (fine_grained_count, coarse_grained_count, total_count) - Remove unused fallback functions (_simple_answer_matching, _extract_prediction_for_display) - Fix multi-image base64 processing bug - Correct dataset name display in summary output - Remove verbose debugging output and unnecessary comments Code reduction: 899 → 604 lines (32.8% reduction) Eliminated potential bugs and improved maintainability while preserving all core functionality
…iguration - Translate all Chinese comments to English in hipho.py, hipho_verifier.py, and prompt_inference.py - Simplify comments while maintaining technical accuracy - Replace hardcoded verifier model configuration with environment variables - Use VLMEvalKit standard environment variable approach for better flexibility - Add support for HIPHO_VERIFIER_* environment variables for model configuration - Improve code maintainability and international accessibility
- Add datasets: for HuggingFace dataset loading - Add scikit-learn: for machine learning utilities - Add pylatexenc==2.10: for LaTeX text processing - Add math-verify: for mathematical answer verification These dependencies are required for the HiPhO physics olympiad dataset evaluation and verification functionality.
Collaborator
|
Hi @MarkovChain-why, it seems this pull request is duplicate with #1293. Can I close the previous one? |
mzr1996
reviewed
Nov 20, 2025
mzr1996
approved these changes
Nov 21, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
update some requirements and refactor the code