ChemPile scripts

This repository stores scripts for ChemPile:

1. token_counter_cli.py -- script used to count tokens across the ChemPile subset.
2. Submodule ddp_trainer:
   • Contains LoRA adapter merging scripts
   • ChemBench evaluation scripts
3. Folder chempile-instruct contains the scripts used to generate the instruction dataset.
4. review-app folder contains resources to reproduce the review app.