Abstract

BioMedAgent is an autonomous, multi-agent LLM system that analyzes biomedical images, scientific papers, and metadata to generate hypotheses, design experiments, and provide clinical insights. It integrates multi-modal reasoning (text + image + table) and agentic collaboration to assist researchers and clinicians in discovering new patterns or interpreting results.

installiation

packages for deep learning

1. conda create -p /home/cli74/Desktop/cbi/public/chenli/envs/chatAI python=3.12
2. pip3 install torch torchvision --index-url https://download.pytorch.org/whl/cu126
3. conda install numpy pandas scikit-learn matplotlib tqdm pillow h5py -y
4. conda install mpi4py -y
5. pip install deepspeed accelerate torchmetrics

packages for LLM and Rag

6. pip install transformers datasets faiss-gpu sentence-transformers langchain openai
7. pip install tiktoken accelerate bitsandbytes
8. pip install pydicom opencv-python scikit-image scikit-learn nibabel imageio
9. pip install unstructured pypdf
10. pip install -U langchain-community==0.3.27 langchain-text-splitters==0.3.11 langchain-huggingface==0.3.0 einops
11. module load cuda12.6/toolkit/12.6.3

AI scientist agent

Device CPU / MPS CUDA (A100/H100)

embedding_model = "biobert-base-uncased"

llm_model = "meta-llama/Llama-3-8B-Instruct"

optionally switch FAISS to the GPU version (faiss-gpu)

Clinical assistant agent