-
Notifications
You must be signed in to change notification settings - Fork 71
Open
Description
Sometimes parsing of the LLM generated checklist fails, throws an error, and stops the entire run.
Error happened when running full morpheus23 test set during evaluation.
Error logs from the failed run were not saved.
This is hard to replicate.
Difficulty in replication also implies that the updated code in this draft PR does not fix the error (unsure if the updated code actually fixes the error, or if the error was just not replicated correctly).
WIP Status: trying to replicate the error again.
Metadata
Metadata
Assignees
Labels
No labels