Skip to content

Checklist parsing error #136

@katherineh123

Description

@katherineh123

Sometimes parsing of the LLM generated checklist fails, throws an error, and stops the entire run.
Error happened when running full morpheus23 test set during evaluation.
Error logs from the failed run were not saved.
This is hard to replicate.

Difficulty in replication also implies that the updated code in this draft PR does not fix the error (unsure if the updated code actually fixes the error, or if the error was just not replicated correctly).

WIP Status: trying to replicate the error again.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions