Skip to content

Fix ChainOfThoughtWithHint #8382

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

Hangzhi
Copy link
Contributor

@Hangzhi Hangzhi commented Jun 16, 2025

Why

Resolve #8205. The community reported that ChainOfThoughtWithHint is broken. By manual verification, it's indeed broken with basic usage as below

class BasicQA(dspy.Signature):
    """Answer questions with short factoid answers."""
    question = dspy.InputField()
    answer = dspy.OutputField(desc="often between 1 and 5 words")

#Pass signature to ChainOfThought module
generate_answer = dspy.ChainOfThoughtWithHint(BasicQA)

# Call the predictor on a particular input alongside a hint.
question='What is the color of the sky?'
hint = "It's what you often see during a sunny day."
pred = generate_answer(question=question, hint=hint)

print(f"Question: {question}")
print(f"Predicted Answer: {pred.answer}")

What

  1. Fix the module
  2. Uncomment the unit test for the ChainOfThoughtWithHint module

Testing

  1. Manual Verification with the BasicQA above
  2. Unit test

@okhat
Copy link
Collaborator

okhat commented Jun 16, 2025

I think we should just remove this in 3.0

@Hangzhi
Copy link
Contributor Author

Hangzhi commented Jun 16, 2025

@okhat Opened another PR to clean up ChainOfThoughtWithHint #8395

@Hangzhi Hangzhi changed the title Fix ChainOfThoughtWithHint Clean up ChainOfThoughtWithHint Jun 16, 2025
@Hangzhi Hangzhi force-pushed the cot_with_hint-fix branch from 81e90a0 to 6fa332c Compare June 16, 2025 15:57
@Hangzhi Hangzhi changed the title Clean up ChainOfThoughtWithHint Fix ChainOfThoughtWithHint Jun 16, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug] ChainOfThoughtWithHint is broken (at least for Azure)
2 participants