Skip to content

Evaluation results do not include full metrics report #1053

@Leo777

Description

@Leo777

Describe the bug
In ADK version 0.4, the end of the execution displayed a detailed evaluation report, including execution details, expected responses, and actual responses.

After migrating to version 1.0.0, this report is no longer shown. Only the test failure status and the overall score are displayed.

To Reproduce
Steps to reproduce the behavior:

  1. Install adk version 0.4
  2. Run pytest your_test.py

Expected behavior
The full evaluation report, including expected vs. actual responses and detailed execution metrics, should be shown at the end of the execution.

Image

Actual Behavior
Only a summary with test failures and score is displayed. The detailed report is missing.

Image

Desktop (please complete the following information):

  • OS: macOS
  • Python version(python -V): 3.13.2
  • ADK version(pip show google-adk):1.1.1

Additional context
Add any other context about the problem here.

my eval test for adk 0.4.0:
`import dotenv
from google.adk.evaluation import AgentEvaluator
import pytest

@pytest.fixture(scope="session", autouse=True)
def load_env():
dotenv.load_dotenv()

def test_book_one_way():
"""Test the agent's basic ability on a few examples."""
AgentEvaluator.evaluate(
"agent_assist_adk",
str(pathlib.Path(file).parent / "data/book_one_way.test.json"),
num_runs=2,
initial_session_file=str(pathlib.Path(file).parent
/ "bob_test_state.json")
)

`

for ADK 1.1.1:

`import dotenv
from google.adk.evaluation import AgentEvaluator
import pytest

@pytest.fixture(scope="session", autouse=True)
def load_env():
dotenv.load_dotenv()

@pytest.mark.asyncio
async def test_book_one_way():
"""Test the agent's basic ability on a few examples."""
await AgentEvaluator.evaluate(
"agent_assist_adk",
str(pathlib.Path(file).parent / "data/book_one_way.test.json"),
num_runs=2,
initial_session_file=str(pathlib.Path(file).parent
/ "bob_test_state.json")
)`

Metadata

Metadata

Assignees

Labels

eval[Component] This issue is related to evaluation

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions