[RAPTOR-13895] Implement inline predictor based on DRUM score

[klichukb]

Rationale

Agentic workflow requires running DRUM with the target custom model code in-place in a codespace session in order to test the model inline end to end.
Currently this is done using "drum server", and communicating over network. While it works, its quite a heavy and slow test, which involves processes, threads and working with stdin/stdout piping to get feedback.

The idea is to use the exiting "drum score" functionality which uses an alternative predictor, that does not spin up a web server, exposing the predictor itself to work with directly.

Provided:
input.json

{
  "model": "gpt-4o-mini",
  "messages": [
    { "role": "system", "content": "You are a helpful assistant." },
    { "role": "user",   "content": "Tell me a joke about penguins." }
  ],
  "temperature": 0.7,
  "top_p": 1.0,
  "max_tokens": 256,
  "n": 1,
  "stream": false,
  "stop": null
}

And the following example

import json

payload = json.loads(open("input.json", "r").read())
code_dir = (
    '/datarobot-user-models/model_templates/python3_dummy_chat'
)

with drum_inline_predictor(target_type=TargetType.AGENTIC_WORKFLOW.value, custom_model_dir=code_dir,
                           target_name='response') as predictor:
    result = predictor.chat(payload)
    print(result)

[yakov-g]

Very nice.
Maybe implement that inline.py as a test

[nullspoon]

Might be a python thing I don't know about (and this probably doesn't really matter that much), but should we send this to stderr?

[nullspoon]

Oh nevermind. This just moves the code up.

[klichukb]

Yeah I just lift-and-shifted it, did not want to alter but it should use logger.exception() instead of this.

[mjnitz02]

@klichukb So I ran a final test on this now after fixing around agents to support this or drum server based on a flag. It worked great for all agents across all testing vectors. Let me know if I can help add stuff to get this PR finished. Thanks!

[engprod-2]

The Needs Review labels were added based on the following file changes.

Team @datarobot/core-modeling (#predictive-ai) was assigned because of changes in files:

custom_model_runner/datarobot_drum/drum/common.py
custom_model_runner/datarobot_drum/drum/drum.py
custom_model_runner/datarobot_drum/drum/main.py
custom_model_runner/datarobot_drum/drum/root_predictors/drum_inline_utils.py
custom_model_runner/datarobot_drum/drum/root_predictors/generic_predictor.py
tests/functional/test_drum_inline_utils.py

Team @datarobot/genai-systems (#genai-systems) was assigned because of changes in files:

custom_model_runner/datarobot_drum/drum/common.py
custom_model_runner/datarobot_drum/drum/drum.py
custom_model_runner/datarobot_drum/drum/main.py
custom_model_runner/datarobot_drum/drum/root_predictors/drum_inline_utils.py
custom_model_runner/datarobot_drum/drum/root_predictors/generic_predictor.py
requirements_test.txt
tests/fixtures/python3_dummy_chat/README.md
tests/fixtures/python3_dummy_chat/custom.py
tests/fixtures/python3_dummy_chat/moderation_config.yaml
tests/functional/test_drum_inline_utils.py

If you think that there are some issues with ownership, please discuss with C&A domain at #sdtk slack channel and create PR to update DRCODEOWNERS\CODEOWNERS file.

[yakov-g]

Looks great!

[klichukb]

@yakov-g @mjnitz02 I tried an e2e test with moderations to make sure that the inline runner executes the moderations pipeline. Its crazy because it triples the duration of tests even with bumping 8G to 16G for the functional test suite. Installing moderations is adding more than 30 minutes, I'm not sure why this is so bad.

I'm skipping the test to get things in for the release, but I think we have to figure out how to get the test in.

[klichukb]

@datarobot/core-modeling folks, I'd need to get in for the branch cut for the agentic work, feel free to provide your review, we can address it.