-
Notifications
You must be signed in to change notification settings - Fork 111
Add functionality for export of latency logs via telemetry #608
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: telemetry
Are you sure you want to change the base?
Conversation
Signed-off-by: Sai Shree Pradhan <[email protected]>
Signed-off-by: Sai Shree Pradhan <[email protected]>
Signed-off-by: Sai Shree Pradhan <[email protected]>
Signed-off-by: Sai Shree Pradhan <[email protected]>
…r operations Signed-off-by: Sai Shree Pradhan <[email protected]>
…ze and get telemetry client Signed-off-by: Sai Shree Pradhan <[email protected]>
Signed-off-by: Sai Shree Pradhan <[email protected]>
Signed-off-by: Sai Shree Pradhan <[email protected]>
Signed-off-by: Sai Shree Pradhan <[email protected]>
Signed-off-by: Sai Shree Pradhan <[email protected]>
Signed-off-by: Sai Shree Pradhan <[email protected]>
Signed-off-by: Sai Shree Pradhan <[email protected]>
Signed-off-by: Sai Shree Pradhan <[email protected]>
Signed-off-by: Sai Shree Pradhan <[email protected]>
Signed-off-by: Sai Shree Pradhan <[email protected]>
Signed-off-by: Sai Shree Pradhan <[email protected]>
Signed-off-by: Sai Shree Pradhan <[email protected]>
Signed-off-by: Sai Shree Pradhan <[email protected]>
…pTelemetryClient class with NOOP_TELEMETRY_CLIENT singleton, updated tests accordingly Signed-off-by: Sai Shree Pradhan <[email protected]>
Signed-off-by: Sai Shree Pradhan <[email protected]>
Signed-off-by: Sai Shree Pradhan <[email protected]>
Signed-off-by: Sai Shree Pradhan <[email protected]>
Signed-off-by: Sai Shree Pradhan <[email protected]>
Signed-off-by: Sai Shree Pradhan <[email protected]>
Signed-off-by: Sai Shree Pradhan <[email protected]>
Signed-off-by: Sai Shree Pradhan <[email protected]>
Signed-off-by: Sai Shree Pradhan <[email protected]>
Signed-off-by: Sai Shree Pradhan <[email protected]>
Signed-off-by: Sai Shree Pradhan <[email protected]>
Signed-off-by: Sai Shree Pradhan <[email protected]>
Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase ( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thanks! similar to your previous PR can we add an example latency log to the description?
if hasattr(instance.thrift_backend, "_use_arrow_native_complex_types"): | ||
return ExecutionResultFormat.INLINE_ARROW | ||
|
||
return ExecutionResultFormat.FORMAT_UNSPECIFIED |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
all of these functions are ifs for Cursor/Connection/ResultSet, can we do something here like have a base class and have implementations for each class?
def log_latency(): | ||
def decorator(func): | ||
@functools.wraps(func) | ||
def wrapper(self, *args, **kwargs): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
what happens if something fails in this decorator? we should be handling that gracefully
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
both the functions: get_telemetry_client and export_latency_log have try/catch blocks in them.
Signed-off-by: Sai Shree Pradhan <[email protected]>
Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase ( |
Signed-off-by: Sai Shree Pradhan <[email protected]>
Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase ( |
Signed-off-by: Sai Shree Pradhan <[email protected]>
Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase ( |
Signed-off-by: Sai Shree Pradhan <[email protected]>
Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase ( |
Signed-off-by: Sai Shree Pradhan <[email protected]>
Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase ( |
…etry policy Signed-off-by: Sai Shree Pradhan <[email protected]>
Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase ( |
What type of PR is this?
Description
Add functionality for export of latency logs via telemetry
Example latency log:
{
"frontend_log_event_id": "30565427-1299-4fbc-bec0-25378cfe912b",
"context": {
"client_context": {
"timestamp_millis": 1750592858108,
"user_agent": "PyDatabricksSqlConnector/4.0.4"
}
},
"entry": {
"sql_driver_log": {
"session_id": "01f04f5e-b1a8-1dd1-962a-e40efd092bdd",
"system_configuration": {
"driver_version": "4.0.4",
"os_name": "Darwin",
"os_version": "24.5.0",
"os_arch": "arm64",
"runtime_name": "Python 3.13.3",
"runtime_version": "3.13.3",
"runtime_vendor": "CPython",
"driver_name": "Databricks SQL Python Connector",
"char_set_encoding": "utf-8",
"locale_name": "en_US"
},
"driver_connection_params": {
"http_path": "/sql/1.0/warehouses/864004c1b3961382",
"mode": "THRIFT",
"host_info": {
"host_url": "e2-dogfood.staging.cloud.databricks.com",
"port": 443
},
"auth_mech": "PAT"
},
"sql_statement_id": "01f04f5e-b1d6-1a1e-afe4-e99e1ccb8805",
"sql_operation": {
"statement_type": "sql",
"is_compressed": true,
"execution_result": "inline_arrow",
"retry_count": 0
},
"operation_latency_ms": 518
}
}
}
How is this tested?
Related Tickets & Documents
PECOBLR-554