-
Notifications
You must be signed in to change notification settings - Fork 13.8k
[FLINK-17224][table] Support precision of TIME type #26954
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
The TimeLongConvert converts the time to millisecond precision. I was wondering if we need to change this to |
@@ -266,7 +266,7 @@ private int convertToTime(JsonParser jsonNode) throws IOException { | |||
LocalTime localTime = parsedTime.query(TemporalQueries.localTime()); | |||
|
|||
// get number of milliseconds of the day | |||
return localTime.toSecondOfDay() * 1000; | |||
return (int) (localTime.toNanoOfDay() / 1000_000); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If we change the way that the json format parses times, then the output will change for existing users, which has implications to the logic of their processing. I suggest that this way of parsing be enabled by a new json format configuration option - so it can be knowingly adopted.
A similar PR exists: |
2e79596
to
8160c1e
Compare
TIME(5), | ||
DEFAULT_TIME, | ||
LocalDateTime.of(1970, 1, 1, 12, 34, 56, 123_000_000)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We are not really supporting TIME(5). As far as I understood, the time converters are storing the time as integer and not as long. We lose the precession (>3) there.
What is the purpose of the change
Add precision supports for the
TIME
function. Currently, Flink only takes precision0
and ignores any other type of precision.Brief change log
JsonToRowDataConverters.convertToTime()
fromlocalTime.toNanoOfDay() * 1000_000
tolocalTime.toNanoOfDay() / 1000_000
JsonRowDataSerDeSchemaTest.testSerDe()
to include comprehensive testing of all supported data typesVerifying this change
This change is already covered by existing tests, such as:
JsonRowDataSerDeSchemaTest.testSerDe()
- Enhanced to comprehensively test TIME type conversion for both JsonParser and non-JsonParser deserializersDateTimeException
This change added tests and can be verified as follows:
testSerDe()
method now tests the complete round-trip serialization/deserialization for TIME typesDateTimeException: Invalid value for HourOfDay
errorisJsonParser=true
andisJsonParser=false
code paths work correctlyDoes this pull request potentially affect one of the following parts:
@Public(Evolving)
: noDocumentation