Hi Kash,

Row validation is only possible when rows are unique. When rows are not unique, the source/target comparison will result in apparently incorrect output. In a separate conversation you mentioned that the underlying issue was that the -tsp flag indiscriminately truncates both char and varchar fields. If the flag only truncated char fields, things would work for your customer. Can you elaborate the specific use case(s) ? Does the customer have a composite primary key where one field is a char and another is varchar ?

Thanks.

Sundar Mudupalli

Hi,

I have created a new branch issue-1368-fixed-length-pks which only trims pk columns that are fixed length characters. I did a test between Teradata and BQ with the following tables containing the same data. Teradata tables:

CREATE SET TABLE udfs.issue1368, FALLBACK (
    ID INTEGER NOT NULL,
    PR_KEY CHAR(30) NOT NULL,
    PR_VARIABLE VARCHAR(30) NOT NULL,
    MY_VALUE CHAR(12) );
CREATE SET TABLE udfs.issue1368_1, FALLBACK (
    ID INTEGER NOT NULL,
    PR_KEY VARCHAR(30) NOT NULL,
    PR_VARIABLE VARCHAR(30) NOT NULL,
    MY_VALUE CHAR(12) );

BQ table:

CREATE TABLE data_validator_issues.issue1368 (
  ID INT64,
  PR_KEY STRING(30),
  PR_VARIABLE STRING(30),
  MY_VALUE STRING
);

The following commands ran successfully.

data-validation validate row -sc teradata -tc bq -pk=pr_key -tbls=udfs.issue1368=data_validator_issues.issue1368 -hash='*'
data-validation validate row -sc teradata -tc bq -pk=pr_key -tbls=udfs.issue1368_1=data_validator_issues.issue1368 -hash='*'

Please test this version with the customer and let me know how this works out.

Sundar Mudupalli