Remove alias of sort key when PagingQueryProvider has sort key with alias and group by clause.

[pongdangx2]

Hello.

Motivation

I tried to build a batch application.

Environment

• Spring boot, Spring batch
• mysql
• kotlin

Problem

I use MySqlPagingQueryProvider like below, and Query built by this provider only works for first page query.
(Because of sort key's alias)

 provider.setSelectClause(
        """
            SELECT 
                  t1.member_id         AS member_id
                , t2.kyc_id            AS kyc_id
                , SUM(t1.balance) AS total_balance
        """.trimIndent()
        )
        provider.setFromClause(
        """
            FROM cash_history t1
             LEFT JOIN kyc t2
               ON t1.member_id = t2.member_id
        """.trimIndent()
        )
        provider.setWhereClause(
        """
            WHERE t1.balance_year_month = '202412'
                  AND kyc_id is NOT NULL
        """.trimIndent()
        )
        provider.setGroupClause("GROUP BY t1.member_id")
        provider.setSortKeys(mapOf("t1.member_id" to Order.ASCENDING))

Generated Remaining Query

SELECT *
FROM (SELECT t1.member_id         AS member_id,
             t2.kyc_id            AS kyc_id,
             SUM(t1.balance) AS total_balance
      FROM cash_history t1
               LEFT JOIN kyc t2
                         ON t1.member_id = t2.member_id
      WHERE t1.balance_year_month = '202412'
        AND kyc_id is NOT NULL
      GROUP BY t1.member_id
     ) AS MAIN_QRY
WHERE ((MAIN_QRY.member_id > ?))
ORDER BY t1.member_id ASC
LIMIT 100

This generated remaining query has order by clause with t1.memeber, and valid scope of t1 alias is in MAIN_QRY.

Solution

• Use getSortKeysWithoutAliases method for building order by clause after MAIN_QRY appended.

public static String generateLimitGroupedSqlQuery(AbstractSqlPagingQueryProvider provider, String limitClause) {
		StringBuilder sql = new StringBuilder();
		sql.append("SELECT * ");
		sql.append(" FROM (");
		sql.append("SELECT ").append(provider.getSelectClause());
		sql.append(" FROM ").append(provider.getFromClause());
		sql.append(provider.getWhereClause() == null ? "" : " WHERE " + provider.getWhereClause());
		buildGroupByClause(provider, sql);
		sql.append(") AS MAIN_QRY ");
		sql.append("WHERE ");
		buildSortConditions(provider, sql);
		sql.append(" ORDER BY ").append(buildSortClause(provider.getSortKeysWithoutAliases()));
		sql.append(" ").append(limitClause);

		return sql.toString();
	}

[fmbenhassine]

Can you please edit your initial comment and explain the problem this PR addresses. I know you linked the previous PR in which the problem is explained, but since we might include this PR in the change log it would be better to have the problem explained here.

Moreover, is this specific to MySQL only?

[pongdangx2]

@fmbenhassine Thank you for the comment.
I edit the comment as you requested.

Using a table alias from a subquery in an outer query is not compliant with the ANSI SQL standard. Given this, it is likely that all DBMSs providing an implementation of PagingQueryProvider in Spring Batch follow this behavior consistently.

Since I'm not familiar with other DBMSs, I only created a test for MysqlPagingQueryProvider.

[pongdangx2]

@fmbenhassine Hello!!

I add some tests for MariaDB, PostgreSQL and SQLite.

I executed query on those DBMSs running in Docker.

I found implementations of PagingQueryProvider that call the method I modified: Db2PagingQueryProvider, DerbyPagingQueryProvider, HanaPagingQueryProvider, MariaDBPagingQueryProvider, MySqlPagingQueryProvider, PostgresPagingQueryProvider, and SqlitePagingQueryProvider. Among these, I have directly created tables and executed queries in all except Db2, Derby, and Hana