Skip to content

Conversation

@idirze
Copy link
Collaborator

@idirze idirze commented Nov 17, 2025

This PR fixes issues with PostgreSQL, adds AWS connector on the Hadoop classpath and reduces the container size by removing unnecessary Hadoop components.

  1. Fix PostgreSQL support and add ability to pass the username as a secret when needed:
  • Ensures schematool correctly uses -dbType postgres
  • Allows (optionally) injecting the DB username from a Kubernetes Secret for better security and flexibility
  1. Add AWS connectors on the Hadoop classpath
  • Use the bundles provided in ${HADOOP_HOME}/share/hadoop/tools/lib/{hadoop-aws-*.jar, aws-java-sdk-bundle-*.jar}
  1. Optimize base image size by removing unnecessary Hadoop components:
  • Removes unused YARN, MapReduce, client libraries, tests, examples, docs, and webapps
  • Keeps only what is necessary for the Hive Metastore to operate

@idirze idirze changed the title Fix PostgreSQL Support and Optimize Image Size Fix PostgreSQL Support, Add AWS connectors on the Hadoop classpath and Optimize Image Size Nov 17, 2025
@idirze idirze requested review from anamhn and mlahouar November 17, 2025 09:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants