Upgrade hadoop common dependencies to support compile/runtime JDK17 support.

Author: dstreev

Writerside/topics/Release-Notes.md

@@ -10,6 +10,12 @@ The latest set of enhancement requests can be found [here](https://github.com/cl
 
 If there is something you'd like to see, add a new issue [here](https://github.com/cloudera-labs/hms-mirror/issues)
 
+## 3.1.0.0
+
+**What's New**
+Upgraded `hadoop-common` to 3.4.1 for JDK17 compile-time compatibility. This avoids the need to `add-exports` for the `java.naming` module.
+
+
 ## 3.0.0.2
 
 **Bug Fixes**

Writerside/topics/troubleshooting.md

@@ -1,5 +1,25 @@
 # Troubleshooting
 
+## Error on Startup `cannot access LdapCtxFactory`
+
+This happens with `hms-mirror` 3.0.0.1-3.0.0.4 and **OracleJDK 17**.
+
+**Error**
+```
+Caused by: java.lang.IllegalStateException: java.lang.IllegalAccessError: class org.apache.hadoop.security.LdapGroupsMapping 
+  (in unnamed module @0x30ee2816) cannot access class com.sun.jndi.ldap.LdapCtxFactory (in module java.naming) 
+   because module java.naming does not export com.sun.jndi.ldap to unnamed module @0x30ee2816
+```
+
+**Solution**
+The OracleJDK 17 is pretty strict about accessing modules with older code compiled with Java 8.  The hadoop common libraries we're using where compiled with JDK 8 and are trying to access the `LdapCtxFactory` class in the `java.naming` module.
+
+Add the following environment variable before starting `hms-mirror`:
+
+```bash
+export JAVA_OPTS="--add-exports java.naming/com.sun.jndi.ldap=ALL-UNNAMED"
+````
+
 ## Application doesn't seem to be making progress
 
 All the counters for table processing aren't moving (review the hms-mirror.log) or (1.6.1.0+) the on screen logging of what tables are being added and metadata collected for has stopped.

pom.xml

@@ -22,13 +22,13 @@
     <parent>
         <groupId>com.cloudera.utils</groupId>
         <artifactId>hadoop.parent</artifactId>
-        <version>1.4.0.0</version>
+        <version>1.5.0.0</version>
         <relativePath>../cloudera-utils</relativePath>
     </parent>
 
     <groupId>com.cloudera.utils.hadoop</groupId>
     <artifactId>hms-mirror</artifactId>
-    <version>3.0.0.4</version>
+    <version>3.1.0.0</version>
     <packaging>jar</packaging>
 
     <name>hms-mirror</name>
@@ -53,7 +53,7 @@
         <commonmark.version>0.17.1</commonmark.version>
         <cdp.scope>compile</cdp.scope>
 
-        <hadoop-cli.version>3.1.0.1</hadoop-cli.version>
+        <hadoop-cli.version>3.2.0.0</hadoop-cli.version>
         <hive-sre.version>3.0.1.8</hive-sre.version>
 
     </properties>
@@ -84,7 +84,7 @@
         <dependency>
             <groupId>org.apache.hadoop</groupId>
             <artifactId>hadoop-mapreduce</artifactId>
-            <version>${hadoop.version}.${cdp.version}</version>
+            <version>${hadoop.version}</version>
             <scope>${cdp.scope}</scope>
             <type>pom</type>
             <exclusions>
@@ -113,7 +113,7 @@
         <dependency>
             <groupId>org.apache.hadoop</groupId>
             <artifactId>hadoop-distcp</artifactId>
-            <version>${hadoop.version}.${cdp.version}</version>
+            <version>${hadoop.version}</version>
             <scope>${cdp.scope}</scope>
         </dependency>
 
@@ -206,6 +206,7 @@
         <plugins>
             <plugin>
                 <artifactId>maven-jar-plugin</artifactId>
+                <version>3.4.2</version>
                 <configuration>
                     <archive>
                         <manifest>

src/main/java/com/cloudera/utils/hms/mirror/datastrategy/StorageMigrationDataStrategy.java

@@ -35,7 +35,7 @@
 import com.cloudera.utils.hms.util.TableUtils;
 import lombok.Getter;
 import lombok.extern.slf4j.Slf4j;
-import org.apache.commons.lang.StringUtils;
+import org.apache.commons.lang3.StringUtils;
 import org.springframework.stereotype.Component;
 
 import java.text.MessageFormat;

src/main/java/com/cloudera/utils/hms/mirror/domain/support/HiveDriverEnum.java

@@ -0,0 +1,4 @@
+package com.cloudera.utils.hms.mirror.domain.support;
+
+public enum HiveDriverEnum {
+}