Skip to content

Conversation

@abstractdog
Copy link
Contributor

No description provided.

@abstractdog
Copy link
Contributor Author

abstractdog commented Apr 2, 2021

@jteagles: could you please take a look? the patch solves the problem with automatic java version detection
it assumes that the tez client runs on the same java version as AM/tasks (because the resolution of the launch opts is done there), I think this is a reasonable assumption.

btw: I used earlier experience from HIVE-23938

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 33s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-1 ❌ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ master Compile Tests _
+1 💚 mvninstall 13m 18s master passed
+1 💚 compile 0m 34s master passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚 compile 0m 33s master passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚 checkstyle 1m 11s master passed
+1 💚 javadoc 0m 45s master passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚 javadoc 0m 36s master passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+0 🆗 spotbugs 1m 32s Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚 findbugs 1m 30s master passed
_ Patch Compile Tests _
+1 💚 mvninstall 0m 20s the patch passed
+1 💚 compile 0m 20s the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚 javac 0m 20s the patch passed
+1 💚 compile 0m 19s the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚 javac 0m 20s the patch passed
+1 💚 checkstyle 0m 13s the patch passed
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 javadoc 0m 21s the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚 javadoc 0m 22s the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚 findbugs 0m 54s the patch passed
_ Other Tests _
+1 💚 unit 1m 55s tez-api in the patch passed.
+1 💚 asflicense 0m 14s The patch does not generate ASF License warnings.
24m 54s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-116/2/artifact/out/Dockerfile
GITHUB PR #116
JIRA Issue TEZ-4299
Optional Tests dupname asflicense javac javadoc unit spotbugs findbugs checkstyle compile
uname Linux f2a627bf7a66 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/tez.sh
git revision master / 2dcbe0b
Default Java Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Test Results https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-116/2/testReport/
Max. process+thread count 401 (vs. ulimit of 5500)
modules C: tez-api U: tez-api
Console output https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-116/2/console
versions git=2.25.1 maven=3.6.3 findbugs=3.0.1
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

public static final String TEZ_AM_LAUNCH_CMD_OPTS = TEZ_AM_PREFIX + "launch.cmd-opts";
public static final String TEZ_AM_LAUNCH_CMD_OPTS_DEFAULT =
"-XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC";
public static final String TEZ_AM_LAUNCH_CMD_OPTS_JDK9_DEFAULT =
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we create TEZ_AM_LAUNCH_CMD_OPTS_JDK8_DEFAULT with the origin value of TEZ_AM_LAUNCH_CMD_OPTS_DEFAULT.
Then we could statically assign TEZ_AM_LAUNCH_CMD_OPTS_DEFAULT to correct value of JDK8 or JDK9+. This would remove the runtime detection.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

okay, I'm doing it in the next commit

}

private static int getJavaVersion() {
String javaVersionString = System.getProperty("java.version");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I originally detected the presence of java.lang.runtime.Version class to avoid parsing the version string. Knowing the actual version could be more useful in the future though so let's try to keep this for now.

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 12m 53s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 1s No case conflicting files found.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-1 ❌ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ master Compile Tests _
+1 💚 mvninstall 13m 23s master passed
+1 💚 compile 0m 35s master passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚 compile 0m 34s master passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚 checkstyle 1m 12s master passed
+1 💚 javadoc 0m 45s master passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚 javadoc 0m 36s master passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+0 🆗 spotbugs 1m 28s Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚 findbugs 1m 25s master passed
_ Patch Compile Tests _
+1 💚 mvninstall 0m 19s the patch passed
+1 💚 compile 0m 22s the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚 javac 0m 22s the patch passed
+1 💚 compile 0m 19s the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚 javac 0m 19s the patch passed
+1 💚 checkstyle 0m 13s the patch passed
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 javadoc 0m 22s the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚 javadoc 0m 22s the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚 findbugs 0m 56s the patch passed
_ Other Tests _
+1 💚 unit 1m 56s tez-api in the patch passed.
+1 💚 asflicense 0m 15s The patch does not generate ASF License warnings.
37m 22s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-116/3/artifact/out/Dockerfile
GITHUB PR #116
JIRA Issue TEZ-4299
Optional Tests dupname asflicense javac javadoc unit spotbugs findbugs checkstyle compile
uname Linux ca86ebb4b3d1 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/tez.sh
git revision master / 2dcbe0b
Default Java Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Test Results https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-116/3/testReport/
Max. process+thread count 327 (vs. ulimit of 5500)
modules C: tez-api U: tez-api
Console output https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-116/3/console
versions git=2.25.1 maven=3.6.3 findbugs=3.0.1
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@abstractdog
Copy link
Contributor Author

@jteagles: are you fine with merging this?

Copy link
Contributor

@jteagles jteagles left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1. Let's put this in master and branch-0.9

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants