HBASE-28389 HBase backup yarn queue parameter ignored #6130

2005hithlj · 2024-07-30T04:32:57Z

https://issues.apache.org/jira/browse/HBASE-28389

2005hithlj · 2024-07-30T04:36:03Z

@DieterDP-ng @Apache9 Could you take a look? Thanks.

Apache-HBase · 2024-07-30T05:04:05Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 48s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	hbaseanti	0m 0s		Patch does not have any anti-patterns.
			_ branch-2 Compile Tests _
+1 💚	mvninstall	3m 5s		branch-2 passed
+1 💚	compile	0m 30s		branch-2 passed
+1 💚	checkstyle	0m 12s		branch-2 passed
+1 💚	spotbugs	0m 34s		branch-2 passed
+1 💚	spotless	0m 46s		branch has no errors when running spotless:check.
			_ Patch Compile Tests _
+1 💚	mvninstall	2m 58s		the patch passed
+1 💚	compile	0m 27s		the patch passed
+1 💚	javac	0m 27s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 12s		the patch passed
+1 💚	spotbugs	0m 40s		the patch passed
+1 💚	hadoopcheck	15m 39s		Patch does not cause any errors with Hadoop 2.10.2 or 3.3.6 3.4.0.
+1 💚	spotless	0m 44s		patch has no errors when running spotless:check.
			_ Other Tests _
+1 💚	asflicense	0m 11s		The patch does not generate ASF License warnings.
		28m 30s

Subsystem	Report/Notes
Docker	ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6130/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#6130
Optional Tests	dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless
uname	Linux be2d5f8823d2 5.4.0-186-generic #206-Ubuntu SMP Fri Apr 26 12:31:10 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2 / `a5cf897`
Default Java	Eclipse Adoptium-11.0.23+9
Max. process+thread count	83 (vs. ulimit of 30000)
modules	C: hbase-backup U: hbase-backup
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6130/1/console
versions	git=2.34.1 maven=3.9.8 spotbugs=4.7.3
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-07-30T05:05:35Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 42s		Docker mode activated.
-0 ⚠️	yetus	0m 6s		Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
			_ Prechecks _
			_ branch-2 Compile Tests _
+1 💚	mvninstall	2m 38s		branch-2 passed
+1 💚	compile	0m 19s		branch-2 passed
+1 💚	javadoc	0m 15s		branch-2 passed
+1 💚	shadedjars	4m 59s		branch has no errors when building our shaded downstream artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	2m 30s		the patch passed
+1 💚	compile	0m 18s		the patch passed
+1 💚	javac	0m 18s		the patch passed
+1 💚	javadoc	0m 14s		the patch passed
+1 💚	shadedjars	5m 1s		patch has no errors when building our shaded downstream artifacts.
			_ Other Tests _
+1 💚	unit	11m 55s		hbase-backup in the patch passed.
		30m 2s

Subsystem	Report/Notes
Docker	ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6130/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
GITHUB PR	#6130
Optional Tests	javac javadoc unit compile shadedjars
uname	Linux d6bfaca083af 5.4.0-182-generic #202-Ubuntu SMP Fri Apr 26 12:29:36 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2 / `a5cf897`
Default Java	Temurin-1.8.0_412-b08
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6130/1/testReport/
Max. process+thread count	3410 (vs. ulimit of 30000)
modules	C: hbase-backup U: hbase-backup
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6130/1/console
versions	git=2.34.1 maven=3.9.8
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-07-30T05:06:07Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 20s		Docker mode activated.
-0 ⚠️	yetus	0m 5s		Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
			_ Prechecks _
			_ branch-2 Compile Tests _
+1 💚	mvninstall	3m 8s		branch-2 passed
+1 💚	compile	0m 22s		branch-2 passed
+1 💚	javadoc	0m 17s		branch-2 passed
+1 💚	shadedjars	5m 30s		branch has no errors when building our shaded downstream artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	2m 58s		the patch passed
+1 💚	compile	0m 21s		the patch passed
+1 💚	javac	0m 21s		the patch passed
+1 💚	javadoc	0m 15s		the patch passed
+1 💚	shadedjars	5m 36s		patch has no errors when building our shaded downstream artifacts.
			_ Other Tests _
+1 💚	unit	10m 40s		hbase-backup in the patch passed.
		30m 35s

Subsystem	Report/Notes
Docker	ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6130/1/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR	#6130
Optional Tests	javac javadoc unit compile shadedjars
uname	Linux e4c3fd1a7bb8 5.4.0-182-generic #202-Ubuntu SMP Fri Apr 26 12:29:36 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2 / `a5cf897`
Default Java	Eclipse Adoptium-17.0.11+9
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6130/1/testReport/
Max. process+thread count	3182 (vs. ulimit of 30000)
modules	C: hbase-backup U: hbase-backup
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6130/1/console
versions	git=2.34.1 maven=3.9.8
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-07-30T05:08:01Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 43s		Docker mode activated.
-0 ⚠️	yetus	0m 5s		Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
			_ Prechecks _
			_ branch-2 Compile Tests _
+1 💚	mvninstall	3m 10s		branch-2 passed
+1 💚	compile	0m 18s		branch-2 passed
+1 💚	javadoc	0m 14s		branch-2 passed
+1 💚	shadedjars	5m 56s		branch has no errors when building our shaded downstream artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	2m 57s		the patch passed
+1 💚	compile	0m 19s		the patch passed
+1 💚	javac	0m 19s		the patch passed
+1 💚	javadoc	0m 14s		the patch passed
+1 💚	shadedjars	5m 58s		patch has no errors when building our shaded downstream artifacts.
			_ Other Tests _
+1 💚	unit	11m 27s		hbase-backup in the patch passed.
		32m 26s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6130/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#6130
Optional Tests	javac javadoc unit compile shadedjars
uname	Linux 7f39f84c6acc 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2 / `a5cf897`
Default Java	Eclipse Adoptium-11.0.23+9
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6130/1/testReport/
Max. process+thread count	3710 (vs. ulimit of 30000)
modules	C: hbase-backup U: hbase-backup
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6130/1/console
versions	git=2.34.1 maven=3.9.8
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

DieterDP-ng

Looks good, thanks for solving this.

DieterDP-ng · 2024-07-30T08:49:17Z

hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/impl/BackupCommands.java


      boolean ignoreChecksum = cmdline.hasOption(OPTION_IGNORECHECKSUM);

      if (cmdline.hasOption(OPTION_YARN_QUEUE_NAME)) {


I suggest moving this block to before the super.execute(); that creates the connection. That way, you can directly update the conf using getConf(). Won't have any change in outcome, but feels a bit simpler.

@DieterDP-ng Thanks for your review. I have revised it. could you take a look? Thanks.

Looks good, no further remarks from me.

Apache-HBase · 2024-07-30T09:49:07Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 42s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	hbaseanti	0m 0s		Patch does not have any anti-patterns.
			_ branch-2 Compile Tests _
+1 💚	mvninstall	3m 9s		branch-2 passed
+1 💚	compile	0m 30s		branch-2 passed
+1 💚	checkstyle	0m 13s		branch-2 passed
+1 💚	spotbugs	0m 34s		branch-2 passed
+1 💚	spotless	0m 47s		branch has no errors when running spotless:check.
			_ Patch Compile Tests _
+1 💚	mvninstall	2m 57s		the patch passed
+1 💚	compile	0m 27s		the patch passed
+1 💚	javac	0m 27s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 11s		the patch passed
+1 💚	spotbugs	0m 39s		the patch passed
+1 💚	hadoopcheck	15m 37s		Patch does not cause any errors with Hadoop 2.10.2 or 3.3.6 3.4.0.
+1 💚	spotless	0m 45s		patch has no errors when running spotless:check.
			_ Other Tests _
+1 💚	asflicense	0m 11s		The patch does not generate ASF License warnings.
		28m 24s

Subsystem	Report/Notes
Docker	ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6130/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#6130
Optional Tests	dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless
uname	Linux 732c11e0be32 5.4.0-186-generic #206-Ubuntu SMP Fri Apr 26 12:31:10 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2 / `3bbd7b0`
Default Java	Eclipse Adoptium-11.0.23+9
Max. process+thread count	78 (vs. ulimit of 30000)
modules	C: hbase-backup U: hbase-backup
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6130/2/console
versions	git=2.34.1 maven=3.9.8 spotbugs=4.7.3
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-07-30T09:50:15Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 46s		Docker mode activated.
-0 ⚠️	yetus	0m 5s		Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
			_ Prechecks _
			_ branch-2 Compile Tests _
+1 💚	mvninstall	2m 25s		branch-2 passed
+1 💚	compile	0m 18s		branch-2 passed
+1 💚	javadoc	0m 14s		branch-2 passed
+1 💚	shadedjars	5m 1s		branch has no errors when building our shaded downstream artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	2m 26s		the patch passed
+1 💚	compile	0m 19s		the patch passed
+1 💚	javac	0m 19s		the patch passed
+1 💚	javadoc	0m 14s		the patch passed
+1 💚	shadedjars	4m 57s		patch has no errors when building our shaded downstream artifacts.
			_ Other Tests _
+1 💚	unit	11m 44s		hbase-backup in the patch passed.
		29m 36s

Subsystem	Report/Notes
Docker	ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6130/2/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
GITHUB PR	#6130
Optional Tests	javac javadoc unit compile shadedjars
uname	Linux 4e08f065b2b5 5.4.0-182-generic #202-Ubuntu SMP Fri Apr 26 12:29:36 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2 / `3bbd7b0`
Default Java	Temurin-1.8.0_412-b08
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6130/2/testReport/
Max. process+thread count	2975 (vs. ulimit of 30000)
modules	C: hbase-backup U: hbase-backup
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6130/2/console
versions	git=2.34.1 maven=3.9.8
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-07-30T09:52:34Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 42s		Docker mode activated.
-0 ⚠️	yetus	0m 5s		Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
			_ Prechecks _
			_ branch-2 Compile Tests _
+1 💚	mvninstall	2m 54s		branch-2 passed
+1 💚	compile	0m 18s		branch-2 passed
+1 💚	javadoc	0m 14s		branch-2 passed
+1 💚	shadedjars	5m 53s		branch has no errors when building our shaded downstream artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	2m 55s		the patch passed
+1 💚	compile	0m 18s		the patch passed
+1 💚	javac	0m 18s		the patch passed
+1 💚	javadoc	0m 13s		the patch passed
+1 💚	shadedjars	5m 56s		patch has no errors when building our shaded downstream artifacts.
			_ Other Tests _
+1 💚	unit	11m 22s		hbase-backup in the patch passed.
		31m 58s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6130/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#6130
Optional Tests	javac javadoc unit compile shadedjars
uname	Linux 983f332cb4a7 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2 / `3bbd7b0`
Default Java	Eclipse Adoptium-11.0.23+9
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6130/2/testReport/
Max. process+thread count	3821 (vs. ulimit of 30000)
modules	C: hbase-backup U: hbase-backup
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6130/2/console
versions	git=2.34.1 maven=3.9.8
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-07-30T09:59:21Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 20s		Docker mode activated.
-0 ⚠️	yetus	0m 5s		Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
			_ Prechecks _
			_ branch-2 Compile Tests _
+1 💚	mvninstall	3m 22s		branch-2 passed
+1 💚	compile	0m 22s		branch-2 passed
+1 💚	javadoc	0m 16s		branch-2 passed
+1 💚	shadedjars	7m 3s		branch has no errors when building our shaded downstream artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	4m 20s		the patch passed
+1 💚	compile	0m 30s		the patch passed
+1 💚	javac	0m 30s		the patch passed
+1 💚	javadoc	0m 21s		the patch passed
+1 💚	shadedjars	7m 14s		patch has no errors when building our shaded downstream artifacts.
			_ Other Tests _
+1 💚	unit	13m 43s		hbase-backup in the patch passed.
		38m 42s

Subsystem	Report/Notes
Docker	ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6130/2/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR	#6130
Optional Tests	javac javadoc unit compile shadedjars
uname	Linux 24d0a658c229 5.4.0-182-generic #202-Ubuntu SMP Fri Apr 26 12:29:36 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2 / `3bbd7b0`
Default Java	Eclipse Adoptium-17.0.11+9
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6130/2/testReport/
Max. process+thread count	3185 (vs. ulimit of 30000)
modules	C: hbase-backup U: hbase-backup
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6130/2/console
versions	git=2.34.1 maven=3.9.8
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Signed-off-by: Duo Zhang <[email protected]> Reviewed-by: DieterDP <[email protected]> (cherry picked from commit 68045db)

DieterDP-ng approved these changes Jul 30, 2024

View reviewed changes

HBASE-28389 HBase backup yarn queue parameter ignored

3bbd7b0

2005hithlj force-pushed the HBASE-28389-branch-2 branch from a5cf897 to 3bbd7b0 Compare July 30, 2024 09:13

Apache9 approved these changes Jul 30, 2024

View reviewed changes

2005hithlj merged commit 68045db into apache:branch-2 Jul 31, 2024

2005hithlj added a commit that referenced this pull request Jul 31, 2024

HBASE-28389 HBase backup yarn queue parameter ignored (#6130)

2747320

Signed-off-by: Duo Zhang <[email protected]> Reviewed-by: DieterDP <[email protected]> (cherry picked from commit 68045db)


		boolean ignoreChecksum = cmdline.hasOption(OPTION_IGNORECHECKSUM);

		if (cmdline.hasOption(OPTION_YARN_QUEUE_NAME)) {

HBASE-28389 HBase backup yarn queue parameter ignored #6130

HBASE-28389 HBase backup yarn queue parameter ignored #6130

Uh oh!

Conversation

2005hithlj commented Jul 30, 2024

Uh oh!

2005hithlj commented Jul 30, 2024

Uh oh!

Apache-HBase commented Jul 30, 2024

Uh oh!

Apache-HBase commented Jul 30, 2024

Uh oh!

Apache-HBase commented Jul 30, 2024

Uh oh!

Apache-HBase commented Jul 30, 2024

Uh oh!

DieterDP-ng left a comment

Choose a reason for hiding this comment

Uh oh!

DieterDP-ng Jul 30, 2024

Choose a reason for hiding this comment

Uh oh!

2005hithlj Jul 30, 2024

Choose a reason for hiding this comment

Uh oh!

DieterDP-ng Jul 30, 2024

Choose a reason for hiding this comment

Uh oh!

Apache-HBase commented Jul 30, 2024

Uh oh!

Apache-HBase commented Jul 30, 2024

Uh oh!

Apache-HBase commented Jul 30, 2024

Uh oh!

Apache-HBase commented Jul 30, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants