Skip to content

HDFS-17710. Don't add journal edits to cache until after persisted to storage #7296

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 1 commit into from
Closed

HDFS-17710. Don't add journal edits to cache until after persisted to storage #7296

wants to merge 1 commit into from
+32 −3

Conversation

@Kimahriman
Copy link
Contributor

@Kimahriman Kimahriman commented Jan 18, 2025

Description of PR

Update the process of JournalNodes journaling new edits to persist edits to disk before adding them to the edit cache. This prevents standby and observer nodes from accidentally reading transactions from the edit cache when tailing logs that have not actually been durably persisted, making the standby or observer have in invalid state thinking the latest committed transaction is higher than it should be.

How was this patch tested?

New UT using a new fault injector method to simulate a disk write failure.

For code changes:

  • Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
  • Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
  • If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
  • If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

@Kimahriman
Copy link
Contributor Author

Kimahriman commented Jan 18, 2025

@ZanderXu @aswinmprabhu since you were most recently working with the Journal code.

@hadoop-yetus
Copy link

hadoop-yetus commented Jan 18, 2025

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 44s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 1s codespell was not available.
+0 🆗 detsecrets 0m 1s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 1 new or modified test files.
_ trunk Compile Tests _
+1 💚 mvninstall 34m 48s trunk passed
+1 💚 compile 1m 18s trunk passed with JDK Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04
+1 💚 compile 1m 9s trunk passed with JDK Private Build-1.8.0_432-8u432-gaus1-0ubuntu220.04-ga
+1 💚 checkstyle 1m 10s trunk passed
+1 💚 mvnsite 1m 19s trunk passed
+1 💚 javadoc 1m 10s trunk passed with JDK Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04
+1 💚 javadoc 1m 39s trunk passed with JDK Private Build-1.8.0_432-8u432-gaus1-0ubuntu220.04-ga
+1 💚 spotbugs 3m 9s trunk passed
+1 💚 shadedclient 34m 48s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 1m 3s the patch passed
+1 💚 compile 1m 9s the patch passed with JDK Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04
+1 💚 javac 1m 9s the patch passed
+1 💚 compile 1m 0s the patch passed with JDK Private Build-1.8.0_432-8u432-gaus1-0ubuntu220.04-ga
+1 💚 javac 1m 0s the patch passed
+1 💚 blanks 0m 1s The patch has no blanks issues.
-0 ⚠️ checkstyle 0m 57s /results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt hadoop-hdfs-project/hadoop-hdfs: The patch generated 2 new + 5 unchanged - 0 fixed = 7 total (was 5)
+1 💚 mvnsite 1m 8s the patch passed
+1 💚 javadoc 0m 57s the patch passed with JDK Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04
+1 💚 javadoc 1m 32s the patch passed with JDK Private Build-1.8.0_432-8u432-gaus1-0ubuntu220.04-ga
+1 💚 spotbugs 3m 3s the patch passed
+1 💚 shadedclient 34m 33s patch has no errors when building and testing our client artifacts.
_ Other Tests _
-1 ❌ unit 224m 39s /patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt hadoop-hdfs in the patch passed.
+1 💚 asflicense 0m 46s The patch does not generate ASF License warnings.
350m 33s
Reason Tests
Failed junit tests hadoop.hdfs.TestDecommission
Subsystem Report/Notes
Docker ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7296/1/artifact/out/Dockerfile
GITHUB PR #7296
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux ddef7e2929aa 5.15.0-130-generic #140-Ubuntu SMP Wed Dec 18 17:59:53 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 4604794
Default Java Private Build-1.8.0_432-8u432-gaus1-0ubuntu220.04-ga
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_432-8u432-gaus1-0ubuntu220.04-ga
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7296/1/testReport/
Max. process+thread count 3983 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7296/1/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@Kimahriman Kimahriman force-pushed the journal-cache-edit-after-persist branch from 4604794 to e609064 Compare January 27, 2025 01:52
@hadoop-yetus
Copy link

hadoop-yetus commented Jan 27, 2025

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 17m 42s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 1 new or modified test files.
_ trunk Compile Tests _
+1 💚 mvninstall 39m 7s trunk passed
+1 💚 compile 1m 20s trunk passed with JDK Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04
+1 💚 compile 1m 10s trunk passed with JDK Private Build-1.8.0_432-8u432-gaus1-0ubuntu220.04-ga
+1 💚 checkstyle 1m 11s trunk passed
+1 💚 mvnsite 1m 18s trunk passed
+1 💚 javadoc 1m 12s trunk passed with JDK Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04
+1 💚 javadoc 1m 35s trunk passed with JDK Private Build-1.8.0_432-8u432-gaus1-0ubuntu220.04-ga
+1 💚 spotbugs 3m 8s trunk passed
+1 💚 shadedclient 35m 57s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 1m 5s the patch passed
+1 💚 compile 1m 12s the patch passed with JDK Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04
+1 💚 javac 1m 12s the patch passed
+1 💚 compile 1m 1s the patch passed with JDK Private Build-1.8.0_432-8u432-gaus1-0ubuntu220.04-ga
+1 💚 javac 1m 1s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
-0 ⚠️ checkstyle 0m 56s /results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt hadoop-hdfs-project/hadoop-hdfs: The patch generated 2 new + 5 unchanged - 0 fixed = 7 total (was 5)
+1 💚 mvnsite 1m 4s the patch passed
+1 💚 javadoc 0m 57s the patch passed with JDK Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04
+1 💚 javadoc 1m 30s the patch passed with JDK Private Build-1.8.0_432-8u432-gaus1-0ubuntu220.04-ga
+1 💚 spotbugs 3m 6s the patch passed
+1 💚 shadedclient 36m 19s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 298m 49s hadoop-hdfs in the patch passed.
+1 💚 asflicense 0m 46s The patch does not generate ASF License warnings.
448m 46s
Subsystem Report/Notes
Docker ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7296/2/artifact/out/Dockerfile
GITHUB PR #7296
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux 0326644bff36 5.15.0-130-generic #140-Ubuntu SMP Wed Dec 18 17:59:53 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / e609064
Default Java Private Build-1.8.0_432-8u432-gaus1-0ubuntu220.04-ga
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.25+9-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_432-8u432-gaus1-0ubuntu220.04-ga
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7296/2/testReport/
Max. process+thread count 3144 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7296/2/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@Kimahriman Kimahriman force-pushed the journal-cache-edit-after-persist branch from e609064 to 4682ddf Compare May 1, 2025 13:21
@hadoop-yetus
Copy link

hadoop-yetus commented May 1, 2025

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 57s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 1 new or modified test files.
_ trunk Compile Tests _
+1 💚 mvninstall 36m 5s trunk passed
+1 💚 compile 1m 22s trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚 compile 1m 12s trunk passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚 checkstyle 1m 9s trunk passed
+1 💚 mvnsite 1m 16s trunk passed
+1 💚 javadoc 1m 9s trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚 javadoc 1m 42s trunk passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚 spotbugs 3m 11s trunk passed
+1 💚 shadedclient 36m 43s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
-1 ❌ mvninstall 0m 59s /patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs.txt hadoop-hdfs in the patch failed.
-1 ❌ compile 1m 11s /patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdkUbuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04.txt hadoop-hdfs in the patch failed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04.
-1 ❌ javac 1m 11s /patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdkUbuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04.txt hadoop-hdfs in the patch failed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04.
-1 ❌ compile 1m 3s /patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06.txt hadoop-hdfs in the patch failed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06.
-1 ❌ javac 1m 3s /patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06.txt hadoop-hdfs in the patch failed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06.
+1 💚 blanks 0m 0s The patch has no blanks issues.
-0 ⚠️ checkstyle 0m 56s /results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt hadoop-hdfs-project/hadoop-hdfs: The patch generated 2 new + 5 unchanged - 0 fixed = 7 total (was 5)
-1 ❌ mvnsite 1m 2s /patch-mvnsite-hadoop-hdfs-project_hadoop-hdfs.txt hadoop-hdfs in the patch failed.
+1 💚 javadoc 0m 57s the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚 javadoc 1m 32s the patch passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
-1 ❌ spotbugs 1m 1s /patch-spotbugs-hadoop-hdfs-project_hadoop-hdfs.txt hadoop-hdfs in the patch failed.
-1 ❌ shadedclient 15m 11s patch has errors when building and testing our client artifacts.
_ Other Tests _
-1 ❌ unit 1m 5s /patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt hadoop-hdfs in the patch failed.
+1 💚 asflicense 0m 35s The patch does not generate ASF License warnings.
104m 3s
Subsystem Report/Notes
Docker ClientAPI=1.49 ServerAPI=1.49 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7296/3/artifact/out/Dockerfile
GITHUB PR #7296
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux d784abc13dea 5.15.0-136-generic #147-Ubuntu SMP Sat Mar 15 15:53:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 4682ddf
Default Java Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7296/3/testReport/
Max. process+thread count 552 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7296/3/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@Kimahriman Kimahriman force-pushed the journal-cache-edit-after-persist branch from 4682ddf to 1f280d2 Compare May 1, 2025 17:19
@hadoop-yetus
Copy link

hadoop-yetus commented May 1, 2025

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 58s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 1s codespell was not available.
+0 🆗 detsecrets 0m 1s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 1 new or modified test files.
_ trunk Compile Tests _
+1 💚 mvninstall 36m 21s trunk passed
+1 💚 compile 1m 20s trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚 compile 1m 8s trunk passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚 checkstyle 1m 11s trunk passed
+1 💚 mvnsite 1m 17s trunk passed
+1 💚 javadoc 1m 10s trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚 javadoc 1m 42s trunk passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚 spotbugs 3m 8s trunk passed
+1 💚 shadedclient 35m 44s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 1m 7s the patch passed
+1 💚 compile 1m 12s the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚 javac 1m 12s the patch passed
+1 💚 compile 1m 1s the patch passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚 javac 1m 1s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
-0 ⚠️ checkstyle 0m 56s /results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt hadoop-hdfs-project/hadoop-hdfs: The patch generated 2 new + 5 unchanged - 0 fixed = 7 total (was 5)
+1 💚 mvnsite 1m 8s the patch passed
+1 💚 javadoc 0m 57s the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚 javadoc 1m 28s the patch passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚 spotbugs 3m 4s the patch passed
+1 💚 shadedclient 35m 43s patch has no errors when building and testing our client artifacts.
_ Other Tests _
-1 ❌ unit 187m 18s /patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt hadoop-hdfs in the patch passed.
-1 ❌ asflicense 0m 43s /results-asflicense.txt The patch generated 30 ASF License warnings.
317m 12s
Reason Tests
Failed junit tests hadoop.fs.TestHDFSFileContextMainOperations
hadoop.fs.TestSWebHdfsFileContextMainOperations
hadoop.hdfs.server.namenode.TestUpgradeDomainBlockPlacementPolicy
hadoop.fs.viewfs.TestViewFileSystemLinkFallback
hadoop.hdfs.TestDatanodeRegistration
hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics
hadoop.hdfs.server.blockmanagement.TestBlockReportLease
hadoop.hdfs.server.namenode.TestFSDirAttrOp
hadoop.hdfs.server.namenode.TestReencryption
hadoop.hdfs.server.namenode.TestDecommissioningStatus
hadoop.fs.viewfs.TestViewFileSystemAtHdfsRoot
hadoop.fs.viewfs.TestViewFsAtHdfsRoot
hadoop.fs.viewfs.TestViewFileSystemLinkMergeSlash
hadoop.fs.viewfs.TestViewFileSystemOverloadSchemeHdfsFileSystemContract
hadoop.fs.viewfs.TestViewFileSystemLinkRegex
hadoop.fs.TestWebHdfsFileContextMainOperations
hadoop.hdfs.server.namenode.TestEditLog
hadoop.hdfs.server.namenode.TestFSNamesystemLockReport
hadoop.fs.viewfs.TestViewFileSystemHdfs
hadoop.hdfs.server.blockmanagement.TestDatanodeManager
hadoop.hdfs.server.namenode.TestSecurityTokenEditLog
hadoop.fs.viewfs.TestViewFsHdfs
hadoop.hdfs.server.namenode.TestProtectedDirectories
hadoop.fs.TestFcHdfsCreateMkdir
Subsystem Report/Notes
Docker ClientAPI=1.49 ServerAPI=1.49 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7296/4/artifact/out/Dockerfile
GITHUB PR #7296
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux b548612937eb 5.15.0-136-generic #147-Ubuntu SMP Sat Mar 15 15:53:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 1f280d2
Default Java Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7296/4/testReport/
Max. process+thread count 2689 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7296/4/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@Kimahriman Kimahriman force-pushed the journal-cache-edit-after-persist branch from 1f280d2 to 5b1e6eb Compare May 2, 2025 12:30
@hadoop-yetus
Copy link

hadoop-yetus commented May 2, 2025

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 35s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 1 new or modified test files.
_ trunk Compile Tests _
+1 💚 mvninstall 35m 48s trunk passed
+1 💚 compile 1m 23s trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚 compile 1m 11s trunk passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚 checkstyle 1m 10s trunk passed
+1 💚 mvnsite 1m 19s trunk passed
+1 💚 javadoc 1m 9s trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚 javadoc 1m 39s trunk passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚 spotbugs 3m 8s trunk passed
+1 💚 shadedclient 35m 58s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 1m 5s the patch passed
+1 💚 compile 1m 12s the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚 javac 1m 12s the patch passed
+1 💚 compile 1m 2s the patch passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚 javac 1m 2s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
-0 ⚠️ checkstyle 0m 56s /results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt hadoop-hdfs-project/hadoop-hdfs: The patch generated 2 new + 5 unchanged - 0 fixed = 7 total (was 5)
+1 💚 mvnsite 1m 6s the patch passed
+1 💚 javadoc 0m 56s the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚 javadoc 1m 28s the patch passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚 spotbugs 3m 7s the patch passed
+1 💚 shadedclient 37m 42s patch has no errors when building and testing our client artifacts.
_ Other Tests _
-1 ❌ unit 258m 13s /patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt hadoop-hdfs in the patch passed.
-1 ❌ asflicense 0m 55s /results-asflicense.txt The patch generated 2925 ASF License warnings.
389m 30s
Reason Tests
Failed junit tests hadoop.fs.viewfs.TestViewFileSystemLinkMergeSlash
hadoop.fs.viewfs.TestViewFsAtHdfsRoot
hadoop.hdfs.TestDatanodeRegistration
hadoop.fs.viewfs.TestViewFileSystemAtHdfsRoot
hadoop.fs.TestSWebHdfsFileContextMainOperations
hadoop.hdfs.server.namenode.TestCacheDirectives
hadoop.fs.TestWebHdfsFileContextMainOperations
hadoop.hdfs.server.blockmanagement.TestBlockReportLease
hadoop.fs.viewfs.TestViewFileSystemLinkRegex
hadoop.fs.viewfs.TestViewFileSystemLinkFallback
hadoop.fs.viewfs.TestViewFileSystemOverloadSchemeHdfsFileSystemContract
hadoop.fs.TestFcHdfsCreateMkdir
hadoop.fs.viewfs.TestViewFsHdfs
hadoop.fs.viewfs.TestViewFileSystemHdfs
hadoop.fs.TestHDFSFileContextMainOperations
hadoop.hdfs.server.blockmanagement.TestDatanodeManager
Subsystem Report/Notes
Docker ClientAPI=1.49 ServerAPI=1.49 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7296/5/artifact/out/Dockerfile
GITHUB PR #7296
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux 84e7c8cc2b04 5.15.0-136-generic #147-Ubuntu SMP Sat Mar 15 15:53:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 5b1e6eb
Default Java Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7296/5/testReport/
Max. process+thread count 3643 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7296/5/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@Kimahriman Kimahriman force-pushed the journal-cache-edit-after-persist branch from 5b1e6eb to 36ac76b Compare May 2, 2025 19:21
@hadoop-yetus
Copy link

hadoop-yetus commented May 3, 2025

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 36s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 1s codespell was not available.
+0 🆗 detsecrets 0m 1s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 1 new or modified test files.
_ trunk Compile Tests _
+1 💚 mvninstall 36m 31s trunk passed
+1 💚 compile 1m 20s trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚 compile 1m 11s trunk passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚 checkstyle 1m 8s trunk passed
+1 💚 mvnsite 1m 16s trunk passed
+1 💚 javadoc 1m 11s trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚 javadoc 1m 44s trunk passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚 spotbugs 3m 12s trunk passed
+1 💚 shadedclient 36m 10s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 1m 4s the patch passed
+1 💚 compile 1m 10s the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚 javac 1m 10s the patch passed
+1 💚 compile 1m 3s the patch passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚 javac 1m 3s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 0m 58s the patch passed
+1 💚 mvnsite 1m 8s the patch passed
+1 💚 javadoc 0m 57s the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚 javadoc 1m 30s the patch passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚 spotbugs 3m 8s the patch passed
+1 💚 shadedclient 36m 38s patch has no errors when building and testing our client artifacts.
_ Other Tests _
-1 ❌ unit 281m 33s /patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt hadoop-hdfs in the patch passed.
-1 ❌ asflicense 0m 57s /results-asflicense.txt The patch generated 666 ASF License warnings.
413m 0s
Reason Tests
Failed junit tests hadoop.fs.viewfs.TestViewFileSystemLinkMergeSlash
hadoop.fs.viewfs.TestViewFsAtHdfsRoot
hadoop.hdfs.TestDatanodeRegistration
hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes
hadoop.fs.viewfs.TestViewFileSystemAtHdfsRoot
hadoop.fs.TestSWebHdfsFileContextMainOperations
hadoop.hdfs.server.namenode.TestCacheDirectives
hadoop.fs.TestWebHdfsFileContextMainOperations
hadoop.hdfs.server.blockmanagement.TestBlockReportLease
hadoop.fs.viewfs.TestViewFileSystemLinkRegex
hadoop.fs.viewfs.TestViewFileSystemLinkFallback
hadoop.fs.viewfs.TestViewFileSystemOverloadSchemeHdfsFileSystemContract
hadoop.fs.TestFcHdfsCreateMkdir
hadoop.fs.viewfs.TestViewFsHdfs
hadoop.fs.viewfs.TestViewFileSystemHdfs
hadoop.fs.TestHDFSFileContextMainOperations
hadoop.hdfs.server.blockmanagement.TestDatanodeManager
Subsystem Report/Notes
Docker ClientAPI=1.49 ServerAPI=1.49 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7296/6/artifact/out/Dockerfile
GITHUB PR #7296
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux dbe3e5e8fcab 5.15.0-136-generic #147-Ubuntu SMP Sat Mar 15 15:53:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 36ac76b
Default Java Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7296/6/testReport/
Max. process+thread count 3564 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7296/6/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@github-actions
Copy link
Contributor

github-actions bot commented Sep 15, 2025

We're closing this stale PR because it has been open for 100 days with no activity. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you feel like this was a mistake, or you would like to continue working on it, please feel free to re-open it and ask for a committer to remove the stale tag and review again.
Thanks all for your contribution.

@github-actions github-actions bot added the Stale label Sep 15, 2025
@Kimahriman Kimahriman force-pushed the journal-cache-edit-after-persist branch from 36ac76b to 04cdd07 Compare September 15, 2025 13:35
@hadoop-yetus
Copy link

hadoop-yetus commented Sep 15, 2025

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 8m 49s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 1 new or modified test files.
_ trunk Compile Tests _
+1 💚 mvninstall 33m 4s trunk passed
+1 💚 compile 0m 43s trunk passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚 compile 0m 40s trunk passed with JDK Private Build-1.8.0_452-8u452-gaus1-0ubuntu120.04-b09
+1 💚 checkstyle 0m 38s trunk passed
+1 💚 mvnsite 0m 44s trunk passed
+1 💚 javadoc 0m 42s trunk passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚 javadoc 1m 3s trunk passed with JDK Private Build-1.8.0_452-8u452-gaus1-0ubuntu120.04-b09
+1 💚 spotbugs 1m 39s trunk passed
+1 💚 shadedclient 21m 39s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 0m 35s the patch passed
+1 💚 compile 0m 40s the patch passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚 javac 0m 40s the patch passed
+1 💚 compile 0m 35s the patch passed with JDK Private Build-1.8.0_452-8u452-gaus1-0ubuntu120.04-b09
+1 💚 javac 0m 35s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 0m 29s the patch passed
+1 💚 mvnsite 0m 38s the patch passed
+1 💚 javadoc 0m 33s the patch passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚 javadoc 1m 0s the patch passed with JDK Private Build-1.8.0_452-8u452-gaus1-0ubuntu120.04-b09
+1 💚 spotbugs 1m 41s the patch passed
+1 💚 shadedclient 21m 33s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 200m 39s hadoop-hdfs in the patch passed.
+1 💚 asflicense 0m 26s The patch does not generate ASF License warnings.
298m 25s
Subsystem Report/Notes
Docker ClientAPI=1.51 ServerAPI=1.51 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7296/7/artifact/out/Dockerfile
GITHUB PR #7296
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux 619fad0717b1 5.15.0-142-generic #152-Ubuntu SMP Mon May 19 10:54:31 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 04cdd07
Default Java Private Build-1.8.0_452-8u452-gaus1-0ubuntu120.04-b09
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_452-8u452-gaus1-0ubuntu120.04-b09
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7296/7/testReport/
Max. process+thread count 3969 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7296/7/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@github-actions github-actions bot closed this Sep 16, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

HDFS Stale trunk

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@Kimahriman @hadoop-yetus