Skip to content

Conversation

@aw-was-here
Copy link
Contributor

Testing Apache Yetus master branch doing PRs for Hadoop

@aw-was-here aw-was-here changed the title Jenkinsfile for Hadoop HADOOP-16035. Jenkinsfile for Hadoop Jan 8, 2019
@apache-yetus
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
0 reexec 453 Docker mode activated.
_ Prechecks _
+1 @author 0 The patch does not contain any @author tags.
_ trunk Compile Tests _
+1 shadedclient 881 branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 shellcheck 0 There were no new shellcheck issues.
+1 shelldocs 14 There were no new shelldocs issues.
+1 whitespace 0 The patch has no whitespace issues.
+1 shadedclient 840 patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 asflicense 27 The patch does not generate ASF License warnings.
2305
Subsystem Report/Notes
Docker Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-459/1/artifact/out/Dockerfile
GITHUB PR #459
Optional Tests dupname asflicense shellcheck shelldocs
uname Linux 4dacb0a402eb 4.4.0-139-generic #165~14.04.1-Ubuntu SMP Wed Oct 31 10:55:11 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/hadoop.sh
git revision trunk / 999f31f
maven version: Apache Maven 3.3.9
shellcheck v0.4.6
Max. process+thread count 342 (vs. ulimit of 5500)
modules C: . U: .
Console output https://builds.apache.org/job/hadoop-multibranch/job/PR-459/1/console
Powered by Apache Yetus 0.9.0-SNAPSHOT http://yetus.apache.org

This message was automatically generated.

@apache-yetus
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
0 reexec 335 Docker mode activated.
_ Prechecks _
+1 @author 0 The patch does not contain any @author tags.
_ trunk Compile Tests _
+1 shadedclient 822 branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 shellcheck 0 There were no new shellcheck issues.
+1 shelldocs 14 There were no new shelldocs issues.
+1 whitespace 0 The patch has no whitespace issues.
+1 shadedclient 805 patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 asflicense 27 The patch does not generate ASF License warnings.
2100
Subsystem Report/Notes
Docker Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-459/2/artifact/out/Dockerfile
GITHUB PR #459
Optional Tests dupname asflicense shellcheck shelldocs
uname Linux bd5bec57d355 4.4.0-139-generic #165~14.04.1-Ubuntu SMP Wed Oct 31 10:55:11 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/hadoop.sh
git revision trunk / 999f31f
maven version: Apache Maven 3.3.9
shellcheck v0.4.6
Max. process+thread count 346 (vs. ulimit of 5500)
modules C: . U: .
Console output https://builds.apache.org/job/hadoop-multibranch/job/PR-459/2/console
Powered by Apache Yetus 0.9.0-SNAPSHOT http://yetus.apache.org

This message was automatically generated.

@steveloughran
Copy link
Contributor

this is slick. What happens if someone comments?

@aw-was-here
Copy link
Contributor Author

Thanks!

It shouldn't do anything if INFRA configured the webhook correctly and Jenkins is functioning properly. In order to make this go live, there are three things that need to happen:

a) Hadoop PMC needs to make and own a github account, then provide an OAuth token for that account (I'm using Apache Yetus' for this test.)
b) Apache Yetus 0.9.0 needs to get released (unless Hadoop wants to hook into Yetus' master)
c) This Jenkinsfile (or something like it) needs to get committed

@aw-was-here
Copy link
Contributor Author

Hadoop PMC needs to make and own a github account, then provide an OAuth token for that account (I'm using Apache Yetus' for this test.)

BTW, the creds for this account are stored in the Jenkins credential folder for this job. I can send someone the link and they can put the details in. I don't need to know them.

@steveloughran
Copy link
Contributor

OK, starting this work. Account will be @hadoop-yetus

@apache-yetus
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
0 reexec 528 Docker mode activated.
_ Prechecks _
+1 @author 0 The patch does not contain any @author tags.
_ trunk Compile Tests _
+1 shadedclient 966 branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 shellcheck 0 There were no new shellcheck issues.
+1 shelldocs 13 There were no new shelldocs issues.
+1 whitespace 0 The patch has no whitespace issues.
+1 shadedclient 909 patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 asflicense 31 The patch does not generate ASF License warnings.
2553
Subsystem Report/Notes
Docker Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-459/3/artifact/out/Dockerfile
GITHUB PR #459
Optional Tests dupname asflicense shellcheck shelldocs
uname Linux 826d1a07357a 4.4.0-139-generic #165~14.04.1-Ubuntu SMP Wed Oct 31 10:55:11 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/hadoop.sh
git revision trunk / 709ddb1
maven version: Apache Maven 3.3.9
shellcheck v0.4.6
Max. process+thread count 339 (vs. ulimit of 5500)
modules C: . U: .
Console output https://builds.apache.org/job/hadoop-multibranch/job/PR-459/3/console
Powered by Apache Yetus 0.9.0-SNAPSHOT http://yetus.apache.org

This message was automatically generated.

@apache-yetus
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
0 reexec 26 Docker mode activated.
_ Prechecks _
+1 @author 0 The patch does not contain any @author tags.
_ trunk Compile Tests _
+1 shadedclient 934 branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 shellcheck 0 There were no new shellcheck issues.
+1 shelldocs 18 There were no new shelldocs issues.
+1 whitespace 0 The patch has no whitespace issues.
+1 shadedclient 717 patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 asflicense 33 The patch does not generate ASF License warnings.
1819
Subsystem Report/Notes
Docker Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-459/4/artifact/out/Dockerfile
GITHUB PR #459
Optional Tests dupname asflicense shellcheck shelldocs
uname Linux 035d7a1cbf25 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/hadoop.sh
git revision trunk / 709ddb1
maven version: Apache Maven 3.3.9
shellcheck v0.4.6
Max. process+thread count 443 (vs. ulimit of 5500)
modules C: . U: .
Console output https://builds.apache.org/job/hadoop-multibranch/job/PR-459/4/console
Powered by Apache Yetus 0.9.0-SNAPSHOT http://yetus.apache.org

This message was automatically generated.

@apache-yetus
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
0 reexec 22 Docker mode activated.
_ Prechecks _
+1 @author 0 The patch does not contain any @author tags.
_ trunk Compile Tests _
+1 shadedclient 892 branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 shellcheck 0 There were no new shellcheck issues.
+1 shelldocs 14 There were no new shelldocs issues.
+1 whitespace 0 The patch has no whitespace issues.
+1 shadedclient 843 patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 asflicense 27 The patch does not generate ASF License warnings.
1832
Subsystem Report/Notes
Docker Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-459/5/artifact/out/Dockerfile
GITHUB PR #459
Optional Tests dupname asflicense shellcheck shelldocs
uname Linux f25545f5984f 4.4.0-139-generic #165~14.04.1-Ubuntu SMP Wed Oct 31 10:55:11 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/hadoop.sh
git revision trunk / 3420e26
maven version: Apache Maven 3.3.9
shellcheck v0.4.6
Max. process+thread count 340 (vs. ulimit of 5500)
modules C: . U: .
Console output https://builds.apache.org/job/hadoop-multibranch/job/PR-459/5/console
Powered by Apache Yetus 0.9.0-SNAPSHOT http://yetus.apache.org

This message was automatically generated.

@aw-was-here
Copy link
Contributor Author

OK, now the build on the jenkins side is green regardless if junit found test results or not. (since not every patch touches java... case in point.)

@apache-yetus
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
0 reexec 31 Docker mode activated.
_ Prechecks _
+1 @author 0 The patch does not contain any @author tags.
_ trunk Compile Tests _
+1 shadedclient 795 branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 shellcheck 0 There were no new shellcheck issues.
+1 shelldocs 14 There were no new shelldocs issues.
+1 whitespace 0 The patch has no whitespace issues.
+1 shadedclient 715 patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 asflicense 26 The patch does not generate ASF License warnings.
1615
Subsystem Report/Notes
Docker Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-459/6/artifact/out/Dockerfile
GITHUB PR #459
Optional Tests dupname asflicense shellcheck shelldocs
uname Linux 5008665bc1a1 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/hadoop.sh
git revision trunk / 852701f
maven version: Apache Maven 3.3.9
shellcheck v0.4.6
Max. process+thread count 443 (vs. ulimit of 5500)
modules C: . U: .
Console output https://builds.apache.org/job/hadoop-multibranch/job/PR-459/6/console
Powered by Apache Yetus 0.9.0 http://yetus.apache.org

This message was automatically generated.

@apache-yetus
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
0 reexec 32 Docker mode activated.
_ Prechecks _
+1 @author 0 The patch does not contain any @author tags.
_ trunk Compile Tests _
+1 shadedclient 806 branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 shellcheck 0 There were no new shellcheck issues.
+1 shelldocs 18 There were no new shelldocs issues.
+1 whitespace 0 The patch has no whitespace issues.
+1 shadedclient 715 patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 asflicense 28 The patch does not generate ASF License warnings.
1638
Subsystem Report/Notes
Docker Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-459/8/artifact/out/Dockerfile
GITHUB PR #459
Optional Tests dupname asflicense shellcheck shelldocs
uname Linux 708479b19413 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/hadoop.sh
git revision trunk / bf08f4a
maven version: Apache Maven 3.3.9
shellcheck v0.4.6
Max. process+thread count 445 (vs. ulimit of 5500)
modules C: . U: .
Console output https://builds.apache.org/job/hadoop-multibranch/job/PR-459/8/console
Powered by Apache Yetus 0.9.0 http://yetus.apache.org

This message was automatically generated.

@apache-yetus
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
0 reexec 29 Docker mode activated.
_ Prechecks _
+1 @author 0 The patch does not contain any @author tags.
_ trunk Compile Tests _
+1 shadedclient 816 branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 shellcheck 0 There were no new shellcheck issues.
+1 shelldocs 18 There were no new shelldocs issues.
+1 whitespace 0 The patch has no whitespace issues.
+1 shadedclient 708 patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 asflicense 26 The patch does not generate ASF License warnings.
1633
Subsystem Report/Notes
Docker Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-459/11/artifact/out/Dockerfile
GITHUB PR #459
Optional Tests dupname asflicense shellcheck shelldocs
uname Linux 315016cc177d 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/hadoop.sh
git revision trunk / 6d7eedf
maven version: Apache Maven 3.3.9
shellcheck v0.4.6
Max. process+thread count 446 (vs. ulimit of 5500)
modules C: . U: .
Console output https://builds.apache.org/job/hadoop-multibranch/job/PR-459/11/console
Powered by Apache Yetus 0.9.0 http://yetus.apache.org

This message was automatically generated.

@aw-was-here
Copy link
Contributor Author

FYI: I have removed the Apache Yetus credentials from the job that was used for testing. Hadoop community will need to provide their own.

@hadoop-yetus
Copy link

hadoop-yetus commented Jan 21, 2019 via email

@hadoop-yetus
Copy link

(that's stevel; its mapping my cloudera email to the elephant right now. We need to get it its own email address)

@aw-was-here
Copy link
Contributor Author

Merged.

@aw-was-here aw-was-here deleted the yetus-test branch February 21, 2019 15:40
shanthoosh pushed a commit to shanthoosh/hadoop that referenced this pull request Oct 15, 2019
Getting the PR out to unblock sanil.
Pending tasks
 1. Add documentation
 2. Add more tests
 3. Currently initialization of the stream happens using createStream on admin. We need to find a way to expose the same functionality to low level users.
 4. To populate the initial stream, clients need to get a handle on producer and produce the messages. Alternatively, we can support serialization of source data and pass it as config to the system to initialize. We have a hook in place for this but not implemented it completely.
  5. Clean up consumed data on the buffer based on the lowest offsets of the consumers.

I will create JIRAs for all the pending tasks and fix them iteratively.

xinyuiscool ^^

Author: Bharath Kumarasubramanian <[email protected]>

Reviewers: Xinyu Liu <[email protected]>

Closes apache#459 from bharathkk/single-node-testing
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants