[SPARK-36719][CORE] Supporting Netty Logging at the network layer #33962

attilapiros · 2021-09-10T13:31:20Z

What changes were proposed in this pull request?

Supporting Netty level logging at the network layer.

To configure Netty level logging a LogHandler must be added to the channel pipeline.
In this PR I have introduced a new class NettyLogger which is able to construct a log handler depending on the log level:

in case of log4j.logger.org.apache.spark.network.util.NettyLogger=DEBUG: a custom log handler is created which does not dump the message contents. This way the log is a bit more compact. Moreover when network level encryption is switched on this level might be sufficient.
in case of log4j.logger.org.apache.spark.network.util.NettyLogger=TRACE: Netty's own log handler is used which dumps the message contents.
otherwise (when the logger is not TRACE or DEBUG) the pipeline does not contain a log handler (there is no runtime penalty for the default setting but a long running app/service must be restarted along with the new log level to have an effect).

Why are the changes needed?

This level of logging proved to be sufficient during debugging some external shuffle related problem.
Compared with the tcpdump this log lines can be more easily correlated with the Spark internal calls.
Moreover the log layout can be configured to contain the thread names that way for a timeout a busy thread could be identified.

Does this PR introduce any user-facing change?

No

How was this patch tested?

Manually.

DEBUG level

╭─attilazsoltpiros@apiros-MBP16 ~/git/attilapiros/spark ‹SPARK-36719*›
╰─$ tail -1 ./conf/log4j.properties
log4j.logger.org.apache.spark.network.util.NettyLogger=DEBUG
╭─attilazsoltpiros@apiros-MBP16 ~/git/attilapiros/spark ‹SPARK-36719*›
╰─$ ./bin/spark-submit --class org.apache.spark.examples.JavaWordCount --master local\[8\]  ./examples/target/original-spark-examples_2.12-3.3.0-SNAPSHOT.jar README.md 2> >(grep NettyLogger) 1> /dev/null
21/09/10 15:24:35 DEBUG NettyLogger: [id: 0xb9d94fcf] REGISTERED
21/09/10 15:24:35 DEBUG NettyLogger: [id: 0xb9d94fcf] CONNECT: /172.30.64.219:61014
21/09/10 15:24:35 DEBUG NettyLogger: [id: 0xb9d94fcf, L:/172.30.64.219:61015 - R:/172.30.64.219:61014] ACTIVE
21/09/10 15:24:35 DEBUG NettyLogger: [id: 0x28101520, L:/172.30.64.219:61014 - R:/172.30.64.219:61015] REGISTERED
21/09/10 15:24:35 DEBUG NettyLogger: [id: 0x28101520, L:/172.30.64.219:61014 - R:/172.30.64.219:61015] ACTIVE
21/09/10 15:24:35 DEBUG NettyLogger: [id: 0xb9d94fcf, L:/172.30.64.219:61015 - R:/172.30.64.219:61014] WRITE 66B
21/09/10 15:24:35 DEBUG NettyLogger: [id: 0xb9d94fcf, L:/172.30.64.219:61015 - R:/172.30.64.219:61014] FLUSH
21/09/10 15:24:35 DEBUG NettyLogger: [id: 0x28101520, L:/172.30.64.219:61014 - R:/172.30.64.219:61015] READ 66B
21/09/10 15:24:35 DEBUG NettyLogger: [id: 0x28101520, L:/172.30.64.219:61014 - R:/172.30.64.219:61015] WRITE: MessageWithHeader [headerLength: 74, bodyLength: 1552705]
21/09/10 15:24:35 DEBUG NettyLogger: [id: 0x28101520, L:/172.30.64.219:61014 - R:/172.30.64.219:61015] FLUSH
21/09/10 15:24:35 DEBUG NettyLogger: [id: 0xb9d94fcf, L:/172.30.64.219:61015 - R:/172.30.64.219:61014] READ 74B
21/09/10 15:24:35 DEBUG NettyLogger: [id: 0x28101520, L:/172.30.64.219:61014 - R:/172.30.64.219:61015] READ COMPLETE
21/09/10 15:24:35 DEBUG NettyLogger: [id: 0xb9d94fcf, L:/172.30.64.219:61015 - R:/172.30.64.219:61014] READ COMPLETE
21/09/10 15:24:35 DEBUG NettyLogger: [id: 0xb9d94fcf, L:/172.30.64.219:61015 - R:/172.30.64.219:61014] READ 2048B
21/09/10 15:24:35 DEBUG NettyLogger: [id: 0xb9d94fcf, L:/172.30.64.219:61015 - R:/172.30.64.219:61014] READ 32768B
21/09/10 15:24:35 DEBUG NettyLogger: [id: 0xb9d94fcf, L:/172.30.64.219:61015 - R:/172.30.64.219:61014] READ 65536B
21/09/10 15:24:35 DEBUG NettyLogger: [id: 0xb9d94fcf, L:/172.30.64.219:61015 - R:/172.30.64.219:61014] READ 65536B
21/09/10 15:24:35 DEBUG NettyLogger: [id: 0xb9d94fcf, L:/172.30.64.219:61015 - R:/172.30.64.219:61014] READ 65536B
21/09/10 15:24:35 DEBUG NettyLogger: [id: 0xb9d94fcf, L:/172.30.64.219:61015 - R:/172.30.64.219:61014] READ 65536B
21/09/10 15:24:35 DEBUG NettyLogger: [id: 0xb9d94fcf, L:/172.30.64.219:61015 - R:/172.30.64.219:61014] READ 65536B
21/09/10 15:24:35 DEBUG NettyLogger: [id: 0xb9d94fcf, L:/172.30.64.219:61015 - R:/172.30.64.219:61014] READ 65536B
21/09/10 15:24:35 DEBUG NettyLogger: [id: 0xb9d94fcf, L:/172.30.64.219:61015 - R:/172.30.64.219:61014] READ 65536B
21/09/10 15:24:35 DEBUG NettyLogger: [id: 0xb9d94fcf, L:/172.30.64.219:61015 - R:/172.30.64.219:61014] READ 65536B
21/09/10 15:24:35 DEBUG NettyLogger: [id: 0xb9d94fcf, L:/172.30.64.219:61015 - R:/172.30.64.219:61014] READ 65536B
21/09/10 15:24:35 DEBUG NettyLogger: [id: 0xb9d94fcf, L:/172.30.64.219:61015 - R:/172.30.64.219:61014] READ 65536B
21/09/10 15:24:35 DEBUG NettyLogger: [id: 0xb9d94fcf, L:/172.30.64.219:61015 - R:/172.30.64.219:61014] READ 65536B
21/09/10 15:24:35 DEBUG NettyLogger: [id: 0xb9d94fcf, L:/172.30.64.219:61015 - R:/172.30.64.219:61014] READ 65536B
21/09/10 15:24:35 DEBUG NettyLogger: [id: 0xb9d94fcf, L:/172.30.64.219:61015 - R:/172.30.64.219:61014] READ 65536B
21/09/10 15:24:35 DEBUG NettyLogger: [id: 0xb9d94fcf, L:/172.30.64.219:61015 - R:/172.30.64.219:61014] READ 65536B
21/09/10 15:24:35 DEBUG NettyLogger: [id: 0xb9d94fcf, L:/172.30.64.219:61015 - R:/172.30.64.219:61014] READ COMPLETE
21/09/10 15:24:35 DEBUG NettyLogger: [id: 0xb9d94fcf, L:/172.30.64.219:61015 - R:/172.30.64.219:61014] READ 65536B
21/09/10 15:24:35 DEBUG NettyLogger: [id: 0xb9d94fcf, L:/172.30.64.219:61015 - R:/172.30.64.219:61014] READ 65536B
21/09/10 15:24:35 DEBUG NettyLogger: [id: 0xb9d94fcf, L:/172.30.64.219:61015 - R:/172.30.64.219:61014] READ 65536B
21/09/10 15:24:35 DEBUG NettyLogger: [id: 0xb9d94fcf, L:/172.30.64.219:61015 - R:/172.30.64.219:61014] READ 65536B
21/09/10 15:24:35 DEBUG NettyLogger: [id: 0xb9d94fcf, L:/172.30.64.219:61015 - R:/172.30.64.219:61014] READ 65536B
21/09/10 15:24:35 DEBUG NettyLogger: [id: 0xb9d94fcf, L:/172.30.64.219:61015 - R:/172.30.64.219:61014] READ 65536B
21/09/10 15:24:35 DEBUG NettyLogger: [id: 0xb9d94fcf, L:/172.30.64.219:61015 - R:/172.30.64.219:61014] READ 65536B
21/09/10 15:24:35 DEBUG NettyLogger: [id: 0xb9d94fcf, L:/172.30.64.219:61015 - R:/172.30.64.219:61014] READ 65536B
21/09/10 15:24:35 DEBUG NettyLogger: [id: 0xb9d94fcf, L:/172.30.64.219:61015 - R:/172.30.64.219:61014] READ 65536B
21/09/10 15:24:35 DEBUG NettyLogger: [id: 0xb9d94fcf, L:/172.30.64.219:61015 - R:/172.30.64.219:61014] READ 10561B
21/09/10 15:24:35 DEBUG NettyLogger: [id: 0xb9d94fcf, L:/172.30.64.219:61015 - R:/172.30.64.219:61014] READ COMPLETE
21/09/10 15:24:40 DEBUG NettyLogger: [id: 0x28101520, L:/172.30.64.219:61014 ! R:/172.30.64.219:61015] INACTIVE
21/09/10 15:24:40 DEBUG NettyLogger: [id: 0xb9d94fcf, L:/172.30.64.219:61015 - R:/172.30.64.219:61014] READ COMPLETE
21/09/10 15:24:40 DEBUG NettyLogger: [id: 0xb9d94fcf, L:/172.30.64.219:61015 ! R:/172.30.64.219:61014] INACTIVE
21/09/10 15:24:40 DEBUG NettyLogger: [id: 0xb9d94fcf, L:/172.30.64.219:61015 ! R:/172.30.64.219:61014] UNREGISTERED
21/09/10 15:24:40 DEBUG NettyLogger: [id: 0x28101520, L:/172.30.64.219:61014 ! R:/172.30.64.219:61015] UNREGISTERED

TRACE level

╭─attilazsoltpiros@apiros-MBP16 ~/git/attilapiros/spark ‹SPARK-36719*›
╰─$ tail -1 ./conf/log4j.properties
log4j.logger.org.apache.spark.network.util.NettyLogger=TRACE
╭─attilazsoltpiros@apiros-MBP16 ~/git/attilapiros/spark ‹SPARK-36719*›
╰─$ ./bin/spark-submit --class org.apache.spark.examples.JavaWordCount --master local\[8\]  ./examples/target/original-spark-examples_2.12-3.3.0-SNAPSHOT.jar README.md  1> /dev/null 2>&1
...
21/09/10 15:29:14 TRACE NettyLogger: [id: 0xf1d25786] REGISTERED
21/09/10 15:29:14 TRACE NettyLogger: [id: 0xf1d25786] CONNECT: /172.30.64.219:61044
21/09/10 15:29:14 TRACE NettyLogger: [id: 0xf1d25786, L:/172.30.64.219:61045 - R:/172.30.64.219:61044] ACTIVE
21/09/10 15:29:14 INFO TransportClientFactory: Successfully created connection to /172.30.64.219:61044 after 37 ms (0 ms spent in bootstraps)
21/09/10 15:29:14 TRACE NettyLogger: [id: 0x362fc693, L:/172.30.64.219:61044 - R:/172.30.64.219:61045] REGISTERED
21/09/10 15:29:14 TRACE NettyLogger: [id: 0x362fc693, L:/172.30.64.219:61044 - R:/172.30.64.219:61045] ACTIVE
21/09/10 15:29:14 INFO Utils: Fetching spark://172.30.64.219:61044/jars/original-spark-examples_2.12-3.3.0-SNAPSHOT.jar to /private/var/folders/t_/fr_vqcyx23vftk81ftz1k5hw0000gn/T/spark-91e059f5-1e29-4727-8602-f81206bbe48b/userFiles-50b48490-8950-4c46-b3d3-61a2c85412a3/fetchFileTemp8803030587223485061.tmp
21/09/10 15:29:14 TRACE NettyLogger: [id: 0xf1d25786, L:/172.30.64.219:61045 - R:/172.30.64.219:61044] WRITE: 66B
         +-------------------------------------------------+
         |  0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f |
+--------+-------------------------------------------------+----------------+
|00000000| 00 00 00 00 00 00 00 42 06 00 00 00 35 2f 6a 61 |.......B....5/ja|
|00000010| 72 73 2f 6f 72 69 67 69 6e 61 6c 2d 73 70 61 72 |rs/original-spar|
|00000020| 6b 2d 65 78 61 6d 70 6c 65 73 5f 32 2e 31 32 2d |k-examples_2.12-|
|00000030| 33 2e 33 2e 30 2d 53 4e 41 50 53 48 4f 54 2e 6a |3.3.0-SNAPSHOT.j|
|00000040| 61 72                                           |ar              |
+--------+-------------------------------------------------+----------------+
21/09/10 15:29:14 TRACE NettyLogger: [id: 0xf1d25786, L:/172.30.64.219:61045 - R:/172.30.64.219:61044] FLUSH
21/09/10 15:29:14 TRACE NettyLogger: [id: 0x362fc693, L:/172.30.64.219:61044 - R:/172.30.64.219:61045] READ: 66B
         +-------------------------------------------------+
         |  0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f |
+--------+-------------------------------------------------+----------------+
|00000000| 00 00 00 00 00 00 00 42 06 00 00 00 35 2f 6a 61 |.......B....5/ja|
|00000010| 72 73 2f 6f 72 69 67 69 6e 61 6c 2d 73 70 61 72 |rs/original-spar|
|00000020| 6b 2d 65 78 61 6d 70 6c 65 73 5f 32 2e 31 32 2d |k-examples_2.12-|
|00000030| 33 2e 33 2e 30 2d 53 4e 41 50 53 48 4f 54 2e 6a |3.3.0-SNAPSHOT.j|
|00000040| 61 72                                           |ar              |
+--------+-------------------------------------------------+----------------+
21/09/10 15:29:14 TRACE NettyLogger: [id: 0x362fc693, L:/172.30.64.219:61044 - R:/172.30.64.219:61045] WRITE: MessageWithHeader [headerLength: 74, bodyLength: 1552705]
21/09/10 15:29:14 TRACE NettyLogger: [id: 0x362fc693, L:/172.30.64.219:61044 - R:/172.30.64.219:61045] FLUSH
21/09/10 15:29:14 TRACE NettyLogger: [id: 0xf1d25786, L:/172.30.64.219:61045 - R:/172.30.64.219:61044] READ: 74B
...

SparkQA · 2021-09-10T14:11:40Z

Kubernetes integration test unable to build dist.

exiting with code: 1
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47655/

SparkQA · 2021-09-10T16:06:31Z

Test build #143151 has finished for PR 33962 at commit c5a05d8.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
public class NettyLogger

common/network-common/src/main/java/org/apache/spark/network/TransportContext.java

dongjoon-hyun

+1, LGTM from my side. Thank you for adding this, @attilapiros .

common/network-common/src/main/java/org/apache/spark/network/TransportContext.java

mridulm · 2021-09-10T20:20:14Z

+CC @otterc
You have worked on similar in past.

mridulm

Nice work @attilapiros, this is very useful.

mridulm · 2021-09-10T20:24:06Z

common/network-common/src/main/java/org/apache/spark/network/util/NettyLogger.java

+
+    protected String format(ChannelHandlerContext ctx, String eventName, Object arg) {
+      if (arg instanceof ByteBuf) {
+        return format(ctx, eventName) + " " + ((ByteBuf) arg).readableBytes() + "B";


super nit: Add a space between value and "B" (here and below) ?

This (no space) is consistent with Netty's own logging.
You can see it in the TRACE output in the PR description.
But I copy it for you (the last line is the proof):

+-------------------------------------------------+ | 0 1 2 3 4 5 6 7 8 9 a b c d e f | +--------+-------------------------------------------------+----------------+ |00000000| 00 00 00 00 00 00 00 42 06 00 00 00 35 2f 6a 61 |.......B....5/ja| |00000010| 72 73 2f 6f 72 69 67 69 6e 61 6c 2d 73 70 61 72 |rs/original-spar| |00000020| 6b 2d 65 78 61 6d 70 6c 65 73 5f 32 2e 31 32 2d |k-examples_2.12-| |00000030| 33 2e 33 2e 30 2d 53 4e 41 50 53 48 4f 54 2e 6a |3.3.0-SNAPSHOT.j| |00000040| 61 72 |ar | +--------+-------------------------------------------------+----------------+ 21/09/10 15:29:14 TRACE NettyLogger: [id: 0xf1d25786, L:/172.30.64.219:61045 - R:/172.30.64.219:61044] FLUSH 21/09/10 15:29:14 TRACE NettyLogger: [id: 0x362fc693, L:/172.30.64.219:61044 - R:/172.30.64.219:61045] READ: 66B

common/network-common/src/main/java/org/apache/spark/network/util/NettyLogger.java

common/network-common/src/main/java/org/apache/spark/network/TransportContext.java

viirya

Looks useful. Thanks for this work!

Is there any document we should add this log4j.logger.org.apache.spark.network.util.NettyLogger?

attilapiros · 2021-09-11T07:46:19Z

@viirya Thanks for the review!

Is there any document we should add this log4j.logger.org.apache.spark.network.util.NettyLogger?

So far I haven't seen any specific logger which is documented. Probably because loggers are more for the developers who are doing the debugging and not for the users. But if you can point me to a document sure I will extend it.

attilapiros · 2021-09-11T07:50:17Z

I have done a quick search and no markdown contains any relevant mention of DEBUG/TRACE:

╭─attilazsoltpiros@apiros-MBP16 ~/git/attilapiros/spark ‹SPARK-36719*›
╰─$ find . -name "*.md" -exec grep DEBUG {} \;  -print
enable extra logging of Kerberos operations in Hadoop by setting the `HADOOP_JAAS_DEBUG`
export HADOOP_JAAS_DEBUG=true
spark.yarn.appMasterEnv.HADOOP_JAAS_DEBUG true
Finally, if the log level for `org.apache.spark.deploy.yarn.Client` is set to `DEBUG`, the log
./docs/running-on-yarn.md
╭─attilazsoltpiros@apiros-MBP16 ~/git/attilapiros/spark ‹SPARK-36719*›
╰─$ find . -name "*.md" -exec grep TRACE {} \;  -print
╭─attilazsoltpiros@apiros-MBP16 ~/git/attilapiros/spark ‹SPARK-36719*›
╰─$

viirya · 2021-09-11T07:56:39Z

So far I haven't seen any specific logger which is documented. Probably because loggers are more for the developers who are doing the debugging and not for the users. But if you can point me to a document sure I will extend it.

Hm, I only see a logging related section "Configuring Logging" in "Spark Configuration" https://spark.apache.org/docs/latest/configuration.html#configuring-logging. Okay to skip it.

SparkQA · 2021-09-11T08:29:36Z

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47666/

SparkQA · 2021-09-11T08:38:21Z

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47666/

SparkQA · 2021-09-11T09:49:45Z

Test build #143162 has finished for PR 33962 at commit 1104d6e.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds no public classes.

viirya · 2021-09-11T17:10:29Z

retest this please

SparkQA · 2021-09-11T18:32:24Z

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47673/

SparkQA · 2021-09-11T18:42:30Z

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47673/

SparkQA · 2021-09-11T20:11:01Z

Test build #143169 has finished for PR 33962 at commit 1104d6e.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

dongjoon-hyun · 2021-09-11T23:13:42Z

Thank you, @attilapiros , @mridulm , @viirya .
Merged to master.

### What changes were proposed in this pull request? Support Netty level logging at the network layer for Celeborn. To configure Netty level logging a LogHandler must be added to the channel pipeline. `NettyLogger` is introduced as a new class which is able to construct a log handler depending on the log level: - In case of `<Logger name="org.apache.celeborn.common.network.util.NettyLogger" level="DEBUG" additivity="false">`: a custom log handler is created which does not dump the message contents. This way the log is a bit more compact. Moreover when network level encryption is switched on this level might be sufficient. - In case of `<Logger name="org.apache.celeborn.common.network.util.NettyLogger" level="TRACE" additivity="false">`: Netty's own log handler is used which dumps the message contents. - Otherwise (when the logger is not `TRACE` or `DEBUG`) the pipeline does not contain a log handler (there is no runtime penalty for the default setting but a long running service must be restarted along with the new log level to have an effect). Backport: - [[SPARK-36719][CORE] Supporting Netty Logging at the network layer](apache/spark#33962) - [[SPARK-45377][CORE] Handle InputStream in NettyLogger](apache/spark#43165) ### Why are the changes needed? This level of logging proved to be sufficient during debugging some external shuffle related problem. Compared with the tcpdump this log lines can be more easily correlated with the Celeborn internal calls. Moreover the log layout can be configured to contain the thread names that way for a timeout a busy thread could be identified. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Local manually test. Closes #2423 from SteNicholas/CELEBORN-1359. Authored-by: SteNicholas <[email protected]> Signed-off-by: Shuang <[email protected]>

### What changes were proposed in this pull request? Support Netty level logging at the network layer for Celeborn. To configure Netty level logging a LogHandler must be added to the channel pipeline. `NettyLogger` is introduced as a new class which is able to construct a log handler depending on the log level: - In case of `<Logger name="org.apache.celeborn.common.network.util.NettyLogger" level="DEBUG" additivity="false">`: a custom log handler is created which does not dump the message contents. This way the log is a bit more compact. Moreover when network level encryption is switched on this level might be sufficient. - In case of `<Logger name="org.apache.celeborn.common.network.util.NettyLogger" level="TRACE" additivity="false">`: Netty's own log handler is used which dumps the message contents. - Otherwise (when the logger is not `TRACE` or `DEBUG`) the pipeline does not contain a log handler (there is no runtime penalty for the default setting but a long running service must be restarted along with the new log level to have an effect). Backport: - [[SPARK-36719][CORE] Supporting Netty Logging at the network layer](apache/spark#33962) - [[SPARK-45377][CORE] Handle InputStream in NettyLogger](apache/spark#43165) ### Why are the changes needed? This level of logging proved to be sufficient during debugging some external shuffle related problem. Compared with the tcpdump this log lines can be more easily correlated with the Celeborn internal calls. Moreover the log layout can be configured to contain the thread names that way for a timeout a busy thread could be identified. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Local manually test. Closes apache#2423 from SteNicholas/CELEBORN-1359. Authored-by: SteNicholas <[email protected]> Signed-off-by: Shuang <[email protected]>

Initial upload.

c5a05d8

github-actions bot added the CORE label Sep 10, 2021

attilapiros changed the title ~~[SPARK-36719][WIP] Supporting Netty Logging at the network layer~~ [SPARK-36719][CORE][WIP] Supporting Netty Logging at the network layer Sep 10, 2021

attilapiros changed the title ~~[SPARK-36719][CORE][WIP] Supporting Netty Logging at the network layer~~ [SPARK-36719][CORE] Supporting Netty Logging at the network layer Sep 10, 2021

dongjoon-hyun reviewed Sep 10, 2021

View reviewed changes

common/network-common/src/main/java/org/apache/spark/network/TransportContext.java Show resolved Hide resolved

dongjoon-hyun approved these changes Sep 10, 2021

View reviewed changes

dongjoon-hyun reviewed Sep 10, 2021

View reviewed changes

common/network-common/src/main/java/org/apache/spark/network/TransportContext.java Show resolved Hide resolved

mridulm approved these changes Sep 10, 2021

View reviewed changes

viirya reviewed Sep 11, 2021

View reviewed changes

common/network-common/src/main/java/org/apache/spark/network/util/NettyLogger.java Outdated Show resolved Hide resolved

viirya reviewed Sep 11, 2021

View reviewed changes

common/network-common/src/main/java/org/apache/spark/network/TransportContext.java Outdated Show resolved Hide resolved

viirya approved these changes Sep 11, 2021

View reviewed changes

applying review comments

1104d6e

dongjoon-hyun closed this in ba81b92 Sep 11, 2021

SteNicholas mentioned this pull request Mar 27, 2024

[CELEBORN-1359] Support Netty Logging at the network layer apache/celeborn#2423

Closed

This was referenced Feb 11, 2025

Support Netty Logging at the network layer apache/fluss#378

Closed

[rpc] Support Netty Logging at the network layer apache/fluss#379

Merged

[SPARK-36719][CORE] Supporting Netty Logging at the network layer #33962

[SPARK-36719][CORE] Supporting Netty Logging at the network layer #33962

Uh oh!

Conversation

attilapiros commented Sep 10, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changes were proposed in this pull request?

Why are the changes needed?

Does this PR introduce any user-facing change?

How was this patch tested?

DEBUG level

TRACE level

Uh oh!

SparkQA commented Sep 10, 2021

Uh oh!

SparkQA commented Sep 10, 2021

Uh oh!

Uh oh!

dongjoon-hyun left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mridulm commented Sep 10, 2021

Uh oh!

mridulm left a comment

Choose a reason for hiding this comment

Uh oh!

mridulm Sep 10, 2021

Choose a reason for hiding this comment

Uh oh!

attilapiros Sep 11, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

viirya left a comment

Choose a reason for hiding this comment

Uh oh!

attilapiros commented Sep 11, 2021

Uh oh!

attilapiros commented Sep 11, 2021

Uh oh!

viirya commented Sep 11, 2021

Uh oh!

SparkQA commented Sep 11, 2021

Uh oh!

SparkQA commented Sep 11, 2021

Uh oh!

SparkQA commented Sep 11, 2021

Uh oh!

viirya commented Sep 11, 2021

Uh oh!

SparkQA commented Sep 11, 2021

Uh oh!

SparkQA commented Sep 11, 2021

Uh oh!

SparkQA commented Sep 11, 2021

Uh oh!

dongjoon-hyun commented Sep 11, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

attilapiros commented Sep 10, 2021 •

edited

Loading

attilapiros Sep 11, 2021 •

edited

Loading