Skip to content

Conversation

@andrewor14
Copy link
Contributor

This is a small step in implementing SPARK-10620, which migrates TaskMetrics to accumulators. This patch is strictly a cleanup patch and introduces no change in functionality. It literally just renames 3 fields for consistency. Today we have:

inputMetrics.recordsRead
outputMetrics.bytesWritten
shuffleReadMetrics.localBlocksFetched
...
shuffleWriteMetrics.shuffleRecordsWritten
shuffleWriteMetrics.shuffleBytesWritten
shuffleWriteMetrics.shuffleWriteTime

The shuffle write ones are kind of redundant. We can drop the shuffle part in the method names. I added backward compatible (but deprecated) methods with the old names.

Parent PR: #10717

@SparkQA
Copy link

SparkQA commented Jan 18, 2016

Test build #49608 has finished for PR 10811 at commit 0a2d9a1.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@andrewor14
Copy link
Contributor Author

retest this please

@andrewor14 andrewor14 changed the title [SPARK-10620] [MINOR] Rename 3 fields in ShuffleWriteMetrics [SPARK-12885] [MINOR] Rename 3 fields in ShuffleWriteMetrics Jan 18, 2016
Conflicts:
	core/src/main/scala/org/apache/spark/executor/TaskMetrics.scala
@andrewor14
Copy link
Contributor Author

retest this please

@SparkQA
Copy link

SparkQA commented Jan 18, 2016

Test build #49616 has finished for PR 10811 at commit 0a2d9a1.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Jan 19, 2016

Test build #49628 has finished for PR 10811 at commit 823156b.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@JoshRosen
Copy link
Contributor

LGTM, so I'm going to merge this now. Thanks!

@asfgit asfgit closed this in 2b5d11f Jan 19, 2016
@andrewor14 andrewor14 deleted the rename-things branch January 19, 2016 06:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants