Skip to content

Conversation

@bomeng
Copy link
Contributor

@bomeng bomeng commented Dec 8, 2015

The original code does not properly handle the cases where the prefix is null, but suffix is not null - the suffix should be used but is not.

The fix is using StringBuilder to construct the proper file name.

@bomeng bomeng changed the title [Spark 12136] [Streaming] rddToFileName does not properly handle prefix and suffix parameters [SPARK-12136] [Streaming] rddToFileName does not properly handle prefix and suffix parameters Dec 8, 2015
@bomeng bomeng changed the title [SPARK-12136] [Streaming] rddToFileName does not properly handle prefix and suffix parameters [SPARK-12136] [STREAMING] rddToFileName does not properly handle prefix and suffix parameters Dec 8, 2015
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's fine; it might be simpler without bothering with StringBuilder but not by much.
(In your code you should use val result and you do not need to assign result each time.)

var result = time.milliseconds.toString
if (prefix != null && prefix.length > 0) {
  result = prefix + "-" result
}
if (suffix != null && suffix.length > 0) {
  result = result + "." + suffix
}
result

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code was changed according to your comments by using String instead.

@SparkQA
Copy link

SparkQA commented Dec 8, 2015

Test build #2179 has finished for PR 10185 at commit 21d7bcd.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):\n * public class JavaQuantileDiscretizerExample\n * public abstract static class PrefixComputer\n

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lets use string interpolation for this.

@srowen
Copy link
Member

srowen commented Dec 9, 2015

LGTM

@SparkQA
Copy link

SparkQA commented Dec 9, 2015

Test build #2189 has finished for PR 10185 at commit 4d82c43.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):\n * public class JavaQuantileDiscretizerExample\n * public abstract static class PrefixComputer\n

asfgit pushed a commit that referenced this pull request Dec 10, 2015
…x and suffix parameters

The original code does not properly handle the cases where the prefix is null, but suffix is not null - the suffix should be used but is not.

The fix is using StringBuilder to construct the proper file name.

Author: bomeng <[email protected]>
Author: Bo Meng <[email protected]>

Closes #10185 from bomeng/SPARK-12136.

(cherry picked from commit e29704f)
Signed-off-by: Sean Owen <[email protected]>
@srowen
Copy link
Member

srowen commented Dec 10, 2015

Merged to master/1.6

@asfgit asfgit closed this in e29704f Dec 10, 2015
@bomeng bomeng deleted the SPARK-12136 branch April 9, 2016 15:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants