[SPARK-19610][SQL] Support parsing multiline CSV files #16976

HyukjinKwon · 2017-02-17T18:35:12Z

What changes were proposed in this pull request?

This PR proposes the support for multiple lines for CSV by resembling the multiline supports in JSON datasource (in case of JSON, per file).

So, this PR introduces wholeFile option which makes the format not splittable and reads each whole file. Since Univocity parser can produces each row from a stream, it should be capable of parsing very large documents when the internal rows are fix in the memory.

How was this patch tested?

Unit tests in CSVSuite and tests.py

Manual tests with a single 9GB CSV file in local file system, for example,

spark.read.option("wholeFile", true).option("inferSchema", true).csv("tmp.csv").count()

HyukjinKwon · 2017-02-17T18:35:43Z

(I should clean it up more and take a look into this deeper. I left open this for running tests.)

SparkQA · 2017-02-17T20:46:26Z

Test build #73062 has finished for PR 16976 at commit 3a303c7.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
abstract class CSVDataSource extends Serializable

SparkQA · 2017-02-17T20:52:48Z

Test build #73063 has finished for PR 16976 at commit 8275d12.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
abstract class CSVDataSource extends Serializable

HyukjinKwon · 2017-02-18T02:07:25Z

cc @NathanHowell, this basically resembles your JSON approach. Could you take a look and see if it makes sense please?

HyukjinKwon · 2017-02-18T02:07:40Z

cc @cloud-fan and @rxin too.

SparkQA · 2017-02-18T03:27:26Z

Test build #73088 has finished for PR 16976 at commit 44c9465.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2017-02-18T04:10:35Z

Test build #73089 has finished for PR 16976 at commit 373eec9.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2017-02-18T14:00:47Z

Test build #73105 has finished for PR 16976 at commit 2ddbf15.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

felixcheung · 2017-02-19T18:29:27Z

hmm, I understand the motivation for this, though my understanding with csv generally either avoid having newline in field or some implementation would require quotes around field value with newline

https://en.wikipedia.org/wiki/Comma-separated_values#General_functionality

HyukjinKwon · 2017-02-19T22:42:59Z

I initially left the similar comment in the same JIRAs. However, that there are a quite bit of similar JIRAs complaining about this and the original CSV datasource tried to support this although that was incorrectly implemented. This tries to match it with JSON one at least and it might be better to provide a way to process such CSV files. Actually, current implementation requires quotes :). (It was told R supports this case too actually).

felixcheung · 2017-02-21T09:40:18Z

I see, fair enough. cc @falaki

cloud-fan

looks pretty good, left some minor comments

cloud-fan · 2017-02-23T06:58:43Z

python/test_support/sql/ages_newlines.csv

wow you are only 25?

I was.. in the last year :).

cloud-fan · 2017-02-23T07:00:19Z

sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVDataSource.scala

I assume this is exactly same with https://github.com/apache/spark/pull/16976/files#diff-56fbd53c6ada276cb4930affe3720be3L77

Yup, just copied and pasted. Just checked that they are same line by line.

cloud-fan · 2017-02-23T07:02:31Z

sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVDataSource.scala

nit: I'd prefer xxx.flatMap(line => parser.parse(line))

cloud-fan · 2017-02-23T07:18:22Z

sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/UnivocityParser.scala

nit: it's safer to add a check

if (!hasNext) { throw new NoSuchElementException("End of stream") }

HyukjinKwon · 2017-02-23T08:13:54Z

Per 4dc48e1, I ran a build with Scala 2.10.

cloud-fan · 2017-02-23T08:25:47Z

sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/UnivocityParser.scala

the normal code path has a more complex way to drop the header, is it ok to simply drop the first parsed record here?

Yup, existing code path processes line by line so comments and empty lines had to manually skipped because it threw an error if each line has only commented line or empty. This is nicely handled in Univocity and the initial version of CSV actually did this in this way.

cloud-fan · 2017-02-23T08:35:21Z

sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala

can we just check the schema and result of df?

assert(df.schema.map(_.name) == Seq("string\"", "integer\n\n", ...)) checkAnswer(df, Row("this is a\nsimple\nstring", ...) :: Row ...)

writing df out and reading it back and comparing the result doesn't mean df has corrected result.

NathanHowell · 2017-02-23T08:38:45Z

sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVDataSource.scala

Could this be csv.flatMap { lines => ...?

I think we have so far promoted the usage of mapPartitions and iterator for a better performance (not checked by myself though). At least, these are being used more commonly up to my knowledge and it makes it consistent. I would like to keep this way if you are fine.

it has better performance if you use mapPartitionsInternal, otherwise these 2 are exactly same...

I see. Thank you. I didn't know.

NathanHowell · 2017-02-23T08:39:08Z

sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVDataSource.scala

Does this work on 0 byte files?

Let me check and add a test.

It seems the original one is not working too. Can I maybe open another JIRA and PR both together (or a follow-up)?

scala> spark.read.csv("a") java.util.NoSuchElementException: next on empty iterator at scala.collection.Iterator$$anon$2.next(Iterator.scala:39) at scala.collection.Iterator$$anon$2.next(Iterator.scala:37) at scala.collection.IndexedSeqLike$Elements.next(IndexedSeqLike.s

It seems we need a bit of fix including other CSV paths.

Another JIRA is a good idea for the line based parser, it could be fixed in this PR or seperately. WholeFileCSVDataSource should be fixed in this PR before merging, since it's new functionality.

Sure, let me try.

Filed in https://issues.apache.org/jira/browse/SPARK-19709 for line based one.

NathanHowell · 2017-02-23T08:40:56Z

sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVDataSource.scala

Might be nice to move this over to CodecStreams (with an appropriate name) and fix up other classes that use this pattern.

NathanHowell · 2017-02-23T08:43:24Z

sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVFileFormat.scala

Should this return None instead of throwing an exception? The JSON parser does, though I'm not sure which approach is recommended. They should be consistent though.

I think returning None is better, the upper-level(DataSource) will handle it and throw exception

This is not related with this JIRA itself. I would like to avoid other additional changes.

NathanHowell · 2017-02-23T08:59:25Z

sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVDataSource.scala

Both subclasses return Some(...) so it would be more clear to change this to def infer(...): StructType and do the option wrapping only in CSVFileFormat.inferSchema

This was copied from JsonDataSource.infer. I would like to keep this in case when we deal with #16976 (comment) and maybe introduce another parent or at least for consistency.

Sounds good to me. Reducing code duplication between the JSON and CSV parsers would be great.

NathanHowell · 2017-02-23T09:00:48Z

sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVDataSource.scala

This should also be tested on empty files, if it's not already.

Yea, indeed it is an issue. #16976 (comment)

SparkQA · 2017-02-23T09:35:59Z

Test build #73334 has finished for PR 16976 at commit 4dc48e1.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

HyukjinKwon · 2017-02-23T10:09:25Z

sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVDataSource.scala

I just used if-else here because getOrElse things seems making this hard to read to me.

HyukjinKwon · 2017-02-23T10:14:27Z

Thanks for your review @NathanHowell and @cloud-fan. I just addressed them and ran a build with Scala 2.10 per 8474933.

SparkQA · 2017-02-23T10:17:41Z

Test build #73336 has finished for PR 16976 at commit e00bdee.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

HyukjinKwon · 2017-02-23T10:19:44Z

sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala

Default quote is ". This behaviour is definately same with original CSV.

scala> Seq("\"1\"").toDF.write.text("/tmp/abc") scala> spark.read.option("header", true).csv("/tmp/abc").show() +---+ | 1| +---+ +---+ scala> spark.read.csv("/tmp/abc").show() +---+ |_c0| +---+ | 1| +---+

so the string" will just become string?

Yup.

" string"

becomes \nstring. It requires quotes for newlines.

SparkQA · 2017-02-23T12:06:33Z

Test build #73341 has finished for PR 16976 at commit 8474933.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2017-02-24T11:27:39Z

Test build #73417 has finished for PR 16976 at commit fbdfc61.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2017-02-24T12:10:25Z

Test build #73423 has finished for PR 16976 at commit 793a3eb.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

cloud-fan · 2017-02-24T19:19:55Z

sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/CodecStreams.scala

this doesn't tell the difference between this method and createInputStream(config: Configuration, file: Path).

We also need a better name for this method.

Sure, let me try to clean up this and other comments.

cloud-fan · 2017-02-24T19:20:17Z

sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/CodecStreams.scala

nit: we can write createInputStream(config, new Path(path)) directly

cloud-fan · 2017-02-24T19:25:23Z

sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVDataSource.scala

nit:

val rdd = new BinaryFileRDD ... rdd.setName(..).values

is more readable

BTW, where is values defined?

(it seems defined here)

cloud-fan · 2017-02-24T19:30:09Z

sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/UnivocityParser.scala

what's expected?

cloud-fan · 2017-02-24T19:31:36Z

sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala

nit: wholeFile instead of bool

cloud-fan · 2017-02-24T19:33:54Z

sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala

so the string" will just become string?

cloud-fan · 2017-02-24T19:35:33Z

sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala

why we test the write path here? wholeFile is only about read path

I meant to test that we are able to read this back without wholeFile option if we remove all newlines.

// Check if the rows are the same if we remove all white spaces. val readBack = spark.read.csv(csvDir) checkAnswer(dfWithoutWhiteSpaces, readBack)

Let me clean up this one too.

HyukjinKwon · 2017-02-25T01:48:30Z

@cloud-fan, I am running a build with 2.10 per b4e6983. I think it is ready for another look.

HyukjinKwon · 2017-02-25T02:18:23Z

sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala

Here also simply I added..

Seq(false, true).foreach { wholeFile => ... .option("wholeFile", wholeFile) ... }

SparkQA · 2017-02-25T03:28:00Z

Test build #73454 has finished for PR 16976 at commit 321d082.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2017-02-25T03:38:44Z

Test build #73455 has finished for PR 16976 at commit b4e6983.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2017-02-25T03:59:58Z

Test build #73456 has finished for PR 16976 at commit 6187f6c.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2017-02-25T04:21:06Z

Test build #73457 has finished for PR 16976 at commit b7d56c6.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2017-02-26T16:51:00Z

Test build #73493 has finished for PR 16976 at commit 22eb29e.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
abstract class CSVDataSource extends Serializable

cloud-fan · 2017-02-28T21:33:35Z

sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala

+
+  test("Empty file produces empty dataframe with empty schema - wholeFile option") {
+    withTempPath { path =>
+      path.createNewFile()


actually you want withTempDir

cloud-fan · 2017-02-28T21:34:28Z

thanks, merging to master!

gatorsmile · 2017-06-05T02:07:56Z

sql/core/src/main/scala/org/apache/spark/sql/DataFrameReader.scala

   * <li>`columnNameOfCorruptRecord` (default is the value specified in
   * `spark.sql.columnNameOfCorruptRecord`): allows renaming the new field having malformed string
   * created by `PERMISSIVE` mode. This overrides `spark.sql.columnNameOfCorruptRecord`.</li>
+   * <li>`wholeFile` (default `false`): parse one record, which may span multiple lines.</li>


multiple lines. -> multiple lines, per file.

No, they are different. JSON reads whole file as a record (basically it is. If it is an array then it will be individual record) whereas CSV reads each record when it meets multiple lines in a column.

Still unable to get your point. Given an example?

Definitely. I wanted to emphasize multiple lines is not per file.

For example, CSV reads multiple records (multiple lines) per file (newline is replaced to \n manually for readability).

"I am Hyukjin Kwon" "Hyukjin Kwon I love Spark!"

scala> spark.read.option("wholeFile", true).csv("test.csv").show() +---------------------+ | _c0| +---------------------+ | I am\nHyukjin Kwon| |Hyukjin Kwon\nI lo...| +---------------------+

Whereas JSON reads the record per file. I am pretty sure object root support is primary.

{ "I am": "HyukjinKwon", "HyukjinKwon": "I love Spark!" }

scala> spark.read.option("wholeFile", true).json("test.json").show() +-------------+-----------+ | HyukjinKwon| I am| +-------------+-----------+ |I love Spark!|HyukjinKwon| +-------------+-----------+

but note that it could (in terms of input/output), work similarly with CSV when the input is a json array.

[{ "I am": "HyukjinKwon", "HyukjinKwon": "I love Spark!" },{ "I am": "HyukjinKwon", "HyukjinKwon": "I love Spark!" }]

scala> spark.read.option("wholeFile", true).json("test.json").show() +-------------+-----------+ | HyukjinKwon| I am| +-------------+-----------+ |I love Spark!|HyukjinKwon| |I love Spark!|HyukjinKwon| +-------------+-----------+

Comparing array case and CSV, they work still differently. JSON, up to my knowledge, parses whole files and produces each record (in case of an array) whereas CSV parses record by record from the stream.

Then, the option wholeFile is misleading to end users for csv. We should rename it to multiLine or others.

Yea, they are similar and different. I wouldn't mind opening a JIRA for this.

In JSON, when wholeFile is on, we only parse one and only one record (i.e., table rows) per file. The semantics are different. Let us fix the option name for CSV in 2.2.

gatorsmile · 2017-06-05T02:08:08Z

python/pyspark/sql/streaming.py

                                          ``spark.sql.columnNameOfCorruptRecord``. If None is set,
                                          it uses the value specified in
                                          ``spark.sql.columnNameOfCorruptRecord``.
+        :param wholeFile: parse one record, which may span multiple lines. If None is


The same here.

gatorsmile · 2017-06-05T02:08:24Z

python/pyspark/sql/readwriter.py

                                          ``spark.sql.columnNameOfCorruptRecord``. If None is set,
                                          it uses the value specified in
                                          ``spark.sql.columnNameOfCorruptRecord``.
+        :param wholeFile: parse records, which may span multiple lines. If None is


The same here.

gatorsmile · 2017-06-05T02:08:39Z

sql/core/src/main/scala/org/apache/spark/sql/streaming/DataStreamReader.scala

   * <li>`columnNameOfCorruptRecord` (default is the value specified in
   * `spark.sql.columnNameOfCorruptRecord`): allows renaming the new field having malformed string
   * created by `PERMISSIVE` mode. This overrides `spark.sql.columnNameOfCorruptRecord`.</li>
+   * <li>`wholeFile` (default `false`): parse one record, which may span multiple lines.</li>


The same here.

gatorsmile · 2017-06-05T02:32:20Z

sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala

+      primitiveFieldAndType.toDF("value").coalesce(1).write.text(path.getAbsolutePath)
+
+      val df = spark.read
+        .option("header", true)


When we have a set of files, how can we guarantee the reading order is what the users like? That means, the first file we read is the file that contains header?

Yea, that might be an issue. I guess this could be a general issue for CSV datasource itself. It sounds worth checking how it works.

Let us check out and open an issue. I will try to help verify it and resolve at my best together.

For non-whole-file parsing, we should skip the first line for each csv file (after we combine the partitioned files to a single file), right?

Ah, sorry, I got you. So, you mean what if other files still have the headers? It will skip. Univocity parser handles this (did I understood correctly?)

raviolli · 2017-09-28T17:36:54Z

Dont think it quite works.

spark.read.option("wholeFile", "true").option("header", "true").option("inferSchema", "true").csv("FILE.csv").show()
text is out of order

spark.read.option("wholeFile", "true").option("header", "true").option("inferSchema", "true").csv("data/FILE.csv").count()
311948

WHILE:
pandas says data count is 15860

HyukjinKwon · 2017-09-28T21:11:31Z

Can you open a JIRA with more description, expected input, and expected output?

smram · 2017-10-04T20:13:25Z

don't think it quite works

+1 i could not get this to work either @raviolli

Does it need the univocity parser to be specified as suggested initially here https://github.com/databricks/spark-csv/issues/175 ?

vishnusram · 2017-11-16T17:49:47Z

The wholeFile option doesn't seem to be working.
Test file(new_line.csv) content:
"num_col1","txt_col","num_col2"
10001,"regular string",20001
10002,"string with
newline",20002

Command and result:

dfu = sqlContext.read.format('com.databricks.spark.csv').option("header","true").option("inferschema","true").option("delimiter",",").option("quote",'"').option("parserLib","univocity").option("wholeFile","true").load('new_line.csv')

dfu.show(3,False)

+--------+--------------+--------+
|num_col1|txt_col |num_col2|
+--------+--------------+--------+
|10001 |regular string|20001 |
|10002 |string with |null |
|newline"|20002 |null |
+--------+--------------+--------+

Expected result:
+--------+--------------+--------+
|num_col1|txt_col |num_col2|
+--------+--------------+--------+
|10001 |regular string|20001 |
|10002 |string with\nnewline||20002 |
+--------+--------------+--------+

Spark version used 2.2
17/11/16 17:15:37 WARN ObjectStore: Failed to get database global_temp, returning NoSuchObjectException
Welcome to
____ __
/ / ___ / /
\ / _ / _ `/ __/ '/
/ / ._/_,// //_\ version 2.2.0
//

Using Python version 2.7.5 (default, May 3 2017 07:55:04)
SparkSession available as 'spark'.

Please let me know if I am missing something here

HyukjinKwon · 2017-11-16T20:22:14Z

It was renamed multiLine before the release. Could we try out it instead?

HyukjinKwon · 2017-11-16T20:28:57Z

Guys, please use the mailing list for a question and JIRA for filing issues ... it's not useful to raise arbitrary questions and issues for this option here.

BTW, please read the API documentation ahead and provide a self-contained reproducer if possible. For a question, please read this ahead SSCCE (http://sscce.org/).

vishnusram · 2017-11-17T00:48:56Z

Thanks Kwon. It worked

HyukjinKwon force-pushed the SPARK-19610 branch from 3a303c7 to 8275d12 Compare February 17, 2017 18:39

HyukjinKwon changed the title ~~[WIP][SPARK-19610][SQL] Support parsing multiline CSV files~~ [SPARK-19610][SQL] Support parsing multiline CSV files Feb 18, 2017

HyukjinKwon force-pushed the SPARK-19610 branch from 32cffa7 to 2ddbf15 Compare February 18, 2017 12:01

cloud-fan approved these changes Feb 23, 2017

View reviewed changes

cloud-fan reviewed Feb 23, 2017

View reviewed changes

NathanHowell reviewed Feb 23, 2017

View reviewed changes

HyukjinKwon commented Feb 23, 2017

View reviewed changes

cloud-fan requested changes Feb 24, 2017

View reviewed changes

HyukjinKwon commented Feb 25, 2017

View reviewed changes

HyukjinKwon force-pushed the SPARK-19610 branch from b7d56c6 to 1492d00 Compare February 26, 2017 14:48

Support parsing multiline CSV files

22eb29e

HyukjinKwon force-pushed the SPARK-19610 branch from 1492d00 to 22eb29e Compare February 26, 2017 14:49

cloud-fan reviewed Feb 28, 2017

View reviewed changes

asfgit closed this in 7e5359b Feb 28, 2017

gatorsmile reviewed Jun 5, 2017

View reviewed changes

HyukjinKwon deleted the SPARK-19610 branch January 2, 2018 03:38

lqshow mentioned this pull request Jan 7, 2018

Spark 2.2.0 读取csv单元格内双引号多行数据的问题 lqshow/notes#7

Open

[SPARK-19610][SQL] Support parsing multiline CSV files #16976

[SPARK-19610][SQL] Support parsing multiline CSV files #16976

Uh oh!

Conversation

HyukjinKwon commented Feb 17, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changes were proposed in this pull request?

How was this patch tested?

Uh oh!

HyukjinKwon commented Feb 17, 2017

Uh oh!

SparkQA commented Feb 17, 2017

Uh oh!

SparkQA commented Feb 17, 2017

Uh oh!

HyukjinKwon commented Feb 18, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

HyukjinKwon commented Feb 18, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

SparkQA commented Feb 18, 2017

Uh oh!

SparkQA commented Feb 18, 2017

Uh oh!

SparkQA commented Feb 18, 2017

Uh oh!

felixcheung commented Feb 19, 2017

Uh oh!

HyukjinKwon commented Feb 19, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

felixcheung commented Feb 21, 2017

Uh oh!

cloud-fan left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

HyukjinKwon commented Feb 23, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

HyukjinKwon Feb 23, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

HyukjinKwon commented Feb 17, 2017 •

edited

Loading

HyukjinKwon commented Feb 18, 2017 •

edited

Loading

HyukjinKwon commented Feb 18, 2017 •

edited

Loading

HyukjinKwon commented Feb 19, 2017 •

edited

Loading

HyukjinKwon Feb 23, 2017 •

edited

Loading

cloud-fan Feb 23, 2017 •

edited

Loading

HyukjinKwon Feb 23, 2017 •

edited

Loading