Skip to content

Conversation

@cloud-fan
Copy link
Contributor

Implement encode/decode for external row based on ClassEncoder.

TODO:

  • code cleanup
  • fix corner cases
  • refactor the encoder interface
  • improve test for product codegen, to cover more corner cases.

@SparkQA
Copy link

SparkQA commented Oct 20, 2015

Test build #43991 has finished for PR 9184 at commit 4751418.

  • This patch fails to build.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):\n * case class CreateRow(children: Seq[Expression]) extends Expression\n

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets put this in its own test suite.

@marmbrus
Copy link
Contributor

This is looking great! Thanks for coming up with a draft so quickly.

@SparkQA
Copy link

SparkQA commented Oct 21, 2015

Test build #44058 has finished for PR 9184 at commit b08da16.

  • This patch fails Scala style tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):\n * case class ClassEncoder[T](\n * case class CreateRow(children: Seq[Expression]) extends Expression\n

@SparkQA
Copy link

SparkQA commented Oct 21, 2015

Test build #44061 has finished for PR 9184 at commit 4551a92.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):\n * case class ClassEncoder[T](\n * case class CreateRow(children: Seq[Expression]) extends Expression\n

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this really the right thing to do? I think we are going to NPE execution. If anything I think we'd want to null out all the columns, though I'm not sure we need to handle this case at all.

@marmbrus
Copy link
Contributor

This looks great. I'm going to merge so that I can resolve any conflicts with #9190. We can continue to address comments / TODOs in a follow up PR.

@asfgit asfgit closed this in ccf536f Oct 21, 2015
asfgit pushed a commit that referenced this pull request Oct 22, 2015
address comments in #9184

Author: Wenchen Fan <[email protected]>

Closes #9212 from cloud-fan/encoder.
kiszk pushed a commit to kiszk/spark-gpu that referenced this pull request Dec 26, 2015
address comments in apache/spark#9184

Author: Wenchen Fan <[email protected]>

Closes #9212 from cloud-fan/encoder.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants