Skip to content

Conversation

@sarutak
Copy link
Member

@sarutak sarutak commented Sep 29, 2023

This PR proposes to update nested_recods.avro to support nullable records.
This change is necessary for this PR.

This change appends new fields f3 and f4 to the existing schema (the last two fields).
f3 is nullable record, and f4 is array of nullable record.

{
    "name": "record1",
    "namespace": "ns1",
    "type": "record",
    "fields": [
        {
            "name": "f1",
            "type": {
                "name": "record2",
                "namespace": "ns2",
                "type": "record",
                "fields": [
                    {
                        "name": "f1_1",
                        "type": "string"
                    },  {
                        "name": "f1_2",
                        "type": "int"
                    },  {
                        "name": "f1_3",
                        "type": {
                            "name": "record3",
                            "namespace": "ns3",
                            "type": "record",
                            "fields": [
                                {
                                    "name": "f1_3_1",
                                    "type": "double"
                                }
                            ]
                        }
                    }
                ]
            }
        },  {
            "name": "f2",
            "type": "array",
            "items": {
                "name": "record4",
                "namespace": "ns4",
                "type": "record",
                "fields": [
                    {
                        "name": "f2_1",
                        "type": "boolean"
                    },  {
                        "name": "f2_2",
                        "type": "float"
                    }
                ]
            }
        },  {
            "name": "f3",
            "type": [
                "null",
                {
                    "name": "record5",
                    "namespace": "ns5",
                    "type": "record",
                    "fields": [
                        {
                            "name": "f3_1",
                            "type": "string"
                        }
                    ]
                }
            ],
            "default": null
        },  {
            "name": "f4",
            "type": "array",
            "items": [
                "null",
                {
                    "name": "record6",
                    "namespace": "ns6",
                    "type": "record",
                    "fields": [
                        {
                            "name": "f4_1",
                            "type": "long"
                        }
                    ]
                }
            ]
        }
    ]
}

And the data represented in JSON is as follows.

{"f1":{"f1_1":"aaa","f1_2":10,"f1_3":{"f1_3_1":3.14}},"f2":[{"f2_1":true,"f2_2":1.2000000476837158},{"f2_1":true,"f2_2":2.200000047683716}],"f3":{"f3_1":"xyz"},"f4":[{"f4_1":200},null]}
{"f1":{"f1_1":"bbb","f1_2":20,"f1_3":{"f1_3_1":3.14}},"f2":[{"f2_1":false,"f2_2":10.199999809265137}],"f3":null,"f4":[null,{"f4_1":300}]}

@sarutak
Copy link
Member Author

sarutak commented Sep 29, 2023

@Samrose-Ahmed
Is this data OK for writing tests for that PR?

@Samrose-Ahmed
Copy link

Yes it is OK

@alamb
Copy link
Contributor

alamb commented Oct 1, 2023

Thank you @sarutak

@alamb alamb merged commit 48672a3 into apache:master Oct 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants