Consider adding support batches

[javierluraschi]

@jimhester I'm planning to step out of Arrow work for a bit, so I'll brain dump what I found here.

From looking at the Arrow sources, specially the Java ones, looks like column bindings would be helpful but not sufficient to interoperate with other platforms like JAVA which expects higher level objects. For instance, we would want support for /arrow/format/Message.fbs which supports column binding but also batching, etc.

I haven't identified which are the API entry points in the Arrow C API, but you can find an example of what a client would need to execute if we had an R API. Code is written in reticulate:

sparklyr/R/arrow_data.R#L45-L65 (mostly, pa$RecordBatch, pa$schema and pa$RecordBatchFileWriter).

In case you need this, there is some executable code in sparklyr as a PoC:

sparklyr/sparklyr#1611

[Namanm1994]

@jimhester hey! Does this code provide us with functionality of reading a parquet file in R using arrow. ??
Or we would have to work on that ?

Thanks.