Compare
Arrow vs ORC
Apache Arrow is a language-independent columnar memory representation for analytics and data interchange. ORC is a columnar file format that organizes rows into stripes with column data, indexes, and statistics to support analytical reads.
Which format should you choose?
Choose Arrow to efficiently pass tabular data between processes or systems in memory. Consider ORC for persistent data-lake or Hive-oriented workloads that benefit from compression, column projection, and stripe reads. Do not confuse Arrow memory layout with Arrow IPC files.
Good fit for Apache Arrow IPC
Exchanging columnar table data between compatible programs and languages · Passing analytical data sets between Arrow-aware tools
- Moving typed data between Arrow libraries and analytical engines
- Random access to record batches in the IPC file format
Good fit for ORC
Store large structured tables for analytics
- Queries reading selected columns and using indexes
Basic information
| Field | Apache Arrow IPC | ORC |
|---|---|---|
| Extension | .arrow, .feather, .arrows | .orc |
| MIME type | application/vnd.apache.arrow.file, application/vnd.apache.arrow.stream | application/vnd.apache.orc |
| Common uses | Exchanging columnar table data between compatible programs and languages · Passing analytical data sets between Arrow-aware tools | Store large structured tables for analytics |
| How to open | Read it with an Apache Arrow library or an analytics tool that supports Arrow IPC. The .arrow extension commonly indicates the file format and .arrows the stream format, but verify what the receiving tool expects and which compression codecs it supports. For Feather files, check compatibility with the older Feather V1 format. | Use an ORC-aware tool to inspect schema, columns, row counts, compression, and sample values. |
| Export or convert | Parquet may suit analytical storage and delivery; CSV may suit simple table exchange. Conversion can change or discard Arrow-specific types, nested structures, dictionary encoding, or compression settings. Check required types and precision, and retain the source data. | When converting to Parquet or CSV, verify types, nulls, timestamps, decimals, partitions, and row counts. |
Choose by use case
Moving typed data between Arrow libraries and analytical engines
Exchanging columnar table data between compatible programs and languages · Passing analytical data sets between Arrow-aware tools
Apache Arrow IPCRandom access to record batches in the IPC file format
Exchanging columnar table data between compatible programs and languages · Passing analytical data sets between Arrow-aware tools
Apache Arrow IPCQueries reading selected columns and using indexes
Store large structured tables for analytics
ORCKey point 1
Arrow specifies in-memory arrays, buffers, and types, enabling approaches such as zero-copy sharing. ORC files use column streams, stripes, indexes, and footer statistics.
More on the differencesKey point 2
Arrow centers on efficient in-memory representation and transport; ORC is designed for compressed persistent storage and selective analytical reads.
More on the differencesMore on the differences
Use Arrow for interoperable in-memory analytics and transfers. Use ORC when the storage and query ecosystem supports its file layout and benefits from column pruning and indexes. Benchmark representative workloads.
- Arrow defines columnar arrays and buffers in memory. Arrow IPC/file formats serialize that representation; ORC defines a separate on-disk structure with stripes and column streams.
- ORC can use compression, column projection, and stripe or row indexes to reduce data read for queries. Arrow's main specification is a memory format, not a replacement for every persistent table format.
- Check reader/writer support and mappings for types, nulls, timestamps, and decimals. Compare file size and query performance using your codecs, engines, and workload.
Frequently asked questions
Is an Arrow table file the same format as ORC?
No. Arrow defines a columnar memory format and separate IPC formats; ORC is its own columnar file format.