Compare
netCDF vs Parquet
netCDF describes scientific datasets with named dimensions, variables, and attributes that can relate coordinates and time. Parquet is a columnar file format with row groups and column chunks, suited to selecting columns in analytical queries.
Which format should you choose?
Choose netCDF for scientific arrays with time, levels, coordinates, and related variables. Consider Parquet for tabular records queried by selected columns.
Good fit for netCDF
Storing and exchanging array-based climate, weather, and Earth-science data
- Describing multidimensional data with dimensions, variables, and attributes
- Portable scientific data workflows using netCDF-aware tools
Good fit for Parquet
Store structured tables for analytics
- Large datasets read by selected columns
Basic information
| Field | netCDF | Parquet |
|---|---|---|
| Extension | .nc, .nc4, .cdf | .parquet |
| MIME type | application/vnd.apache.parquet | |
| Common uses | Storing and exchanging array-based climate, weather, and Earth-science data | Store structured tables for analytics |
| How to open | Use netCDF-aware visualization or analysis software or a library. Inspect variables, dimensions, attributes, and the file variant. Check coordinate variables, units, calendar, missing values, and any CF conventions against what your reader expects. | Use a Parquet-aware tool to inspect schema, column types, row counts, and values. |
| Export or convert | When converting to CSV, HDF5, Zarr, or another format, check whether dimensions, variable types, attributes, coordinates, and time calendars are preserved. Conversion to the classic model may lose extended features; CSV cannot fully represent hierarchy, types, and attributes. | From CSV or other formats, check types, nulls, timestamps, encoding, partitions, and row counts. |
Choose by use case
Describing multidimensional data with dimensions, variables, and attributes
Storing and exchanging array-based climate, weather, and Earth-science data
netCDFPortable scientific data workflows using netCDF-aware tools
Storing and exchanging array-based climate, weather, and Earth-science data
netCDFLarge datasets read by selected columns
Store structured tables for analytics
ParquetKey point 1
netCDF variables are arrays associated with named dimensions and may have coordinate variables and attributes; Parquet locates column chunks within row groups using file metadata.
More on the differencesKey point 2
netCDF has Classic and enhanced models, with groups and additional types in netCDF-4; Parquet encodes tabular columns in pages and row groups.
More on the differencesMore on the differences
Use netCDF when scientific meaning depends on array dimensions, coordinates, time, and units. Use Parquet for analytical table workloads. Map the models explicitly when converting, rather than flattening scientific relationships implicitly.
- netCDF associates N-dimensional variables with named dimensions, coordinates, and attributes. Parquet organizes table data into column chunks within row groups.
- netCDF-4 may support groups and types unavailable to classic-model readers. Parquet metadata locates selected columns so readers can avoid unrelated column data.
- On conversion validate variable-to-column mapping, dimensions, units, coordinates and CRS, time, missing values, types, groups, and nested types. Tune compression and row-group layout for the target reads.
Frequently asked questions
Can Parquet preserve a netCDF scientific dataset without mapping?
No. Define how dimensions, coordinates, units, time, missing values, and variables map to rows and columns, then validate the converted data.