I'm exploring file storage format options for Python and stumbled on feather. I noticed the last release was back in 2017 and was concerned about its long term existence.
Web searches are pulling back posts that all seem to stop around 2017.
Asked
Active
Viewed 3,007 times
1 Answers
6
The feather format is still relevant and support for more data types, especially on the R side has improved a lot recently. A remarkable change is that it is no longer released as a separate package but comes as part of arrow / https://arrow.apache.org/. There it is actively developed.
The other alternative format that the community is leading towards is Apache Parquet. There are some differences between feather and Parquet so that you may choose one over the other, e.g. Feather writes the data as-is and Parquet encodes and compresses it to achieve much smaller files. Additionally Parquet is also available in the Java world which might come in handy. Feather and Parquet are both available in R in the arrow library and in Python as part of pyarrow.
Micah Kornfield
- 1,325
- 5
- 10
Uwe L. Korn
- 8,080
- 1
- 30
- 42