Python Pandas to_orc() Method

---

---

The ORC (Optimized Row Columnar) format is a binary columnar serialization designed for efficient reading and writing of DataFrames, similar to the Parquet Format. It provides compact and high-speed columnar data storage, facilitating efficient data exchange across different data analysis tools.

The Pandas DataFrame.to_orc() method allows you to save DataFrames in the ORC format, enabling efficient data storage and sharing across different tools. However, there are some limitations −

• Timezone handling: Timezones in datetime columns are not preserved when saving DataFrames in ORC format.

• Platform support: Windows operating system is not supported for to_orc() and read_orc() as of now.

Note: The to_orc() method requires the pyarrow library (version >= 7.0.0). It is highly recommended to install the pyarrow library using Conda package manager to prevent compatibility issues. You can install it using the following command −

 conda install pyarrow 

Syntax

Following is the syntax of the Python Pandas to_orc() method −

 DataFrame.to_orc(path=None, *, engine='pyarrow', index=None, engine_kwargs=None) 

Parameters

The Python Pandas DataFrame.to_orc() method accepts the below parameters −

• path: Specifies the file path or file-like object where the ORC file will be saved. If None, a bytes object is returned.

• engine: It specifies ORC library to use. Currently, only 'pyarrow' is supported.

• index: Whether to include the DataFrame's index in the output ORC file. If set to False exclude the index. If True, Includes the index in the output file. By default it is set to None, indexes other than RangeIndex will be included as columns.

• engine_kwargs: Additional keyword arguments passed to pyarrow.orc.write_table().

Return Value

The Pandas DataFrame.to_orc() method returns bytes object, if path parameter is set to None. Otherwise, returns None but saves the DataFrame as an ORC file at the specified path.

Errors

• The to_orc() method raises a NotImplementedError if the DataFrame columns contains unsupported dtypes, such as category, unsigned integers, interval, period, or sparse data.

• ValueError is raised if the pyarrow engine is not used.

Example: Saving a DataFrame as an ORC File

Here is a basic example demonstrating saving a Pandas DataFrame object into an ORC file format using the DataFrame.to_orc() method.

 import pandas as pd # Create a DataFrame df = pd.DataFrame({"Col_1": range(5), "Col_2": range(5, 10)}) print("Original DataFrame:") print(df) # Save the DataFrame as an ORC file df.to_orc("df_orc_file.orc") print("\nDataFrame is successfully saved as an ORC file.") 

When we run above program, it produces following result −

 Original DataFrame: 

	Col_1	Col_2
0	0	5
1	1	6
2	2	7
3	3	8
4	4	9

DataFrame is successfully saved as an ORC file.

If you visit the path where the ORC files are saved, you can observe the generated orc file.

Example: Save Pandas DataFrame to In-Memory orc

This example saves a Pandas DataFrame object into a in-memory ORC file using the DataFrame.to_orc() method.

 import pandas as pd import io # Create a DataFrame df = pd.DataFrame(data={"Col1": [1, 2], "Col2": [3.0, 4.0]}) print("Original DataFrame:") print(df) # Save the DataFrame to an in-memory buffer buffer = io.BytesIO() df.to_orc(buffer) # Read the ORC file from the buffer output = pd.read_orc(buffer) print("\nDataFrame saved as an in-memory ORC file:") print(output) 

While executing the above code we get the following output −

 Original DataFrame: 

	Col_1	Col_2
0	1	3.0
1	2	4.0

Saved DataFrame as an in-memory orc file:

	Col_1	Col_2
0	1	3.0
1	2	4.0

Example: Saving ORC file Without Index

The ORC format does not support serializing custom indexes directly. To avoid the errors, you can reset the DataFrame index before saving it as an ORC file.

 import pandas as pd # Create a DataFrame with an index df = pd.DataFrame({"Col1": [1, 2, 3], "Col2": ["a", "b", "c"]}, index=["r1", "r2", "r3"]) print("Original DataFrame:") print(df) # Save the DataFrame without its index df.reset_index().to_orc("data_no_index.orc") # Read the file output = pd.read_orc("data_no_index.orc") print("\nSaved DataFrame without index:") print(output) 

Following is an output of the above code −

 Original DataFrame: 

	Col_1	Col_2
r1	1	a
r2	2	b
r3	3	c

Saved DataFrame without index: index

		Col_1	Col_2
0	r1	1	a
1	r2	2	b
2	r3	3	c