Python Pandas to_pickle() Method

---

---

The to_pickle() method in Python's Pandas library allows you to serialize the Pandas objects, such as DataFrame, or Series, into a file or file-like object in the pickle format. This allows you to save your data and load it later for future use. This method can also handle compression, support different file formats, and allow customization using various parameters.

Pickle is a Python-specific file format used for serializing and deserializing Python objects. Serialization, also known as pickling, refers to the process of converting a Python Pandas object (like DataFrame or Series) into a byte stream, which can be stored or transmitted.

Syntax

Following is the syntax of the Python Pandas to_pickle() method −

 DataFrame.to_pickle(path, *, compression='infer', protocol=5, storage_options=None) 

When using the to_pickle() method on a Series object, you should call it as Series.to_pickle().

Parameters

The Python Pandas to_pickle() method accepts the below parameters −

• path; This parameter accepts a string, path object, or file-like object, representing the location where the pickled object will be stored.

• compression: Specifies the compression method to use. If set to 'infer', the method will automatically detect the compression type based on the file extension (e.g., .gz, .bz2, .zip). You can also pass a dictionary to customize compression methods such as gzip, zip, bz2, zstd, etc. If set to None, no compression will be applied.

• protocol: This parameter takes an integer indicating which pickle protocol to use for serializing the object. The default is the HIGHEST_PROTOCOL. The possible values are 0, 1, 2, 3, 4, and 5. If a negative value is provided, it is equivalent to using the default protocol.

• storage_options: Additional options for connecting to certain storage back-ends (e.g., AWS S3, Google Cloud Storage).

Return Value

The Pandas to_pickle() method returns None, but saves the DataFrame or Series as a serialized object at the specified path.

Example: Saving a DataFrame to a Pickle File

Here is a basic example demonstrating saving a Pandas DataFrame object into a pickle file using the DataFrame.to_pickle() method.

 import pandas as pd # Create a DataFrame df = pd.DataFrame({"Col_1": range(5), "Col_2": range(5, 10)}) print("Original DataFrame:") print(df) # Save the DataFrame as a pickle file df.to_pickle("df_pickle_file.pkl") print("\nDataFrame is successfully saved as a pickle file.") 

When we run above program, it produces following result −

 Original DataFrame: 

	Col_1	Col_2
0	0	5
1	1	6
2	2	7
3	3	8
4	4	9

DataFrame is successfully saved as a pickle file.

If you check the folder where the pickle file was saved, you will find the generated file.

Example: Saving Pandas Series to a Pickle File

This example saves a Pandas Series object into a pickle file using the Series.to_pickle() method.

 import pandas as pd # Creating a Pandas Series s = pd.Series([1, 2, 3, 4], index=["cat", "dog", "fish", "mouse"]) # Display the Input Series print("Original Series:") print(s) # Save the Series as a pickle file s.to_pickle("series_to_pickle_file.pkl") print("\nPandas Series is successfully saved as a pickle file.") 

While executing the above code we get the following output −

 Original Series: cat 1 dog 2 fish 3 mouse 4 dtype: int64 Pandas Series is successfully saved as a pickle file. 

Example: Saving pickle file with Compression

The following example demonstrates how to use the to_pickle() method to compress a Pandas DataFrame using gzip compression.

 import pandas as pd # Create a DataFrame df = pd.DataFrame({"Col_1": range(5), "Col_2": range(5, 10)}) print("Original DataFrame:") print(df) # Save the DataFrame to a pickle file with gzip compression df.to_pickle("dataframe_compressed.pkl", compression="gzip") print("\nDataFrame is successfully saved as a pickle file with gzip compression.") 

Following is an output of the above code −

 Original DataFrame: 

	Col_1	Col_2
0	0	5
1	1	6
2	2	7
3	3	8
4	4	9

DataFrame is successfully saved as a pickle file with gzip compression.

Example: Saving Pickle with a Custom Compression Method

The to_pickle() method can also accepts a dictionary for customizing the compression method. Here, we apply a custom compression method (zip) with specific compression level.

 import pandas as pd # Create a DataFrame df = pd.DataFrame({"Col_1": [1, 2, 3, 4, 5], "Col_2": ["a", "b", "c", "d", "e"]}) print("Original DataFrame:") print(df) # Save the DataFrame to a pickle file with custom zip compression df.to_pickle("dataframe_custom_compressed.pkl", compression={'method': 'zip', 'compresslevel': 2}) print("\nDataFrame is successfully saved as a pickle file with custom zip compression.") 

Following is an output of the above code −

 Original DataFrame: 

	Col_1	Col_2
0	1	a
1	2	b
2	3	c
3	4	d
4	5	e

DataFrame is successfully saved as a pickle file with custom zip compression.