EASILY CALLING VARIABLE IN EXECUTION

Asked 1 years ago, Updated 1 years ago, 72 views

I use Spyder in Python 3.
Is there a way to save and invoke variables of huge size that are obtained by executing code somewhere?
The background is a loop statement that returns millions of lines of results, and of course the variable disappears every time I change a file, and I run this loop statement again when I start over.This loop statement that has already been executed and the results are known has been taken up for an hour or two every time.

python python3 spyder

2022-09-30 21:35

1 Answers

Python has a module that can store objects called pickles (pickles translated into Japanese) at high speed.
data allows you to easily save and read the following code:
· Official documentation pickle

import pickle

with open('data.pickle', 'wb') as f:
    pickle.dump(data,f,pickle.HIGHEST_PROTOCOL)

with open('data.pickle','rb') asf:
    data=pickle.load(f)

If the variable of a large size is Pandas' DataFrame, it is easier to save and load in a pickle.

df.to_pickle('data.pickle')
df = pd.read_pickle('data.pickle')

In the case of numpy ndarray, it is convenient to use it because it can be saved in binary format.If you try using the code below, you can save it in seconds and load it in less than a second.If you use a pickle, the processing time is about the same.

import numpy as np

na=np.random.rand (10000000,10)
np.save ('data.npy', na)
# If you want to compress it
# np.savez ('data.npz', na)

na=np.load('data.npy')


2022-09-30 21:35

If you have any answers or tips


© 2024 OneMinuteCode. All rights reserved.