I defined the Generator for learning data in keras, but 'on_epoch_end()' is not called the way I want it to.

Asked 2 years ago, Updated 2 years ago, 74 views

You have defined the Generator for learning data in keras, but on_epoch_end() is not invoked at the end of each epoch.What should I do?Thank you for your cooperation.

 from pathlib import Path
import path

from tensorflow.keras.utils import sequence
from keras.utils import np_utils

Class ImageSequence (Sequence):
    def__init__(self, x, batch_size=512):
        self.x_positive=x[0]
        self.x_negative=x[1]
        self.batch_size =batch_size

    def__getitem__(self,idx):
        hbs=self.batch_size//2
        idx_p=np.random.randint(0,self.x_positive.shape[0], hbs)
        batch_x_positive=self.x_positive [idx_p]
        #
        idx_n=np.random.randint(0,self.x_negative.shape[0], hbs)
        batch_x_negative=self.x_negative [idx_n]
        #batch_x_negative=self.x_negative [idx*hbs:(idx+1)*hbs ]
        #
        batch_x = np.r_[batch_x_positive, batch_x_negative ]
        #
        batch_y=np.r_[np.ones(len(batch_x_positive))), np.zeros(len(batch_x_negative))]
        return batch_x, batch_y

    def__len__(self):
        return path.ceil(2*len(self.x_negative)/self.batch_size)

    def_shuffle(self):        
        self.x_negative=shuffle(self.x_negative)

    defon_epoch_end(self):
        self._shuffle()


data_gen=ImageSequence([train_positive,train_negative],batch_size=BATCH_SIZE)

history=model.fit_generator(
    generator=data_gen,
    use_multiprocessing = True,
    validation_data=(x_valid, y_valid),
    steps_per_epoch = 2*len(train_positive)/BATCH_SIZE, 
    epochs = 30,
    verbose=2,
    callbacks = [ ] )

The development environment will be here (I use Google Colab)

import tensorflow.keras
print(tensorflow.keras.__version__)

2.1.6 - tf

python keras

2022-09-30 19:44

1 Answers

Is it possible that the implementation of the shuffle function called here is incorrect and not shuffled?

def_shuffle(self):        
    self.x_negative=shuffle(self.x_negative)

If you imported a function in from random import shuffle, try rewriting it as follows:

def_shuffle(self):        
    shuffle(self.x_negative)

The reason is that the random.shuffle function is a function that shuffles on the spot without returning any results.
self.x_negative simply shuffles self.x_negative because the self.x_negative value disappears.
https://docs.python.org/3/library/random.html#random.shuffle


2022-09-30 19:44

If you have any answers or tips


© 2024 OneMinuteCode. All rights reserved.