0

I am trying to implement a simple image classification model on cifar10 using tensorflow. Pycharm crashes automatically without giving a reason. I tried other NLP models it works fine, but when I execute any vision models the problem persists.

My specification:

Windows 11

NVIDIA GTX 2060 - 6GB

MEMORY- 16GB

Tensorflow 2.9.0

# Cifar 10 neural network

import tensorflow as tf
import tensorflow_datasets as tfds
import os

os.environ['TF_CPP_MIN_LOG_LEVEL'] = "2"

if __name__ == '__main__':

    (ds_train, ds_test), ds_info = tfds.load(
        name='cifar10',
        split=['train', 'test'],
        shuffle_files=True,
        as_supervised=True,
        with_info=True
    )
    print(ds_info)

    print(tf.config.list_physical_devices('GPU'))
    for img, label in ds_train.take(1):
        print(img.shape)
        print(label.shape)

    train_size = int(0.8 * tf.data.experimental.cardinality(ds_train).numpy())

    train = ds_train.take(train_size)
    val = ds_train.skip(train_size)

    autotune = tf.data.AUTOTUNE
    train = train.cache().shuffle(1000).batch(32).prefetch(autotune)
    val = val.cache().batch(32).prefetch(autotune)
    test = ds_test.cache().batch(32).prefetch(autotune)

    for img, label in train.take(1):
        print(img.shape)
        print(label.shape)

    augmentation = tf.keras.Sequential([
        tf.keras.layers.RandomRotation(factor=0.2),
        tf.keras.layers.RandomContrast(factor=0.4)
    ])

    rescaled = tf.keras.layers.Rescaling(1. / 255)
    IMG_HEIGHT = IMG_WIDTH = 32
    NUM_CLASSES = 10

    model = tf.keras.Sequential([
        tf.keras.layers.Input(shape=(IMG_HEIGHT, IMG_WIDTH, 3)),
        rescaled,
        augmentation,
        tf.keras.layers.Conv2D(16, 3, padding='SAME'),
        tf.keras.layers.MaxPooling2D(),
        tf.keras.layers.Conv2D(32, 3, padding='SAME'),
        tf.keras.layers.MaxPooling2D(),
        tf.keras.layers.Conv2D(64, 3, padding='SAME'),
        tf.keras.layers.Flatten(),
        tf.keras.layers.Dense(128, activation='relu'),
        tf.keras.layers.Dense(NUM_CLASSES)
    ])

    print(model.summary())

    model.compile(loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
                  optimizer=tf.keras.optimizers.Adam(learning_rate=0.001),
                  metrics=['accuracy'])

    earlystop = tf.keras.callbacks.EarlyStopping(patience=15)
    checkpoint = tf.keras.callbacks.ModelCheckpoint('best_cifar10/', save_best_only=True)

    history = model.fit(train, validation_data=val, epochs=150, callbacks=[earlystop, checkpoint])

Output:

D:\Gopu\PycharmProject\TFEXAM\venv\Scripts\python.exe D:/Gopu/PycharmProject/TFEXAM/cifar10.py
tfds.core.DatasetInfo(
    name='cifar10',
    full_name='cifar10/3.0.2',
    description="""
    The CIFAR-10 dataset consists of 60000 32x32 colour images in 10 classes, with 6000 images per class. There are 50000 training images and 10000 test images.
    """,
    homepage='https://www.cs.toronto.edu/~kriz/cifar.html',
    data_path='~\\tensorflow_datasets\\cifar10\\3.0.2',
    file_format=tfrecord,
    download_size=162.17 MiB,
    dataset_size=132.40 MiB,
    features=FeaturesDict({
        'id': Text(shape=(), dtype=tf.string),
        'image': Image(shape=(32, 32, 3), dtype=tf.uint8),
        'label': ClassLabel(shape=(), dtype=tf.int64, num_classes=10),
    }),
    supervised_keys=('image', 'label'),
    disable_shuffling=False,
    splits={
        'test': <SplitInfo num_examples=10000, num_shards=1>,
        'train': <SplitInfo num_examples=50000, num_shards=1>,
    },
    citation="""@TECHREPORT{Krizhevsky09learningmultiple,
        author = {Alex Krizhevsky},
        title = {Learning multiple layers of features from tiny images},
        institution = {},
        year = {2009}
    }""",
)
[PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU')]
(32, 32, 3)
()
(32, 32, 32, 3)
(32,)
WARNING:tensorflow:Using a while_loop for converting RngReadAndSkip
WARNING:tensorflow:Using a while_loop for converting RngReadAndSkip
WARNING:tensorflow:Using a while_loop for converting Bitcast
WARNING:tensorflow:Using a while_loop for converting Bitcast
WARNING:tensorflow:Using a while_loop for converting Bitcast
WARNING:tensorflow:Using a while_loop for converting Bitcast
WARNING:tensorflow:Using a while_loop for converting StatelessRandomUniformFullIntV2
WARNING:tensorflow:Using a while_loop for converting StatelessRandomUniformFullIntV2
WARNING:tensorflow:Using a while_loop for converting StatelessRandomGetKeyCounter
WARNING:tensorflow:Using a while_loop for converting StatelessRandomGetKeyCounter
WARNING:tensorflow:Using a while_loop for converting StatelessRandomUniformV2
WARNING:tensorflow:Using a while_loop for converting StatelessRandomUniformV2
WARNING:tensorflow:Using a while_loop for converting AdjustContrastv2
WARNING:tensorflow:Using a while_loop for converting AdjustContrastv2
WARNING:tensorflow:Using a while_loop for converting RngReadAndSkip
WARNING:tensorflow:Using a while_loop for converting RngReadAndSkip
WARNING:tensorflow:Using a while_loop for converting Bitcast
WARNING:tensorflow:Using a while_loop for converting Bitcast
WARNING:tensorflow:Using a while_loop for converting Bitcast
WARNING:tensorflow:Using a while_loop for converting Bitcast
WARNING:tensorflow:Using a while_loop for converting StatelessRandomUniformFullIntV2
WARNING:tensorflow:Using a while_loop for converting StatelessRandomUniformFullIntV2
WARNING:tensorflow:Using a while_loop for converting StatelessRandomGetKeyCounter
WARNING:tensorflow:Using a while_loop for converting StatelessRandomGetKeyCounter
WARNING:tensorflow:Using a while_loop for converting StatelessRandomUniformV2
WARNING:tensorflow:Using a while_loop for converting StatelessRandomUniformV2
WARNING:tensorflow:Using a while_loop for converting AdjustContrastv2
WARNING:tensorflow:Using a while_loop for converting AdjustContrastv2
Model: "sequential_1"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 rescaling (Rescaling)       (None, 32, 32, 3)         0         
                                                                 
 sequential (Sequential)     (None, 32, 32, 3)         0         
                                                                 
 conv2d (Conv2D)             (None, 32, 32, 16)        448       
                                                                 
 max_pooling2d (MaxPooling2D  (None, 16, 16, 16)       0         
 )                                                               
                                                                 
 conv2d_1 (Conv2D)           (None, 16, 16, 32)        4640      
                                                                 
 max_pooling2d_1 (MaxPooling  (None, 8, 8, 32)         0         
 2D)                                                             
                                                                 
 conv2d_2 (Conv2D)           (None, 8, 8, 64)          18496     
                                                                 
 flatten (Flatten)           (None, 4096)              0         
                                                                 
 dense (Dense)               (None, 128)               524416    
                                                                 
 dense_1 (Dense)             (None, 10)                1290      
                                                                 
=================================================================
Total params: 549,290
Trainable params: 549,290
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/150
WARNING:tensorflow:Using a while_loop for converting RngReadAndSkip
WARNING:tensorflow:Using a while_loop for converting RngReadAndSkip
WARNING:tensorflow:Using a while_loop for converting Bitcast
WARNING:tensorflow:Using a while_loop for converting Bitcast
WARNING:tensorflow:Using a while_loop for converting Bitcast
WARNING:tensorflow:Using a while_loop for converting Bitcast
WARNING:tensorflow:Using a while_loop for converting StatelessRandomUniformFullIntV2
WARNING:tensorflow:Using a while_loop for converting StatelessRandomUniformFullIntV2
WARNING:tensorflow:Using a while_loop for converting StatelessRandomGetKeyCounter
WARNING:tensorflow:Using a while_loop for converting StatelessRandomGetKeyCounter
WARNING:tensorflow:Using a while_loop for converting StatelessRandomUniformV2
WARNING:tensorflow:Using a while_loop for converting StatelessRandomUniformV2
WARNING:tensorflow:Using a while_loop for converting AdjustContrastv2
WARNING:tensorflow:Using a while_loop for converting AdjustContrastv2
WARNING:tensorflow:Using a while_loop for converting RngReadAndSkip
WARNING:tensorflow:Using a while_loop for converting RngReadAndSkip
WARNING:tensorflow:Using a while_loop for converting Bitcast
WARNING:tensorflow:Using a while_loop for converting Bitcast
WARNING:tensorflow:Using a while_loop for converting Bitcast
WARNING:tensorflow:Using a while_loop for converting Bitcast
WARNING:tensorflow:Using a while_loop for converting StatelessRandomUniformFullIntV2
WARNING:tensorflow:Using a while_loop for converting StatelessRandomUniformFullIntV2
WARNING:tensorflow:Using a while_loop for converting StatelessRandomGetKeyCounter
WARNING:tensorflow:Using a while_loop for converting StatelessRandomGetKeyCounter
WARNING:tensorflow:Using a while_loop for converting StatelessRandomUniformV2
WARNING:tensorflow:Using a while_loop for converting StatelessRandomUniformV2
WARNING:tensorflow:Using a while_loop for converting AdjustContrastv2
WARNING:tensorflow:Using a while_loop for converting AdjustContrastv2

Process finished with exit code -1073740791 (0xC0000409)
Christoph Rackwitz
  • 11,317
  • 4
  • 27
  • 36
Na462
  • 11
  • 2
  • your *pycharm* didn't *crash*. the python process crashed. – Christoph Rackwitz Aug 03 '22 at 11:50
  • `0xC0000409` means `STATUS_STACK_BUFFER_OVERRUN`. see if that helps you. – Christoph Rackwitz Aug 03 '22 at 11:51
  • I tried reducing batch size, buffer size and also clearing the keras backend session, still doesnt work. Any ideas? – Na462 Aug 03 '22 at 15:36
  • I'd guess something in the C++ (?) part of tensorflow crashes. reduce your problem further, until it stops crashing. see if CPU vs GPU execution makes a difference (if not, stick to CPU-only). investigate. come up with a [mre]. – Christoph Rackwitz Aug 03 '22 at 15:42
  • 1
    If I run it on CPU it works, but when on GPU it dont. I think the problem lies with keras version(from other github issues). I want to downgrade my keras version but i want to make sure that it is compatible with tensorflow 2.9.0. How to know which keras version is compatible with it? – Na462 Aug 03 '22 at 15:53
  • Check out my answer here, could solve your issue: https://stackoverflow.com/a/73543224/5372188 – JAD Aug 30 '22 at 13:29

0 Answers0