TensorFlow classifiying integers for divisibility by 3 does not work

Question

I want to learn TensorFlow. I found a code which classifies integers for divisibility by 2 here. It works very well and accuracy is 100%. I only had to add an import command for numpy at the very beginning.

Now I wanted to change it to classify for divisibility by 3 instead of 2. I changed one single line. I changed

Y.append( to_categorical(v%2, 2) )

to

Y.append( to_categorical(0 if v%3 == 0 else 1, 2) )

But now it no longer works. It always predicts 1 and the accuracy is 0.67. How can that be as I didn't change the style of the code? I only changed the classification function. I tried to use different loss functions, add an hidden layer and also different activation functions. Nothing helped. I want to know why the code no longer works after applying this little change. Here is my code:

import numpy as np
from keras.models import Sequential
from keras.layers import Dense
from keras.utils import to_categorical


# Helper function to convert a number 
# to its fixed width binary representation
def conv(x):
  a = format(x, '032b')
  l = list(str(a))
  l = np.array(list(map(int, l)))
  return l

# input data
data = [conv(i) for i in range(100000)]
X = np.array(data)


Y= list() # empty list of results
for v in range(100000):
  Y.append( to_categorical(0 if v%3 == 0 else 1, 2) )
  
Y = np.array(Y) # we need np.array


# Sequential is a fully connected network
model = Sequential()

# 32 inputs and 1 neuron in the first layer (hidden layer)
model.add(Dense(1, input_dim=32, activation='relu'))

# 2 output layer 
model.add(Dense(2, activation='sigmoid'))


model.compile(loss='binary_crossentropy', 
              optimizer='adam', 
              metrics=['accuracy'])

# epochs is the number of times to retrain over the same data set
# batch_size is how may elements to process in parallel at one go
model.fit(X, Y, epochs=5, batch_size=100, verbose=1)
weights, biases = model.layers[0].get_weights()
print("weights",weights.size, weights, "biases", biases)
model.summary()

I have also read Neural network always predicts the same class but nothing helped.

Answer 1

In this code you use binary representation of a number and checking if such number is divisible by 2 using this representation is pretty simple - you check least significant byte. For any power of two it is similar.

However, divisibility by 3 is not as simple because rule here is more complicated and you may try some more complex model.

For example adding more hidden layers can improve our results. I used this model:

model = Sequential()

model.add(Dense(32, input_dim=32, activation='relu'))
model.add(Dense(32, activation='relu'))

model.add(Dense(2, activation='sigmoid'))

and got results:

...
Epoch 20/20
1000/1000 [==============================] - 1s 1ms/step - loss: 0.0594 - accuracy: 0.9700

You can try with different architectures to get better score.