5

Trying to build a Wavelet Neural Network using Keras/Tensorflow. For this Neural Network I am supposed to use a Wavelet function as my activation function.

I have tried doing this by simply calling creating a custom activation function. However there seems to be an issue in regards to the backpropagation

import numpy as np
import pandas as pd
import pywt
import matplotlib.pyplot as plt
import tensorflow as tf
from keras.models import Model
import keras.layers as kl
from keras.layers import Input, Dense
import keras as kr
from keras.layers import Activation
from keras import backend as K
from keras.utils.generic_utils import get_custom_objects


def custom_activation(x):
  return pywt.dwt(x, 'db1') -1

get_custom_objects().update({'custom_activation':Activation(custom_activation)})

model = Sequential()
model.add(Dense(12, input_dim=8, activation=custom_activation))
model.add(Dense(8, activation=custom_activation)
model.add(Dense(1, activation=custom_activation)

i get the following error for running the code in its entirety

SyntaxError: invalid syntax

if i run 

model = Sequential()
model.add(Dense(12, input_dim=8, activation=custom_activation))
model.add(Dense(8, activation=custom_activation)

i get the following error SyntaxError: unexpected EOF while parsing

and if i run

model = Sequential() model.add(Dense(12, input_dim=8, activation=custom_activation))

I get the following error

TypeError: Cannot convert DType to numpy.dtype

mate00
  • 2,727
  • 5
  • 26
  • 34
King
  • 51
  • 3

1 Answers1

1

model.add() is a function call. You must close parenthesis, otherwise it is a syntax error.

These two lines in your code example will cause a syntax error.

model.add(Dense(8, activation=custom_activation)
model.add(Dense(1, activation=custom_activation)

Regarding the 2nd question:

I get the following error

TypeError: Cannot convert DType to numpy.dtype

This seems like a numpy function was invoked with the incorrect arguments. Perhaps you can try to figure out which line in the script caused the error.

Also, an activation function must be written in keras backend operations. Or you need to manually compute the gradients for it. Neural network training requires being able to compute the gradients of a function on the reverse pass in order to adjust the weights. As far as I understand it you can't just call an arbitrary python library as an activation function; you have to either re-implement its operations using tensor operations or you have the option of using python operations on eager tensors if you know how to compute the gradients manually.

Pedro Marques
  • 2,642
  • 1
  • 10
  • 10