6

I am trying to implement a slightly modified binary crossentropy loss function for a model in Keras. From Keras, binary_crossentropy is defined as:

def binary_crossentropy(y_true, y_pred):
  return K.mean(K.binary_crossentropy(y_true, y_pred), axis=-1)

The data I have is grouped (i.e there is a column indicating group1, group2, etc.) but there is a varying number of rows for each group (i.e group1 has 52 observations, group2 has 101 observations, etc.).

IDEALLY, I would like to find the mean binary crossentropy for each of these groups, and return the maximum mean binary crossentropy (largest average binary crossentropy, by group).

There doesn't seem to be any out-of-the-box solutions for using groups, and I wasn't able to come up with a solution. The information about which group an observation belongs to gets lost and isn't passed into y_true and y_pred, and I'm not sure how k-fold cv would exactly change what/how many observations are passed as y_true and y_pred. If there was a way to retain the group information through the Sequential model, that would probably be the solution. Code would accomplish something like: 

def custom_loss(y_true, y_pred):
  max_bc = []
  for group in groups:
     max_bc += [K.mean(K.binary_crossentropy(y_true, y_pred), axis=-1)]
  return max_bc

If the above isn't possible, an alternative measure could be something like the 75th percentile value of the tensor. Something like:

def custom_loss(y_true, y_pred):
  return K.binary_crossentropy(y_true, y_pred)[len(y_true)*0.75]

But I know for sure that is wrong-- I'm just not familiar enough with Keras and Tensorflow to get the right code.

EDIT: I might've just found a way to do the percentiles, but the outcome isn't as desired... Would still be great to get insight on the first part. 

def custom_loss(y_true, y_pred):
  e = K.binary_crossentropy(y_true,y_pred)
  return distributions.percentile(e, q=75.)
atester
  • 61
  • 3
  • Would you care to explain the format, shape, ranges and meaning of your input and output true data? How do you identify the groups? – Daniel Möller May 17 '18 at 23:52
  • 1) Groups are identified in its own "groups" column.......2) y_true is 0 or 1 ...... 3) y_pred is probability between 0 and 1 ...... 4) There are roughly 400,000 observations There isn't any extractable meaning of y_true and y_pred that would be helpful for this problem. – atester May 20 '18 at 21:51
  • This answer solves the problem https://stackoverflow.com/questions/46858016/keras-custom-loss-function-to-pass-arguments-other-than-y-true-and-y-pred – Eyal Shulman Aug 21 '19 at 20:55

0 Answers0