1

I have tried a vanila enc-dec arch as following (english to french NMT)

My encoder-decoder architecture

I want to know how to integrate keras attention layer here. Either from the keras docs or any other attention module from third party repo is also welcome. I just need to integrate it and see how it works and finetune it.

Full code is available here.

Not showing any code in this post because it's large and complex.

Sayan Dey
  • 771
  • 6
  • 13
  • Where do you want to add your attention layer? – Aniket Bote Aug 30 '20 at 07:18
  • That's what I want to know actually, from the docs I know you gotta generate the attention weights from decoder and encoder inputs. I want something by doing changes in this existing architecture. this is a french to english translation architecture. – Sayan Dey Aug 30 '20 at 07:51
  • @AniketBote I have implemented what I was looking for, have put an answer. – Sayan Dey Aug 31 '20 at 14:25

1 Answers1

1

Finally I have resolved the issue. I am using a third-party-attention layer by Thushan Ganegedara. Used it's Attentionlayer class. And integrated that in my architecture as following.

Architecture with attention

Sayan Dey
  • 771
  • 6
  • 13