Stack Overflow
Stack Overflow

Questions tagged [elmo]

A tag for ELMo, a way to model words into a deep contextualized representation. It is part of and developed by AllenNLP.

ELMo is a deep contextualized word representation that models both (1) complex characteristics of word use (e.g., syntax and semantics), and (2) how these uses vary across linguistic contexts (i.e., to model polysemy). These word vectors are learned functions of the internal states of a deep bidirectional language model (biLM), which is pre-trained on a large text corpus. They can be easily added to existing models and significantly improve the state of the art across a broad range of challenging NLP problems, including question answering, textual entailment and sentiment analysis.

75 questions
12
votes
1 answer

What is the difference between Sentence Encodings and Contextualized Word Embeddings?

I have seen both terms used while reading papers about BERT and ELMo so I wonder if there is a difference between them.
Rodrigo
  • 133
  • 1
  • 5
6
votes
1 answer

Swap gnews with ELMo in the simple colab tutorial

I'm working on this colab notebook: https://colab.research.google.com/github/tensorflow/hub/blob/master/examples/colab/tf2_text_classification.ipynb I'd like to replace the gnews swivel embeddings with the ELMo embeddings. So, replace model =…
Mittenchops
  • 18,633
  • 33
  • 128
  • 246
4
votes
1 answer

How to get similar words in pre trained ELMO embedding?

How to get similar word for a given word in the pre trained ELMO Embedding? For Example: In Glove, we have glove_model.most_similar() to find the most similar word and its embedding for any given word. Similarly do we have anything in ELMO?
Anvitha
  • 89
  • 4
3
votes
1 answer

Training SVM classifier (word embeddings vs. sentence embeddings)

I want to experiment with different embeddings such Word2Vec, ELMo, and BERT but I'm a little confused about whether to use the word embeddings or sentence embeddings, and why. I'm using the embeddings as features input to SVM classifier. Thank you.
NST
  • 115
  • 10
3
votes
1 answer

Strongly increasing memory consumption when using ELMo from Tensorflow-Hub

I am currently trying to compare the similarity of millions of documents. For a first test on a CPU I reduced them to around 50 characters each and try to get the ELMo Embedding for 10 of them at a time like this: ELMO =…
Daniel Töws
  • 347
  • 1
  • 5
  • 21
3
votes
0 answers

Using Elmo models to predict the masked word in a sentence

I have a homework which requires me to build an algorithm that can guess a missing word from a sentence. For example, when the input sentence is : " I took my **** for a walk this morning" , I want the output to guess the missing word(dog). My…
ihatepointers
  • 43
  • 6
3
votes
1 answer

Understanding ELMo's number of presentations

I am trying my hand at ELMo by simply using it as part of a larger PyTorch model. A basic example is given here. This is a torch.nn.Module subclass that computes any number of ELMo representations and introduces trainable scalar weights for each.…
Bram Vanroy
  • 27,032
  • 24
  • 137
  • 239
3
votes
1 answer

Paragraph embedding with ELMo

I'm trying to understand how to prepare paragraphs for ELMo vectorization. The docs only show how to embed multiple sentences/words at the time. eg. sentences = [["the", "cat", "is", "on", "the", "mat"], ["dogs", "are", "in", "the", "fog",…
tensa11
  • 113
  • 1
  • 2
  • 7
3
votes
1 answer

ELMo - How to train trainable parameters

I am new to tensorflow-hub and came across the ELMo model (https://www.tensorflow.org/hub/modules/google/elmo/2). According to the original paper, the ELMo representation is a weighted average of hidden state activations and these weights are…
Sujay S Kumar
  • 621
  • 1
  • 5
  • 10
2
votes
1 answer

I am trying to use Elmo Embedding using TensorFlow 2 and Getting this error for model.fit command: 'NoneType' object has no attribute 'outer_context'

Importing Elmo Embedding Layer from TF-hub Using TF 2 # Imported Elmo Layer elmo_model_path = "https://tfhub.dev/google/elmo/3" elmo_layer = hub.KerasLayer(elmo_model_path, input_shape=[], dtype=tf.string, trainable=False) # Creating Model model =…
Saikat Mukherjee
  • 21
  • 1
2
votes
2 answers

Keras TensorFlow Hub: Getting started with simple ELMO network

I'm trying to get a simple ELMO model working from TensorFlow hub but, it's turning out to be a challenge. When I run the my code, I'm getting the error: "Inputs to eager execution function cannot be Keras symbolic tensors, but found [
webber
  • 1,834
  • 5
  • 24
  • 56
2
votes
0 answers

Can I train an ELMo model from scratch using allennlp.modules.elmo.Elmo?

I've been looking to train my own ELMo model for the past week and came across these two implementations allenai/bilm-tf & allenai/allennlp. I've been facing a few roadblocks for a few techniques I've tried and would like to clarify my findings, so…
Goh Jia Yi
  • 329
  • 3
  • 16
2
votes
1 answer

Calculate cosine similarity for elmo model

I am trying to calculate the cosine similarity of wordsim set using the Elmo model. This may not make sense since it is designed for sentence word embedding, but I want to see how the model performs in the situations like these. The Elmo I am using…
tonywang
  • 181
  • 2
  • 13
2
votes
1 answer

How to use BERT and Elmo embedding with sklearn

I created a text classifier that uses Tf-Idf using sklearn, and I want to use BERT and Elmo embedding instead of Tf-Idf. How would one do that ? I'm getting Bert embedding using the code below: from flair.data import Sentence from flair.embeddings…
Juned Ansari
  • 5,035
  • 7
  • 56
  • 89
2
votes
0 answers

Certificate validation failed for TensorFlow hub, Elmo module

I am trying to import the Elmo module using the url: elmo = hub.Module("https://tfhub.dev/google/elmo/3", trainable=True) but am unable to load it as it is giving an error: URLError: urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate…
JBSKR
  • 33
  • 5
1
2 3 4 5