Stack Overflow
Stack Overflow

Questions tagged [faiss]

77 questions
8
votes
3 answers

faiss ERROR: Could not find a version that satisfies the requirement faiss (from versions: none)

When Running Installation: pip install faiss I am getting this error: ERROR: Could not find a version that satisfies the requirement faiss (from versions: none) ERROR: No matching distribution found for faiss also when i use conda: conda install…
predactor
  • 772
  • 4
  • 11
  • 29
7
votes
1 answer

Installing Faiss on Lambda Stack

I want to install Faiss-GPU on Lambda Stack by conda install -c pytorch faiss-gpu but there is no conda installed. I tried the solution mentioned here: Installing faiss on Google Colaboratory with this version of faiss…
MAMProgr
  • 400
  • 5
  • 12
5
votes
0 answers

How to add index to python FAISS incrementally

I am using Faiss to index my huge dataset embeddings, embedding generated from bert model. I want to add the embeddings incrementally, it is working fine if I only add it with faiss.IndexFlatL2 , but the problem is while saving it the size of it is…
DevPy
  • 439
  • 6
  • 17
5
votes
4 answers

Installing faiss on Google Colaboratory

I try to follow instruction on the MUSE project. They require PyTorch and Faiss. PyTorch is easy to install. But I found problem with installing Faiss. The instruction on MUSE tell me to use conda install faiss-cpu -c pytorch But Google Colab…
korakot
  • 37,818
  • 16
  • 123
  • 144
4
votes
3 answers

Update an element in faiss index

I am using faiss indexflatIP to store vectors related to some words. I also use another list to store words (the vector of the nth element in the list is nth vector in faiss index). I have two questions: Is there a better way to relate words to…
Nilou
  • 145
  • 2
  • 10
3
votes
1 answer

FAISS with C++ indexing 512D vectors

I have a collection of 512D std::vector to store face embeddings. I create my index and perform training on a subset of the data. int d = 512; size_t nb = this->templates.size() // 95000 size_t nt = 50000; // training data…
Brian
  • 234
  • 3
  • 8
3
votes
0 answers

Is there a useful technique to map FAISS ids to the appropriate metadata?

I wanted to ask your opinion about saving ids with corresponding metadata. Think of the data I might have. I want to use many processes to add various vectors to FAISS in real-time. There is a need for storage to UUIDs to map those vector ids to…
Rəşad Abdulxalıqov
  • 41
  • 3
3
votes
3 answers

Huggingface Transformers FAISS index scores

Huggingface transformers library has a pretty awesome feature: it can create a FAISS index on embeddings dataset which allows searching for the nearest neighbors. train_ds['train'].add_faiss_index("embedding") scores, sample =…
Nik
  • 161
  • 1
  • 13
3
votes
1 answer

What is the equivalent of python's faiss.normalize_L2() in C++?

I want to perfom similarity search using FAISS for 100k facial embeddings in C++. For the distance calculator I would like to use cosine similarity. For this purpose, I choose faiss::IndexFlatIP .But according to the documentation we need to…
Sabbir Talukdar
  • 115
  • 2
  • 11
3
votes
2 answers

faiss: How to retrieve vector by id from python

I have a faiss index and want to use some of the embeddings in my python script. Selection of Embeddings should be done by id. As faiss is written in C++, swig is used as an API. I guess the function I need is reconstruct : /** Reconstruct a stored…
chefhose
  • 2,399
  • 1
  • 21
  • 32
2
votes
0 answers

How to save vector embeddings `FAISS` in MongoDB?

I am using LangChain for building some stuff and came across one of the most prominent index-based vector database FAISS. Following is the command of how I am using the FAISS vector database: from langchain.document_loaders import PyMuPDFLoader from…
Shuhul Handoo
  • 122
  • 8
2
votes
0 answers

Issue in deleting vector from Faiss index

I used remove_ids method to remove some vectors from faiss index, After deleting a vectory , search result are very incorrect. Anybody have some information regarding this? how can i correctly delete vector from faiss index without effecting search…
Ankit Maheshwari
  • 21
  • 3
2
votes
0 answers

Why is the total query time the same when using GPU-optimized Faiss or CPU-based Faiss for index construction and search?

I currently have a dataset of 5 million samples and I would like to retrieve the 20 most similar samples from this dataset for each of these samples. To test the efficiency of this process, I have written the GPU version of Faiss index and CPU…
袁梓晽
  • 21
  • 1
2
votes
1 answer

C++ Faiss - How to search in subsets

according to the faiss wiki page (link), you should be able to use SearchParameters to selectively include or exclude ids in a search. Now the information there is a bit strange, because the field "sel" does not exist at all. Also the names were…
Christopher K
  • 37
  • 1
  • 6
2
votes
1 answer

can I 'inner-search' most similar vectors within a FAISS index?

I have a FAISS index populated with 8M embedding vectors. I don't have the embedding vectors anymore, only the index, and it is expensive to recompute the embeddings. Can I search the index for the top-k most similar vectors to each of the index's…
user7980
  • 71
  • 2
1
2 3 4 5 6