0

I'm working on the one database per user pattern using CouchDB, my main database contains all my data and i want to create multiple sub-database using CouchDB Replication feature with a filter (i'm filtering the bigger database using : DateMin, DateMax, an Array of IDs, some other conditions based on each user)

https://docs.couchdb.org/en/3.2.0/replication/index.html

https://docs.couchdb.org/en/3.2.0/json-structure.html#replication-settings

When i have a small amount of information for filtering, everything works perfectly, my problem comes when the Array of IDs that i'm using for filtering contains too much element:

To run the filter, CouchDB is calling an HTTP:GET https://docs.couchdb.org/en/3.2.0/api/database/changes.html#db-changes which is restricted by the number of character see (What is the maximum length of a URL in different browsers?)

But CouchDB has the same request but with HTTP:POST https://docs.couchdb.org/en/3.2.0/api/database/changes.html#post--db-_changes

and i want to know how to specify to CouchDB to use the HTTP:POST for filtering

Additionnal informations :

CouchDB v. 3.1.0

My Replicator Doc :

{
  "source": "DB_URL_SOURCE.com",
  "target": "DB_URL_TARGET.com,
  "filter": "mydesigndoc/by_date_and_activities",
  "query_params": {
    "weeksMin": 10,
    "weeksMax": 10,
    "clindividu": true,
    "activitesID": "3988814,3989866,3743378,3742882,2595215,1259813,2596111",
    "cl": true
  },
  "continuous": true
}

HTTP:GET request :

http://localhost:5984/my-bigdatabase/_changes
    ?filter =mydesigndoc%2Fby_date_and_activities
    &activites=
    &weeksMax=
    &weeksMin=
    &feed=continuous
    &style=all_docs
    &since=0
    &timeout=10000
NLF.Noorlink
  • 11
  • 3
  • There are some fixes with POST and _changes filtering in version 3.2.0. Could you try it with the latest version?. e.g. https://github.com/apache/couchdb/pull/3373 – Juanjo Rodriguez Mar 21 '22 at 12:53
  • Hi Juanjo Rodriguez thanks for the answer Yep i'll try this and keep you inform about the result – NLF.Noorlink Mar 21 '22 at 13:32
  • @JuanjoRodriguez same problem, Tested on v3.2.0, CouchDB still use HTTP:GET **_change** to filter the replication, and if i set a big array as params for filtering the request and the replication fails – NLF.Noorlink Mar 21 '22 at 13:49
  • It seems that filters allways use GET (https://github.com/apache/couchdb/blob/3.x/src/couch_replicator/src/couch_replicator_api_wrap.erl#L494) but selectors can solve your problem if yo can convert your filter in a mango expression. – Juanjo Rodriguez Mar 22 '22 at 12:48
  • Hi Juanji, yeah we got the same idea and it works perfectly :D My only problem now is to adapt my code to programmatically create the replication doc with the selector ^^' Thanks for helping ! – NLF.Noorlink Mar 23 '22 at 12:36

0 Answers0