Creating Solr shards in Alfresco 6.1 CE

Question

After migrating to Alfresco CE 6.1 (201901 GA) and Alfresco Search Services we have massive search performance problems. So I tried to configure sharding in solr by following Creating Solr shards manually. I was able to create the shards and indexing seams to work but when I configure Alfresco to use sharding in alfresco-global.properties

solr6.store.mappings.value.solrMappingAlfresco.nodeString=solr.mycompany.com:8983/solr/#/alfresco
solr6.store.mappings.value.solrMappingAlfresco.numShards=4
solr6.store.mappings.value.solrMappingAlfresco.replicationFactor=1

solr6.store.mappings.value.solrMappingArchive.nodeString=solr.mycompany.com:8983/solr/#/archive
solr6.store.mappings.value.solrMappingArchive.numShards=4
solr6.store.mappings.value.solrMappingArchive.replicationFactor=1

I get a "Request failed 405"

/solr/#/alfresco-2/alfresco?wt=json&fl=DBID%2Cscore&rows=50&shards=solr.mycompany.com:8983%2Fsolr%2F%23%2Falfresco-0,solr.mycompany.com:8983%2Fsolr%2F%23%2Falfresco-1,solr.mycompany.com:8983%2Fsolr%2F%23%2Falfresco-2,solr.mycompany.com:8983%2Fsolr%2F%23%2Falfresco-3&df=TEXT&start=0&locale=*&alternativeDic=DEFAULT_DICTIONARY&sort=%40cm%3Amodified+desc&fq=%7B%21afts%7DAUTHORITY_FILTER_FROM_JSON&fq=%7B%21afts%7DTENANT_FILTER_FROM_JSON
        at org.alfresco.repo.search.impl.solr.AbstractSolrQueryHTTPClient.postQuery(AbstractSolrQueryHTTPClient.java:79)
        at org.alfresco.repo.search.impl.solr.SolrQueryHTTPClient.postSolrQuery(SolrQueryHTTPClient.java:1116)
        at es.keensoft.repo.search.impl.solr.EnhancedSolrQueryHTTPClient.postSolrQuery(EnhancedSolrQueryHTTPClient.java:43)
        at org.alfresco.repo.search.impl.solr.SolrQueryHTTPClient.executeQuery(SolrQueryHTTPClient.java:583)
        at org.alfresco.repo.search.impl.solr.SolrQueryLanguage.executeQuery(SolrQueryLanguage.java:52)
        at org.alfresco.repo.search.impl.solr.SolrSearchService.query(SolrSearchService.java:354)
        at org.alfresco.repo.search.SearcherComponent.query(SearcherComponent.java:84)
        at jdk.internal.reflect.GeneratedMethodAccessor789.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.base/java.lang.reflect.Method.invoke(Unknown Source)
        at org.alfresco.repo.management.subsystems.SubsystemProxyFactory$1.invoke(SubsystemProxyFactory.java:79)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
        at com.sun.proxy.$Proxy36.query(Unknown Source)
        at org.alfresco.repo.search.impl.SearchServiceSubSystemDelegator.query(SearchServiceSubSystemDelegator.java:133)
        at jdk.internal.reflect.GeneratedMethodAccessor789.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.base/java.lang.reflect.Method.invoke(Unknown Source)
        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
        at net.sf.acegisecurity.intercept.method.aopalliance.MethodSecurityInterceptor.invoke(MethodSecurityInterceptor.java:80)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
        at org.alfresco.repo.security.permissions.impl.ExceptionTranslatorMethodInterceptor.invoke(ExceptionTranslatorMethodInterceptor.java:53)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
        at org.alfresco.repo.audit.AuditMethodInterceptor.invoke(AuditMethodInterceptor.java:166)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
        at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:294)
        at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
        at com.sun.proxy.$Proxy36.query(Unknown Source)
        at org.alfresco.repo.jscript.Search.queryResultMeta(Search.java:1016)
        ... 100 more

For better readability the decoded POST url to solr:

/solr/#/alfresco-2/alfresco?wt=json&fl=DBID,score&rows=50&shards=solr.mycompany.com:8983/solr/#/alfresco-0,solr.mycompany.com:8983/solr/#/alfresco-1,solr.mycompany.com:8983/solr/#/alfresco-2,solr.mycompany.com:8983/solr/#/alfresco-3&df=TEXT&start=0&locale=*&alternativeDic=DEFAULT_DICTIONARY&sort=@cm:modified+desc&fq={!afts}AUTHORITY_FILTER_FROM_JSON&fq={!afts}TENANT_FILTER_FROM_JSON

Was anybody already successfully in configuring sharding in Alfresco Search Services (1.3.x, 1.4) and Alfresco 6.1 Community? Does anybody have a hint what is going wrong? Alfresco seems not to use the solr standard urls for searching in the sharded index so I'm somehow lost how to get this working.

some success: in alfresco-global.properties it should be /solr/alfresco (without the hash sign). additionally: when configuring solr without ssl shareded access works but doesn't scale better if running only from one jvm — Heiko Robert, Dec 09 '19 at 15:27
MCVE Please show the relevant code and state the exact problem or error. A description with a stack trace is not enough. Also see [How to create a Minimal, Complete, and Verifiable example](http://stackoverflow.com/help/mcve). — jww, Dec 14 '19 at 17:24
@jww: what you missing in my MCVE? I described the product, the exact config how to reproduce the error, the stacktrace I got and the documentation I used. https://stackoverflow.com/help/minimal-reproducible-example — Heiko Robert, Dec 15 '19 at 19:06

score 0 · Accepted Answer · answered Dec 15 '19 at 19:39

0

to answer my own question: the example Creating Solr shards manually doc example should be without the hash sign in the url but does not work if solr runs with SSL support.

solr6.store.mappings.value.solrMappingAlfresco.nodeString=solr.mycompany.com:8983/solr/alfresco
solr6.store.mappings.value.solrMappingAlfresco.numShards=4
solr6.store.mappings.value.solrMappingAlfresco.replicationFactor=1

solr6.store.mappings.value.solrMappingArchive.nodeString=solr.mycompany.com:8983/solr/archive
solr6.store.mappings.value.solrMappingArchive.numShards=4
solr6.store.mappings.value.solrMappingArchive.replicationFactor=1

as an alternative it is possible to use the DynamicShardRegistration which is not documented to work in the community version but is working if you set

useDynamicShardRegistration=true

in alfresco-global.properties

see also the discussion in https://github.com/Alfresco/SearchServices/issues/232

answered Dec 15 '19 at 19:39

Heiko Robert

2,488
11
12

You answered it yourself, I'm too late, solr4 sharding doesn't work if SSL is enable. But it work with solr6 https://docs.alfresco.com/6.0/references/sharding-best-practices.html – prignony Jan 14 '20 at 13:40
At the moment for us sharding is not the right route to go. Sharding only helps to get better performance if you also have multiple storage systems (given you can't put the whole index in RAM). If running on a shared storage like a SAN, sharding puts only overhead on your stack. Search performance is the same for our performance problem - with or without sharding. Alfresco 5.2/solr4 searches are 3-5 times faster on the same infrastructure / ~. My feeling is that the bad seach performance is caused by changes in solr6 from solr4 or in a different way to build the index done by the ASS team. – Heiko Robert Jan 15 '20 at 14:14
If you have an enterprise edition, ask your TAM for his advice it is part of your subscription :) Did you change the way you index your files? Fulltext index vs properties index? There is also a way to divide your index in smaller / bigger chunck. Maybe a reindex using rerank? – prignony Jan 17 '20 at 17:05
Its a CE. We didn't change anything except upgrade from solr4 to ASS and Alfresco 6.1. Index is recreated using rerank and reactivated identifiers/suggestable propeties – Heiko Robert Jan 22 '20 at 16:18
We don't have the time and budget this month to dig deeper but the most striking difference between solr4 and ASS is that CPU is waiting and memory unused most of the time although jvm has a lot of resources (RAM/cpu). So I guess some changes in solr6 and/or ASS causes this bad performance. Sharding does not work around inefficent coding ... – Heiko Robert Jan 22 '20 at 16:28
I see that there is a big bug that slow everything down for version 6.4.0, 6.4.1 of solr fixed in 6.4.2 could you check the version you are running? https://cwiki.apache.org/confluence/display/SOLR/SolrPerformanceProblems – prignony Jan 22 '20 at 22:09
@prignony Thanks for your feedback! Alfresco Search Services 1.4.0 embeds solr 6.6.5 – Heiko Robert Jan 24 '20 at 08:59

Creating Solr shards in Alfresco 6.1 CE

1 Answers1

Linked