0

I am using alfresco 5.2.3 enterprise with solr6 search services. Everything works fine when I deploy our application custom code inside the alfresco-platform jar and alfresco-share jar. Now, when I install alfresco records management amp file, the search stops working. I am not able to search even a single document or folder. RM amp version: alfresco-rm-enterprise-repo-2.7.0.amp and alfresco-rm-enterprise-share-2.7.0.amp

There are three different instances: repo (where alfresco.war sits), share (where share.war and ADF sits) and index server (where indexes are maintained).

I install alfresco-rm-enterprise-repo-2.7.0.amp on repo, and alfresco-rm-enterprise-share-2.7.0.amp on share. And restart the servers. RM installation is successful without any errors. But search is not at all working after this.

Is it possible that after RM installation, some indexes are corrupted, and we need to conduct reindexing ? Can that resolve this issue ?

NOTE: The versions of alfresco and RM are already in the supported stack as per the alfresco documentation link: https://docs.alfresco.com/5.2/concepts/supported-platforms-ACS.html

Any help would be appreciated.

Sanket Mehta
  • 622
  • 6
  • 10
  • 1
    Just give Alfresco Support a ring, they'll help you out. That's part of what you're paying for with Alfresco Enterprise edition! – Gagravarr Dec 05 '19 at 11:22
  • 1
    I have a client running this exact configuration and we do not have this problem. What do you mean when you say, "when I deploy our application custom code inside the alfresco-platform jar and alfresco-share jar". Are you touching the Alfresco JARs? Or are you using AMPs for your custom code? – Jeff Potts Dec 05 '19 at 15:32
  • No, we are using the JAR approach for our application/custom code deployment (not the amps approach). We are not touching the OOTB alfresco or share jars, just the name of our custom jar is like alfresco-platform-0.0.1.jar and alfresco-share-0.0.1.jar – Sanket Mehta Dec 06 '19 at 04:59
  • 1
    You need to figure out why the search stops working, there has to be something in the logs, try elevating log4j.properties to "debug" or something along those lines. – Lista Dec 06 '19 at 12:43
  • Yes, the logs have multiple instances of the following statements: org.alfresco.service.namespace.NamespaceException: Namespace prefix rma is not mapped to a namespace URI . I am currently waiting for getting access on my index server (which I don't have as of now), once I get it, am planning to reindex the contents and restart the index server. Hopefully that may resolve the issue. Will keep the updates posted. – Sanket Mehta Dec 09 '19 at 07:13
  • Update on my previous comment: I got access to index server, I deleted the indexes (alfrescoModels, alfresco core index folder, archive core index folder) for reindexing purpose. After I restart alfresco repo and solr server, the new indexes are not getting created at all on solr server. And logs show the following error: org.alfresco.error.AlfrescoRuntimeException: 111929898 GetModelsDiff return status is 403 ...org.alfresco.error.AlfrescoRuntimeException: 111929899 api/solr/aclchangesets return status:403 ... – Sanket Mehta Jan 16 '20 at 12:37

2 Answers2

0

Finally, the problem is resolved. The keystore, truststore certificate files were the culprit. New keystore, truststore files were required to be generated as the communication between ACS and Index server was not happening and resulting into GetModelsDiff 403 error in the logs. Additionally, we ensured the following settings were put up in ACS and index server files:


ACS alfresco-global.properties:

alfresco.host=alfresco-dev-repo.domain.com
alfresco.port=443
alfresco.protocol=https

share.host=alfresco-dev-repo.domain.com
share.port=443
share.protocol=https

db.ssl_params=&useSSL=true&requireSSL=true&verifyServerCertificate=true&trustCertificateKeyStoreUrl=file:///opt/alfresco-content-services/alf_data/keystore/ssl.truststore&trustCertificateKeyStoreType=JCEKS&trustCertificateKeyStorePassword=kT9X6oe68t
db.url=jdbc:mysql://${db.host}/${db.name}?${db.params}${db.ssl_params}

index.subsystem.name=solr6
dir.keystore=${dir.root}/keystore
solr.host=alfresco-dev-index.domain.com
solr.port.ssl=8983
solr.port=80
solr.secureComms=https

#ssl encryption
encryption.ssl.keystore.location=${dir.keystore}/ssl.keystore
encryption.ssl.keystore.type=JCEKS
encryption.ssl.keystore.keyMetaData.location=${dir.keystore}/ssl-keystore-passwords.properties
encryption.ssl.truststore.location=${dir.keystore}/ssl.truststore
encryption.ssl.truststore.type=JCEKS
encryption.ssl.truststore.keyMetaData.location=${dir.keystore}/ssl-truststore-passwords.properties

Solr Configuration:

solr.in.sh file:

SOLR_PORT=8983

SOLR_SSL_KEY_STORE=/opt/alfresco-search-services/solrhome/keystore/ssl.keystore
SOLR_SSL_KEY_STORE_PASSWORD=kT9X6oe68t
SOLR_SSL_TRUST_STORE=/opt/alfresco-search-services/solrhome/keystore/ssl.truststore
SOLR_SSL_TRUST_STORE_PASSWORD=kT9X6oe68t
SOLR_SSL_NEED_CLIENT_AUTH=true
SOLR_SSL_WANT_CLIENT_AUTH=false

alfresco core > solrcore.properties AND archive core > solrcore.properties

alfresco.secureComms=https
data.dir.root=/opt/alfresco-search-services/solrhome/
alfresco.port.ssl=8443
alfresco.encryption.ssl.keystore.passwordFileLocation=ssl-keystore-passwords.properties
alfresco.encryption.ssl.truststore.passwordFileLocation=ssl-truststore-passwords.properties
alfresco.baseUrl=/alfresco
alfresco.host=alfdevhostname.domain.com
alfresco.encryption.ssl.keystore.provider=
alfresco.encryption.ssl.truststore.type=JCEKS
alfresco.encryption.ssl.truststore.provider=
alfresco.encryption.ssl.keystore.type=JCEKS
alfresco.encryption.ssl.keystore.location=ssl.keystore
alfresco.port=80
alfresco.version=5.2.3
alfresco.encryption.ssl.truststore.location=ssl.truststore

No need of touching the files under this location:

/opt/alfresco-search-services/solrhome/templates/rerank/conf

And finally the most important part:

Latest/Updated Certificate files placed under:

/opt/alfresco-search-services/solrhome/keystore

And the same certificate files placed under:

/opt/alfresco-search-services/solrhome/alfresco/conf

and

/opt/alfresco-search-services/solrhome/archive/conf

and on ACS server:

/opt/alfresco-content-services/alf_data/keystore

On top of it, if the issue is still not getting resolved, you can try the following:

  • Set solr.secureComms=none in alf-global, and alfresco.secureComms=none in archive core and alfresco core, and restart both entities to see if the normal HTTP connection is working without SSL or HTTPS
  • Validate with infra/netwk team is certificates installed r correct or not
  • Try pointing directly the IP address of alfresco and solr to each other, instead of host name –as it might be coming through LB
  • Try Telnet solr host from alfresco repo server, and also vice-versa
  • Put -Djavax.net.debug=all under alfresco > tomcat/scripts/ctl.sh and see if you get any useful information
  • Check not just the alfresco.log, solr.log, see access-logs if you can find 404 or 200 status responses. OR curl on solr machine against the URL that is logged in localhost-access logs.
  • Starting/stopping solr with root user – ideally should be another dedicated user for solr
  • Ideally certificates should be copied from alfresco (alf_data/keystore) to solr server, not from solr to alfresco server. But if not working, you can try the other way around.
  • The alfresco.host, share.host, alfresco.port, share.port in alf-global should match with properties in solrhome/alfresco/conf/solrcore.properties + solrhome/archive/conf/solrcore.properties
  • Try putting debugger on i.e debug statements on from alfresco repo side as well as solr side to capture any unknown or hidden exceptions/errors.
  • You can also check the solr-admin console page from browser and check the logs from there.
Sanket Mehta
  • 622
  • 6
  • 10
0

I faced similar issue on Alfresco 6.2.2 with alfresco-insight-engine 2.0.0. Multiple errors like below I had faced one by one after changing the configurations :-

If certificates are not matching between ACS, Solr OR between ACS, Solr and AWS OR certificates generated are incorrect OR certificates compatible only with particular java version OR certificates not added to truststore correctly, then you may get:

javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException  , 


unable to find valid certification path to requested target , 

Caused by: java.lang.RuntimeException: Unexpected error: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty

I checked the configuration (certificate) was imported correctly at AWS side. And no restriction was applied at AWS side. But, finally I was able to resolve with the following combination: Alfresco side

Server.xml:

<Connector port="8443" protocol="org.apache.coyote.http11.Http11Protocol"
    SSLEnabled="true" maxThreads="150" scheme="https"
    keystoreFile="/app/tomcat/keystores/ssl.keystore"
    keystorePass="pwd" keystoreType="JCEKS"
    secure="true" connectionTimeout="240000"
    truststoreFile="/app/tomcat/keystores/ssl.truststore"
    truststorePass="pwd" truststoreType="JCEKS"
    clientAuth="false" sslProtocol="TLS" />

alfresco-global.properties:

index.subsystem.name=solr6
solr.secureComms=https
solr.port=8984
solr.port.ssl=8984
solr.host=domainname

alfresco.context=alfresco
alfresco.host=host
alfresco.port=8443
alfresco.protocol=https
#
share.context=share
share.host=host
share.port=8443
share.protocol=https

#ssl encryption
encryption.ssl.keystore.location=/app/tomcat/keystores/ssl.keystore
encryption.ssl.keystore.type=JCEKS
encryption.ssl.keystore.keyMetaData.location=/app/tomcat/keystores/ssl-keystore-passwords.properties
encryption.ssl.truststore.location=/app/tomcat/keystores/ssl.truststore
encryption.ssl.truststore.type=JCEKS
encryption.ssl.truststore.keyMetaData.location=/app/tomcat/keystores/ssl-truststore-passwords.properties

solr side

solr.in.sh

    SOLR_SOLR_HOST=domainname
    SOLR_ALFRESCO_HOST=domainname
    SOLR_SSL_CUSTOM="-Dsolr.ssl.checkPeerName=false -Dsolr.allow.unsafe.resourceloading=true"
SOLR_OPTS="$SOLR_SSL_CUSTOM"
    SOLR_PORT=8984
    SOLR_HOST=domainname
    SOLR_SSL_KEY_STORE=/app/alfresco-insight-engine/solrhome/keystore/ssl.repo.client.keystore
    SOLR_SSL_KEY_STORE_PASSWORD=pwd
    SOLR_SSL_KEY_STORE_TYPE=JCEKS
    SOLR_SSL_TRUST_STORE=/app/alfresco-insight-engine/solrhome/keystore/ssl.repo.client.truststore
    SOLR_SSL_TRUST_STORE_PASSWORD=pwd
    SOLR_SSL_TRUST_STORE_TYPE=JCEKS
    SOLR_SSL_NEED_CLIENT_AUTH=false
    SOLR_SSL_WANT_CLIENT_AUTH=true

solrcore.properties (both cores)

alfresco.encryption.ssl.truststore.location=ssl.repo.client.truststore
alfresco.encryption.ssl.keystore.provider=
alfresco.encryption.ssl.truststore.type=JCEKS
alfresco.host=ip-10-233-4-126.ap-east-1.compute.internal
alfresco.encryption.ssl.keystore.location=ssl.repo.client.keystore
alfresco.encryption.ssl.truststore.provider=
alfresco.port.ssl=8443
alfresco.encryption.ssl.truststore.passwordFileLocation=ssl-truststore-passwords.properties
alfresco.port=8080
alfresco.encryption.ssl.keystore.type=JCEKS
alfresco.secureComms=https
alfresco.encryption.ssl.keystore.passwordFileLocation=ssl-keystore-passwords.properties

solrcore.properties (under rerank/conf)

alfresco.host=domainname
alfresco.port=8080
alfresco.port.ssl=8443
alfresco.secureComms=https

alfresco.encryption.ssl.keystore.type=JCEKS
alfresco.encryption.ssl.keystore.provider=
alfresco.encryption.ssl.keystore.location=ssl.repo.client.keystore
alfresco.encryption.ssl.keystore.passwordFileLocation=ssl-keystore-passwords.properties
alfresco.encryption.ssl.truststore.type=JCEKS
alfresco.encryption.ssl.truststore.provider=
alfresco.encryption.ssl.truststore.location=ssl.repo.client.truststore
alfresco.encryption.ssl.truststore.passwordFileLocation=ssl-truststore-passwords.properties

The alfresco keystore files (used/pointed to by Alfresco) are under /app/tomcat/keystores. And solr keystore files (used/pointed to by solr) are under /app/alfresco-insight-engine/solrhome/keystore.

NOTE: We have copied the solr keystores files to following locations also: /app/alfresco-insight-engine/solrhome/alfresco/conf , /app/alfresco-insight-engine/solrhome/archive/conf , /app/alfresco-insight-engine/solrhome/templates/rerank/conf

NOTE: If it's just a certificate not added to truststore cacerts, then you can add the certificate to the cacerts using this link: Error - trustAnchors parameter must be non-empty

Other points which can be checked if above does not work:

  • Check if java version is a supported one (in supported stack) and certificates are correctly getting added to the truststore.
  • Check the java version from alfresco's admin summary page and verify if certificates get added into the correct java
  • Check if solr host, port and ssl port is correctly picked up. Verify this location - http://domainname/alfresco/s/enterprise/admin/admin-searchservice , as port might be picked up from here which might not match with the one in alfresco-global.properties file. In case of mismatching properties between alf-global and admin-searchservice URL, you may get “Connection refused” error in alfresco logs when alfresco tries to connect to solr.
  • If JKS type of certi has become obsolete, try generating PKCS12 or JCEKS type certi.
  • When solr is running on 8983 (http) as well as 8984 (https/ssl), you may get error "Unsupported or unrecognized SSL message". Try stopping one which is not used.
  • If https with 8984 solr url is not accessible from browser, then try importing the correct certificate at AWS, and also try adding following entry in /app/alfresco-insight-engine/solr/server/etc/jetty-ssl.xml file: FALSE
Sanket Mehta
  • 622
  • 6
  • 10