Elastic Search Conflict with Spark Connector, rest high level client no such field error

Question

I am using elasticsearch version 7.9.2, and it is conflicting with nebula spark connector, a spark connector for nebula graph. I have seen cases where spark conflicts with elasticsearch, but with no existing solution.

Failed to instantiate [org.elasticsearch.client.RestHighLevelClient]: Factory method 'restHighLevelClient' threw exception; nested exception is java.lang.NoSuchFieldError: INSTANCE
    at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:185)
    at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:653)
    ... 47 common frames omitted
Caused by: java.lang.NoSuchFieldError: INSTANCE
    at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager$InternalAddressResolver.<init>(PoolingNHttpClientConnectionManager.java:619)
    at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.<init>(PoolingNHttpClientConnectionManager.java:165)
    at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.<init>(PoolingNHttpClientConnectionManager.java:149)
    at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.<init>(PoolingNHttpClientConnectionManager.java:121)
    at org.apache.http.impl.nio.client.HttpAsyncClientBuilder.build(HttpAsyncClientBuilder.java:668)
    at java.security.AccessController.doPrivileged(Native Method)
    at org.elasticsearch.client.RestClientBuilder.createHttpClient(RestClientBuilder.java:219)
    at java.security.AccessController.doPrivileged(Native Method)
    at org.elasticsearch.client.RestClientBuilder.build(RestClientBuilder.java:191)
    at org.elasticsearch.client.RestHighLevelClient.<init>(RestHighLevelClient.java:287)
    at org.elasticsearch.client.RestHighLevelClient.<init>(RestHighLevelClient.java:279)
    at com.bybit.byassets.collection.config.ElasticSearchConfig.restHighLevelClient(ElasticSearchConfig.java:54)
    at com.bybit.byassets.collection.config.ElasticSearchConfig$$EnhancerBySpringCGLIB$$e050e7f6.CGLIB$restHighLevelClient$0(<generated>)
    at com.bybit.byassets.collection.config.ElasticSearchConfig$$EnhancerBySpringCGLIB$$e050e7f6$$FastClassBySpringCGLIB$$43cd2bed.invoke(<generated>)
    at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:244)
    at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:331)
    at com.bybit.byassets.collection.config.ElasticSearchConfig$$EnhancerBySpringCGLIB$$e050e7f6.restHighLevelClient(<generated>)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:154)
    ... 48 common frames omitted

I have tried shading the nebula spark connector jar, but the error still persists. Any help is appreciated. Thank you!

score 0 · Answer 1 · answered Aug 01 '22 at 07:41

0

This looks similar to this question and this.

Could you check all dependencies on httpClient, there should be one old version before INSTANCE field was introduced, we should exclude that httpClient.

answered Aug 01 '22 at 07:41

Wey Gu

575
1
6
11

The conflict has been solved, but now there is a new dependency conflict regarding jackson: com/fasterxml/jackson/databind/cfg/MapperBuilder.streamFactory()Lcom/fasterxml/jackson/core/TokenStreamFactory; @7: areturn Reason: Type 'com/fasterxml/jackson/core/JsonFactory' (current frame, stack[0]) is not assignable to 'com/fasterxml/jackson/core/TokenStreamFactory' (from method signature) – sim Aug 01 '22 at 09:50
@sim you could check dependecy with tools like https://www.jetbrains.com/help/idea/dependencies-analysis.html to see if multiple projects depends on different `jackson`, and if conflic exists, exclude one of them will help you pass this. – Wey Gu Oct 24 '22 at 14:37

Elastic Search Conflict with Spark Connector, rest high level client no such field error

1 Answers1