4

I am using Rails, Tire and Elasticsearch on EC2 single server setup, no sharding or replication (this is the Jenkins CI Server). With a custom initializer as such:

analysis:
  filter:        
    name_synonyms:
      type: synonym
      synonyms_path: <%= Rails.root.join("config", "synonyms", "name_synonyms.txt") %>

This file runs thru Erubis and the synonyms path is converted to something like this: 

/root/workspace/project-project-0f317744a1870b4baf61bbaeb390ebe1/config/synonyms/term_synonyms.txt

When I list the files in the sever I see the following: 

root@ip-XX-XXX-XX-XXX:~/workspace/project-project-0f317744a1870b4baf61bbaeb390ebe1/config/synonyms# ls -la
total 20
drwxr-xr-x 2 root root 4096 Feb 11 18:25 .
drwxr-xr-x 7 root root 4096 Feb 11 18:25 ..
-rw-r--r-- 1 root root 3117 Feb 11 18:25 location_synonyms.txt
-rw-r--r-- 1 root root 3999 Feb 11 18:25 name_synonyms.txt
-rw-r--r-- 1 root root 2144 Feb 11 18:25 term_synonyms.txt

Which is exactly I am expecting, however I see the following error when running rake spec

500 : {"error":"IndexCreationException[[test_facilities] failed to create index]; nested: FailedToResolveConfigException[Failed to resolve config path [/root/workspace/project-project-0f317744a1870b4baf61bbaeb390ebe1/config/synonyms/term_synonyms.txt], tried file path [/root/workspace/project-project-0f317744a1870b4baf61bbaeb390ebe1/config/synonyms/term_synonyms.txt], path file [/etc/elasticsearch/root/workspace/project-project-0f317744a1870b4baf61bbaeb390ebe1/config/synonyms/term_synonyms.txt], and classpath]; ","status":500}

It seems to me Elasticsearch is not capable of loading the files although the path is correct, could be a load order issue, I am really not too sure.

brupm
  • 1,183
  • 1
  • 11
  • 25
  • How many nodes do you have in your cluster? Does this file exist on all nodes? – imotov Feb 11 '13 at 19:11
  • This is a single EC2 instance. It's the CI instance running Jenkins. So it should technically only know about itself. @imotov – brupm Feb 11 '13 at 19:13
  • 1
    Turns out this is related to read/execute perms on the files and parent folders. – brupm May 07 '14 at 21:22
  • 1
    Just had the same issue, chmod o+x $HOME did the trick. Default mode for home folders are 0700 which prevents elasticsearch user from cd'ing into your home folder to read the config file. – Rodrigo Kochenburger May 07 '14 at 21:25

2 Answers2

5

You should look at elasticsearch log files. In my case /var/log/elasticsearch/elasticsearch.log

In my case, I had this:

    [2016-02-26 10:28:26,321][WARN ][cluster.action.shard     ] [Batragon] [myindex-1][2] received shard failed for [myindex-1][2], node[mynode], [P], v[1223], s[INITIALIZING], a[id=myid], unassigned_info[[reason=ALLOCATION_FAILED], at[2016-02-26T09:28:26.168Z], details[failed to create index, failure IndexCreationException[failed to create index]; nested: AccessControlException[access denied ("java.io.FilePermission" "/home/myuser/elasticsearch/my_synonyms.txt" "read")]; ]], indexUUID [haZBzLsuSmmxteIq-1K0vw], message [failed to create index], failure [IndexCreationException[failed to create index]; nested: AccessControlException[access denied ("java.io.FilePermission" "/home/myuser/elasticsearch/my_synonyms.txt" "read")]; ]
[myindex-1] IndexCreationException[failed to create index]; nested: AccessControlException[access denied ("java.io.FilePermission" "/home/myuser/elasticsearch/my_synonyms.txt" "read")];
  at org.elasticsearch.indices.IndicesService.createIndex(IndicesService.java:360)
  at org.elasticsearch.indices.cluster.IndicesClusterStateService.applyNewIndices(IndicesClusterStateService.java:307)
  at org.elasticsearch.indices.cluster.IndicesClusterStateService.clusterChanged(IndicesClusterStateService.java:176)
  at org.elasticsearch.cluster.service.InternalClusterService$UpdateTask.run(InternalClusterService.java:494)
  at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:231)
  at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:194)
  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  at java.lang.Thread.run(Thread.java:745)
  Caused by: java.security.AccessControlException: access denied ("java.io.FilePermission" "/home/myuser/elasticsearch/my_synonyms.txt" "read")
  ...

I solved this by granting read permissions to the java.policy file (in my case /usr/lib/jvm/java-7-oracle/jre/lib/security/java.policy):

grant {
    permission java.io.FilePermission "/home/myuser/elasticsearch/my_synonyms.txt", "read";
};

and restarting elasticsearch service.

You can grant permission at directory level in a more generic way, to any file in a directory, like this, for example:

    grant {
    permission java.io.FilePermission "/home/myuser/current/config/elasticsearch/-", "read";
    permission java.io.FilePermission "/home/myuser/current/config/elasticsearch/", "read";
};

Execute permissions on the whole path up to the file are necessary too. In my case: current and current/config

jogaco
  • 712
  • 8
  • 25
0

The path to the file is included in the ES index mappings. The file itself is expected to live on the ES server when the index is being created, but you have defined the path to this file relative to your Rails application.

Mike Herold
  • 69
  • 2