11

We use rancher template for hadoop+yarn, but it seems that hadoop is unable to deal with using container names as hostnames (eg. hadoop_namenode-primary_1).

Caused by: java.net.URISyntaxException: Illegal character in hostname at index 13: http://hadoop_datanode_1:50075/webhdfs/v1/skystore/tmp/devtest_onedir/2016_08_19_02_35_35_32f7/header.json?op=CREATE&user.name=hdfs&namenoderpcaddress=10.42.14.252:8020&overwrite=true

Am I doing it wrong or is there some workaround?

As I see it the problem is caused but using container names as host names while rancher creates containers with underscores. Have no idea how to fix it though...

Ambrish
  • 3,627
  • 2
  • 27
  • 42
Jendas
  • 3,359
  • 3
  • 27
  • 55
  • During the period when the standards were being laid for the valid and invalid hostnames, the common terminal interface was the keyboard of the Teletype (TTY) ASR-33. Now this keyborad never had a underscore(_) key in it (check this pic http://www.pdp8.net/asr33/pics/kbd_top.shtml?large) and hence it was not possible to have a hostname with an underscore(_) in it. Check this link for more details: http://www.quora.com/Domain-Name-System-DNS/Why-are-underscores-not-allowed-in-DNS-host-names. Give hostnames without underscore, it will work. – Amal G Jose Aug 22 '16 at 22:11
  • As much as this is interesting, I don't really think that this will help us. I know that the underscore is the problem and that we need to change it. We have no idea how to force rancher to do that though. – Jendas Aug 23 '16 at 07:43
  • What is your docker version? – Ambrish Aug 23 '16 at 11:46
  • docker version is `1.11.2, build b9f10c9` – Jendas Aug 24 '16 at 08:43

2 Answers2

1

Oracle bug-database states that: Underscore is not a valid character in a hostname according to RFC 2396, RFC 952, and RFC 1123. Please refer this below link:

http://bugs.java.com/bugdatabase/view_bug.do?bug_id=5049974

Better idea could be to replace underscore with hypen in the host name.

Update 1:

hadoop is unable to deal with using container names as hostnames

It is not so. It is about the character underscore in hostname.

Update 2:

We have no idea how to force rancher to do that though

You are not supposed to proceed in that direction. The reason is that, still other applications might throw out the same exception which is again trouble. Best choice would be changing the host name.

Marco99
  • 1,639
  • 1
  • 19
  • 32
  • As much as I like your answer, it doesn't really solve our problem. We can either force rancher to change container name ( which you don't recomend ) or make it use something else as hostname ( which we don't know how to do ) – Jendas Aug 29 '16 at 14:07
  • Please share the linux-name and Hadoop distribution you use. – Marco99 Aug 29 '16 at 14:44
  • We have Ubuntu 14.04 and 16.04 on our machines. The dockerized hadoop is run on Ubuntu 14.04 and Hadoop is used in combination with yarn. More details can be found in rancher community repository - https://github.com/rancher/community-catalog/tree/master/templates/hadoop/0 – Jendas Aug 29 '16 at 18:16
  • Please check if these links are of any use: (1) http://ubuntuhandbook.org/index.php/2014/04/change-hostname-ubuntu1404/ (2) http://ubuntuhandbook.org/index.php/2016/06/change-hostname-ubuntu-16-04-without-restart/ – Marco99 Aug 30 '16 at 03:59
0

There should be nothing wrong with the underscores according to RFC 3986

2.3. Unreserved Characters

Characters that are allowed in a URI but do not have a reserved purpose are called unreserved. These include uppercase and lowercase letters, decimal digits, hyphen, period, underscore, and tilde.

unreserved = ALPHA / DIGIT / "-" / "." / "_" / "~"

Reading a little further we see a possible explanation of why things are breaking

However, URI comparison implementations do not always perform normalization prior to comparison

It should be possible to escape those characters, meaning hadoop_datanode_1 should be equivalent to hadoop%5Fdatanode%5F1. The documentation explicitly says not to construct your URIs this way, at least that is how I read it.

For consistency, percent-encoded octets in the ranges of ALPHA (%41-%5A and %61-%7A), DIGIT (%30-%39), hyphen (%2D), period (%2E), underscore (%5F), or tilde (%7E) should not be created by URI producers and, when found in a URI, should be decoded to their corresponding unreserved characters by URI normalizers.

Community
  • 1
  • 1
chewpoclypse
  • 500
  • 5
  • 20
  • 2
    You are missing sec. 3.2.2 of RFC 3986: "A registered name intended for lookup in the DNS uses the syntax defined in Section 3.5 of [RFC1034] and Section 2.1 of [RFC1123]. Such a name consists of a sequence of domain labels separated by ".", each domain label starting and ending with an alphanumeric character and possibly also containing "-" characters." As for the question, I have no idea, but a search found https://github.com/rancher/rancher/issues/5379 from July. – LHP Aug 25 '16 at 20:00