12

Colleagues!

I'm running php 5.3 (5.3.8) with memcache (2.2.6) client library (http://pecl.php.net/package/memcache) to deal with memcached server. My goal is to have failover solution for sessions engine, namely:

  • Only native php sessions support (no custom handlers)
  • Few memcached servers in the pool

What I expect is that in case if one of memcached servers is down, php will attempt to utilize the second server in the pool [will successfully connect it and become happy], however when first memcached server in the pool is down I'm receiving the following error:

Session start failed. Original message: session_start(): Server 10.0.10.111 (tcp 11211) failed with: Connection refused (111)

while relevant php settings are:

session.save_handler memcache
session.save_path tcp://10.0.10.111:11211?persistent=1&weight=1&timeout=1&retry_interval=10, tcp://10.0.10.110:11211?persistent=1&weight=1&timeout=1&retry_interval=10

and memcache settings (while I think that it's near to standard) are:

Directive   Local Value
memcache.allow_failover 1
memcache.chunk_size 8192
memcache.default_port   11211
memcache.default_timeout_ms 1000
memcache.hash_function  crc32
memcache.hash_strategy  standard
memcache.max_failover_attempts  20

Memcached still running on the second server and perfectly accessible from the WEB server:

telnet 10.0.10.110  11211
Trying 10.0.10.110...
Connected to 10.0.10.110 (10.0.10.110).
Escape character is '^]'.
get aaa
END
quit
Connection closed by foreign host.

So in other words, instead of querying all of the listed servers sequentially it crashes after unsuccessful attempt to connect the first server in the queue. Finally I do realize that there are releases of 3.0.x client library available, however it does not look too reliable for me as it still in beta version.

Please advice how can I get desired behavior with standard PHP, client lib and server.

Thanks a lot!

Best, Eugene

Eugene
  • 362
  • 1
  • 4
  • 11
  • Are you certain that the session.save_path directive allows multiple (comma separated) values? I didn't see any real examples perusing the php docs. – Mike Purcell Feb 23 '12 at 00:40
  • 1
    @Mike For sure it does http://php.net/manual/en/memcached.sessions.php : 'Defines a comma separated of hostname:port entries to use for session server pool, for example "sess1:11211, sess2:11211".' – Eugene Feb 23 '12 at 01:11
  • Thanks for the link, didn't see that page, it should be at least referenced from the session docs, kind of buried under memcached. In the OP you mention that the error states; 'Server 10.0.10.111 (tcp 11211) failed', and you gave example that you can telnet into 10.0.10.110, can you confirm that you can indeed telnet into 10.0.10.111? – Mike Purcell Feb 23 '12 at 01:19
  • @Mike No, there is no issues with the question itself. As you can see from "session.save_path" variable above, it has two associated servers: 10.111 and 10.110. I expect it to connect 10.110 when 10.111 is down. While it fails. **My example is to prove the fact that while having alive second server in the pool (10.110) it crashes after unable to connect first server (10.111)** – Eugene Feb 23 '12 at 01:38
  • Friend, I understand the question. What I am saying is that the error message you are receiving is that the connection is **refused** to server 10.111, but the successful connection via telnet you demonstrated in your question was to 10.110, I would like to see the same confirmation for 10.111 to ensure there are no firewall settings which may be causing your issue. – Mike Purcell Feb 23 '12 at 01:45
  • @Mike Ok, sorry. Yes there are no firewall issues. Once I run server at 10.111 back, error message is gone... Here is really something with servers querying strategy. moreover I believe that when I have tested it yesterday somehow it was working accordingly to desired scenario - no error when server is running on 10.111 **or** 10.110; error message mentioning both servers when both were down. No configuration settings or anything else has been changed from that moment... so it seems to be stochastic process which is really confusing – Eugene Feb 23 '12 at 01:53
  • You'd probably want to write your own session handler. – TheOnly92 Feb 23 '12 at 02:05
  • @TheOnly92 Well, this is something what I would like to avoid... If it supports comma separated servers and has parameter allow_failover (memcache driver) it should have some positive impact, right? Moreover, own handler is not a silver bullet as it will be built on top of the same memcache driver which actually causes this strange behavior - I bet that the error has been thrown by 'memchache' session.save_handler which is obviously a part of the memcache extension... – Eugene Feb 23 '12 at 02:14

3 Answers3

7

Use the Memcached extension. Note that there are two memcache plugins for PHP. One is called Memcache, the other is called Memcached. Yes, that's confusing, but true anyway.

The Memcache plugin supports those complex URL's you're using, with the protocol identifier (tcp) and the parameters (persistency and so on), while the Memcached plugin supports connection pools.

The documentation you're mentioning in the comments above (http://www.php.net/manual/en/memcached.sessions.php) is about the Memcached extension, not about Memcache.

Update: Some interesting read: https://serverfault.com/questions/164350/can-a-pool-of-memcache-daemons-be-used-to-share-sessions-more-efficiently

Community
  • 1
  • 1
Rolf van de Krol
  • 410
  • 3
  • 8
  • I noticed this as well when looking through the docs, but according to said docs, his implementation should work as well. – Mike Purcell Feb 23 '12 at 23:18
  • Well, i can't find in the docs that connection pools should work for the memcache plugin. Actually there isn't much documentation at all about the memcache session handler. – Rolf van de Krol Feb 23 '12 at 23:29
  • Check out these docs: http://php.net/manual/en/memcached.sessions.php, according to the PHP official and comments underneath, it should work. – Mike Purcell Feb 23 '12 at 23:44
  • This documentation and comments do not say that connection pools work for the memcache (note: no 'd') extension. The say it works for the memcached (note: with 'd') extension. – Rolf van de Krol Feb 23 '12 at 23:52
  • It can be inferred, that the capability to supply 2 servers via the same line should work for memcache as it does for memcached. Connection pooling aside, if one server cannot be accessed, the other server in the list should be able to manage the session handling. – Mike Purcell Feb 24 '12 at 00:11
  • I'm sorry, but exactly what line in the documentation makes you infer that supplying 2 servers should work for memcache? – Rolf van de Krol Feb 24 '12 at 00:24
  • Please refer to the comments under the OP. That exact question was my first comment. – Mike Purcell Feb 24 '12 at 00:27
  • Thank you guys. In fact this failover is working for memcache, and all of the mess have been caused by error handling. See my answer for more details – Eugene Feb 24 '12 at 13:46
7

I would like to thank everybody who participated this question, the answer is the following: in reality memcache (not memcached) as session handler supports comma separated servers as the session.save_path, moreover it supports failover. The error mentioned above Session start failed. Original message: session_start(): Server 10.0.10.111 (tcp 11211) failed with: Connection refused (111) had only 8th (Notice) level. In fact engine just informs you about the fact that one of the servers is unavailable (which is logical, as otherwise how will you know?) and then successfully connects to the second server and using it.

So all of the misunderstanding has been caused by weak documentation, memcache/memcached confusions and paranoid (E_ALL) settings of my custom error handler. In the meantime the issue has been resolved by ignoring notices referring to error Connection refused (111) in the session establishing context

Eugene
  • 362
  • 1
  • 4
  • 11
  • Ic, so the message was generated because the server was in fact down, but because the other server was functioning, your session handling was behaving as expected? – Mike Purcell Feb 24 '12 at 16:29
  • Lol, not gonna lie, I feel dumb for not having recognized that earlier. And to clarify, this isn't connection pooling, it's redundancy? – Mike Purcell Feb 24 '12 at 17:14
  • @MikePurcell I feel the same. "this isn't connection pooling, it's redundancy" - sorry not sure what do you mean, can you rephrase please? – Eugene Feb 24 '12 at 18:04
  • As Rolf mentioned, memcacheD supports connection pooling, in that several servers act as one huge pool, so if a server goes it is a transparent failover to the running servers. And because the data is distributed, nothing is lost. With respect to memcache(d) if you list multiple servers, they don't share the same values. – Mike Purcell Feb 24 '12 at 18:11
  • Memcache does the same. In my understanding benefit from memcached usage is the fact that YOU can query specific memcached server in the pool, while memcache decides by itself. Memcache also supports two mapping strategies: http://www.php.net/manual/en/memcache.ini.php#ini.memcache.hash-strategy for keys distribution purpose. What is interesting 3.x version (which still beta) supports:memcache.redundancy and memcache.session_redundancy which supposed to be used to determing hpw many times every key (or session data) will be replicated accross multiple servers – Eugene Feb 24 '12 at 21:18
  • Interesting. Least your issue is resolved. Thanks for the info. – Mike Purcell Feb 24 '12 at 21:22
  • @Mike thank you for the help. To give you even more insight at the moment I'm running memcache (adjusted to ignore server unavailability notices) with http://repcached.lab.klab.org/ (which master2master replicated memcached; its latest version available here http://mdounin.ru); the reason to have master2master replication is the necessity to invalidate cached content across all of the servers (time to time it's seriously needed to delete cached content). Staging tests have been passed, next week it should go to production. (highloaded env), can update this ticket with results of wild testing – Eugene Feb 24 '12 at 21:48
0

You must change the hash strategy

Change your config to 

memcache.hash_strategy  consistent

When you make the hash strategy to consistent memcache copies the data across multiple servers. If one of the servers is down, it retries to copy it on the next request.

boksiora
  • 1,168
  • 13
  • 16