0

I am aware of discussions like PHP Session ID Duplication? and How unique is the php session id but I have come up with an issue that seems to act like duplicate sessions and cannot figure out what may be causing it.

There is a Yii 1.1 application running on PHP 5.5, and the setup includes two Apache Web Servers, Load Balanced (BigIp), and two Memcache Servers for User Data caching as well as Session Handling. Average traffic is around 10,000 visitors a day logging into the app.

First incident was reported after a load balancer failover. However, since then, there has been a session logging mechanism implementation over MySQL in order to track generated Sessions to each logged-in user and prevent the case of another user logging in with a previously generated session.

If a duplicate session is captured, the user is logged out, and session is regenerated.

The result so far indicates that almost every day there is at least one or two cases of users having acquired a duplicate session, so the implemented mechanism kicks in to settle it.

Although it is strongly suggested that such an event is highly unlikely to be met in an application's lifetime, I still monitor logs suggesting that something weird is going on. I could use some advice on what could cause the issue.

Another issue that I believe may somehow be connected, is that although one's session expires after 10 minutes of no activity, it is quite often noticed that a user will have logged in 3-4 times within 10 minutes, which in my eyes seem like an indication of memcache randomly losing sessions.

The setup doesn't use sticky sessions, which means each user request is served randomly over server 1 or server 2.

Does it seem like

  1. A resources setup issue? (Apache, BigIP etc.?)
  2. Application issue? Generating over and over same sessions?
  3. Memcache issue?
  4. Something else?

I was making some further searches and I came across this term that I think better suits what I am describing: "Session crossover".

Community
  • 1
  • 1
nikolas
  • 723
  • 2
  • 17
  • 37
  • 1
    Hi, if you are not using sticky sessions in LB and save session data in memcache it's maybe a good idea to save the session data only in one of the servers. maybe thats the reason for "logged in 3-4 times within 10 minutes". (see http://serverfault.com/questions/164350/can-a-pool-of-memcache-daemons-be-used-to-share-sessions-more-efficiently). Also it is possible that memcache loose data (when memcache space is full) – Evil_skunk Jan 17 '16 at 14:57
  • i mentioned the sticky session case, because i ve seen many references for that approach, however, i think that when memcached is in the "game" you don't have to "worry" about this.. although in the case i describe, seems not to working 100% normaly – nikolas Jan 17 '16 at 15:05

3 Answers3

0

Question is to generic to give a concrete answer, try to get more information by inspecting the system and changing things.

Its clear your sessionstorage must be shared if you do not use sticky sessions, maybe memcache is running twice?

Try sticky sessions and see if it makes a difference.

Try other session storage e.g. files in tempfs together with sticky sessions.

Check randomness of sessionids, maybe random generator is tricked by seting the same seeds. Sometimes it help to switch the way php is executed modphp->fcgi

Good luck

key_
  • 577
  • 4
  • 15
  • can you please elaborate on memcache running twice?i understand that my question evolves examination over different apsects, but it hasn't been possible so far to determine if memcache, load balancer, php, etc is to blame. I can only tell for sure, that the case was highly reproducable, during load balancer's failover. At that particular time, almost all users sessions seems to get mixed – nikolas Jan 17 '16 at 20:24
  • according to this https://www.digitalocean.com/community/tutorials/how-to-share-php-sessions-on-multiple-memcached-servers-on-ubuntu-14-04 "Without this Memcached setup, if your application is being load balanced on multiple servers, it would be necessary to configure session stickiness on the load balancer. This maintains user experience and prevents them from being logged off suddenly. Configuring Memcached to handle sessions will ensure all cloud servers in the Memcached pool have the same set of session data, which eliminates the need to be sticky with one server to preserve the session." – nikolas Jan 17 '16 at 20:32
  • I hope you have staging system where you can analyze the problem. So what i suggest is to change your setup and see what happens. E.g. running only one shared memcache server and if that works you know where to search for the problem. – key_ Jan 18 '16 at 07:21
0

Well, just practically thinking. The session-data is stored in a file. As long as this session-file exists, the session is active. Hence, you can determine if a specific session-id is in use.

So what you can do is:

  1. create unique session-id with uniqid();
  2. check if session-file with this unique-id exists, if yes, create another id via uniqid(), if not you found your usable session-id;
  3. Create session via session_id($uniqueId).

Warning, this is a practical approach, I would expect that PHP should not get duplicate session-id's, and when that is the case that should be reported to the PHP-project as a bug to fix.

  • the session data is stored in memcache, and it is supposed to last for at least ten minutes, unless there is any activity. According to this comment, http://stackoverflow.com/a/27298129/1234825 facing a duplicate session should be extemely rare, but still according to logs, there is every day at least a couple of cases where users have been assigned same session id – nikolas Jan 17 '16 at 20:21
  • Two things, first is that there are sites running php that have +mln users a day, so there is definitely a fix. Second is just a hunch, what happens if the app stores the session-id, the next day the user opens the app again, the app will send the session-id to the server, but now this session-id is in use by someone else? I believe normally the app will time-out / delete the session-id and communicate a blank to your server, meaning, no session-id is set. Did you check if your session time-out time at the client is the same time-out time at the server? –  Jan 17 '16 at 20:34
  • you make a good point Peter, i was doing some search now and i came across a term that may describe even better this case "session crossover" i came across it over here http://mail-archives.apache.org/mod_mbox/tomcat-users/200404.mbox/<00cf01c4222d$63c71280$f21e10ac@frodo> so i tend to belive that it is not mainly duplicate session generation, but session crossover among users. The cookie on the client side is configured to expire within 10 minutes of inactivity, and i belive that the same time is configured over server side – nikolas Jan 17 '16 at 20:46
0

How are your memcache servers connected? Are they both used for reading and writing? And by what strategy?

Also I know from Redis that it (randomly) drops keys when it's out of memory, maybe memcache has something similar?

Noto Yota multimedialab
  • 51
  • 4
  • there is no special treatment over one memcache server over the over, they are both used with a weight of 50, and they are both used for reading and writing. i m not familiar with a issue in memcache dropping randomly keys, apart from when it is full, but i really doudt that it reaches anywhere close to its full limit – nikolas Jan 17 '16 at 20:16
  • If you use both memcaches for reading and writing, how do you keep them in sync, is there possibly a latency? – Noto Yota multimedialab Jan 18 '16 at 17:36
  • as i mentioned on another comment, you don't need to use stickyness with memcache https://www.digitalocean.com/community/tutorials/how-to-share-php-sessions-on-multiple-memcached-servers-on-ubuntu-14-04 – nikolas Jan 18 '16 at 19:15