3

I'm new to troubleshooting load distribution on Load-balancer and need your help to troubleshoot one issue. There are 11 members of the pool but I see the request are going to only 2-3 at a time. When those servers gets exhausted due to load,I see their monitors going down and the request starts going to other 2-3 members. I checked persistence is not turned ON and the other members are/were healthy too. Can please someone guide me what to check next to figure out the issue ?

Below are the some setting on the load balancer for the pool . 

allow-nat yes
allow-snat yes
app-service none
autoscale-group-id none
description "Flux Pool for p6_pool"
gateway-failsafe-device none
ignore-persisted-weight disabled
ip-tos-to-client pass-through
ip-tos-to-server pass-through
link-qos-to-client pass-through
link-qos-to-server pass-through
load-balancing-mode least-connections-member
metadata none
min-active-members 0
min-up-members 0
min-up-members-action failover
min-up-members-checking disabled
monitor tcp
partition Common
profiles none
queue-depth-limit 0
queue-on-connection-limit disabled
queue-time-limit 0
reselect-tries 0
service-down-action none
slow-ramp-time 10

Sanitized output from the pool and virtual configuration 

uname@(ab-f5-2)(cfg-sync In Sync)(Active)(/Common)(tmos)# list ltm virtual vs_virtual_name_int_https
ltm virtual vs_virtual_name_int_https {
    destination 172.ip_address:443
    ip-protocol tcp
    mask 255.255.255.255
    pool p_poolName_8443
    profiles {
        http { }
        serverssl-insecure-compatible {
            context serverside
        }
        star.mc-int.vals19 {
            context clientside
        }
        tcp-lan-optimized {
            context serverside
        }
        ab-tcp-wan-optimized {
            context clientside
        }
    }
    rules {
        CIP
    }
    source 0.0.0.0/0
    source-address-translation {
        pool Int-SNAT
        type snat
    }
    translate-address enabled
    translate-port enabled
    vs-index 1999
}

List ltm pool pool_name output 

ltm pool p_poolName_port {
    description "Flux Pool for p_poolName_port"
    load-balancing-mode least-connections-member
    members {
        172.ip.11:port {
            address 172.ip.11
            session monitor-enabled
            state up
        }
        172.ip.2:port {
            address 172.ip.2
            session monitor-enabled
            state up
        }
        172.ip.1:port {
            address 172.ip.1
            session monitor-enabled
            state up
        }
        172.ip.64:port {
            address 172.ip.64
            session monitor-enabled
            state up
        }
        172.ip.68:port {
            address 172.ip.68
            session monitor-enabled
            state up
        }
        172.ip.74:port {
            address 172.ip.74
            session monitor-enabled
            state up
        }
        172.ip.180:port {
            address 172.ip.180
            session monitor-enabled
            state up
        }
        172.ip.188:port {
            address 172.ip.188
            session monitor-enabled
            state up
   }
        172.ip.5:port {
            address 172.ip.5
            session monitor-enabled
            state up
        }
        172.ip.6:port {
            address 172.ip.6
            session monitor-enabled
            state up
        }
        172.ip.7:port {
            address 172.ip.7
            session monitor-enabled
            state up
        }
    }
    monitor tcp
}

ltm rule CIP Output 

ltm rule CIP {
    when HTTP_REQUEST {

HTTP::header insert CIP [IP::client_addr]
HTTP::header insert x-CIP [IP::client_addr]

}
}
Learner
  • 1,544
  • 8
  • 29
  • 55
  • Are the members configured with different `Priority Group` values ? It's easy to check with the GUI. – Eugène Adell Jan 04 '20 at 19:41
  • Thanks for responding Eugene, i had verified that too the priority of all members is same ( priority group 0) and min-active-member value is 0 too – Learner Jan 05 '20 at 20:26
  • There are unexpected behaviors when using One Connect, see [K10430: Causes of Uneven Load balancing](https://support.f5.com/csp/article/K10430) and its related pages. Please check this setting at the VS level. – Eugène Adell Jan 05 '20 at 21:31
  • Can you show a sanitised output of the VS and pool config? `tmsh list ltm virtual ` and `tmsh list ltm pool ` – Paul Dawson Jan 07 '20 at 22:49
  • Hello Paul, Thanks for replying, adding the sanitised output of both in the question – Learner Jan 08 '20 at 18:27
  • @EugèneAdell : Thanks, for adding , I learned about the one conect profile. But I see that is not enabled in this case – Learner Jan 08 '20 at 18:35
  • 1
    You have an iRule called CIP bound to the VS, what's it doing? `tmsh list ltm rule CIP` – Paul Dawson Jan 08 '20 at 20:28
  • @PaulDawson, Thanks for responding, The rule is just to forward the client IP as the source IP address – Learner Jan 09 '20 at 21:27
  • How many connections are established at the same time ? Stats don't have any meaning when there is insufficient data. – Eugène Adell Jan 13 '20 at 14:04
  • @EugèneAdell, thanks for responding. I'm not sure about the same time but I see within a minute 50 requests are being served. – Learner Jan 13 '20 at 20:16
  • OK, I believe you have enough traffic to see the balancing happen. – Eugène Adell Jan 13 '20 at 21:30
  • @EugèneAdell, just wondering if the number of requests are not enough at the same time but those are over the time, will it not distribute to all . For example, there are say 100 request in one hour and one instance is capable to serve all . Will the request not be evenly distributed .. In my opinion it will, – Learner Jan 14 '20 at 05:26

1 Answers1

0

It was found later that OneConnect profile was turned on and most of the time same connection(s) were getting reused.

Learner
  • 1,544
  • 8
  • 29
  • 55