[selenium-users] Using AWS Application Load Balancers for nodes

[manoj mandava]

We have a few AWS EC2 instances spinning up multiple Chrome node Docker
containers. These instances are behind the Application Load Balancer, and
when we're registering the nodes to the hub, we're passing in the DNS name
of the Load Balancer as the REMOTE_HOST parameter. When doing this, I am
only seeing one Chrome web driver registered in the hub, which makes sense
since we're using the same REMOTE_HOST for all the instances. But there are
multiple EC2 instances behind that load balancer and multiple Chrome nodes
on each instance. So, my question is, would the Selenium hub know that
there are multiple Chrome nodes behind that load balancer and send more
requests to the load balancer DNS name or will the hub think there's only
one Chrome node?

[⇜Krishnan Mahadevan⇝]

Am not an expert on AWS, but it looks like you may have a problem here if
you are basically having all the nodes register themselves using the same
LB name through the REMOTE_HOST, then the Hub is going to be seeing just 1
node.

If my memory serves me correctly, I believe that the Hub's registry uses
the REMOTE_HOST when provided (It gets internally translated into a Node
configuration parameter called nodeId) as a way to uniquely identify the
node, in its set of nodes. So if all nodes are registering themselves with
the same value, then the Hub is literally overwriting its registry to refer
to the last node that got registered (so all the previous ones got
removed). That explains why perhaps you are seeing just 1 chrome webdriver
registered in the hub.

To answer your question, the hub will not see the multiple chrome nodes
behind the LB. It will just think that it has 1 node registered to it and
your tests will run according to that (since its a docker image here, you
will run 1 test at a time)

The hub would need to be able to uniquely see each and every individual
node, and it would also need to be able to uniquely reach out to a node
when forwarding a test session. If you put in a load balancer between the
hub and the node, then the LB can very well route a test session that was
intended by the hub to node2, instead of sending it to node1 (As the hub
had wanted)


Thanks & Regards
Krishnan Mahadevan

"All the desirable things in life are either illegal, expensive, fattening
or in love with someone else!"

Post by manoj mandava
We have a few AWS EC2 instances spinning up multiple Chrome node Docker
containers. These instances are behind the Application Load Balancer, and
when we're registering the nodes to the hub, we're passing in the DNS name
of the Load Balancer as the REMOTE_HOST parameter. When doing this, I am
only seeing one Chrome web driver registered in the hub, which makes sense
since we're using the same REMOTE_HOST for all the instances. But there are
multiple EC2 instances behind that load balancer and multiple Chrome nodes
on each instance. So, my question is, would the Selenium hub know that
there are multiple Chrome nodes behind that load balancer and send more
requests to the load balancer DNS name or will the hub think there's only
one Chrome node?
--
You received this message because you are subscribed to the Google Groups
"Selenium Users" group.
To unsubscribe from this group and stop receiving emails from it, send an
To view this discussion on the web visit
https://groups.google.com/d/msgid/selenium-users/dd89eaf1-f9aa-42c5-b82f-4fe2efde2701%40googlegroups.com
<https://groups.google.com/d/msgid/selenium-users/dd89eaf1-f9aa-42c5-b82f-4fe2efde2701%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.

[David]

Interesting situation. May I ask why an LB is used here? Don't have control
over the infrastructure setup (maintained by IT/Ops)? Selenium Grid in a
way is meant to be like an LB, so you shouldn't want to use an LB with Grid
I'd think.

If anything, it would probably make more sense in a case of running
multiple Grid hubs & LB the hubs, with the Grid nodes unmanaged without any
LB. Then the selenium driver clients can use single grid URL with LB
routing to any of the available Grid hubs. But that only makes sense when
you need to run tests at high scale with many grids + nodes.

Post by manoj mandava
We have a few AWS EC2 instances spinning up multiple Chrome node Docker
containers. These instances are behind the Application Load Balancer, and
when we're registering the nodes to the hub, we're passing in the DNS name
of the Load Balancer as the REMOTE_HOST parameter. When doing this, I am
only seeing one Chrome web driver registered in the hub, which makes sense
since we're using the same REMOTE_HOST for all the instances. But there are
multiple EC2 instances behind that load balancer and multiple Chrome nodes
on each instance. So, my question is, would the Selenium hub know that
there are multiple Chrome nodes behind that load balancer and send more
requests to the load balancer DNS name or will the hub think there's only
one Chrome node?

[manoj mandava]

So we're using ECS to manage our node containers, and the only way to use
dynamic port mapping on ECS is through an ELB. So, if I define my container
mapping ports to be 4444:4444, ECS can't scale my containers on the same
EC2 instance since port 4444 is already being used by that first container.
So, we have to use dynamic port mapping where ECS assigns a random port
mapping to each container and then connects that port to the LB. So,
ideally traffic would hit the LB DNS and the load balancer would distribute
the load accordingly. But with the Grid, that's an issue because I can't
register each one of my nodes when using the load balancer.

Post by David
Interesting situation. May I ask why an LB is used here? Don't have
control over the infrastructure setup (maintained by IT/Ops)? Selenium Grid
in a way is meant to be like an LB, so you shouldn't want to use an LB with
Grid I'd think.
If anything, it would probably make more sense in a case of running
multiple Grid hubs & LB the hubs, with the Grid nodes unmanaged without any
LB. Then the selenium driver clients can use single grid URL with LB
routing to any of the available Grid hubs. But that only makes sense when
you need to run tests at high scale with many grids + nodes.

Post by manoj mandava
We have a few AWS EC2 instances spinning up multiple Chrome node Docker
containers. These instances are behind the Application Load Balancer, and
when we're registering the nodes to the hub, we're passing in the DNS name
of the Load Balancer as the REMOTE_HOST parameter. When doing this, I am
only seeing one Chrome web driver registered in the hub, which makes sense
since we're using the same REMOTE_HOST for all the instances. But there are
multiple EC2 instances behind that load balancer and multiple Chrome nodes
on each instance. So, my question is, would the Selenium hub know that
there are multiple Chrome nodes behind that load balancer and send more
requests to the load balancer DNS name or will the hub think there's only
one Chrome node?

[David]

I wonder if there's a way to have the LB deterministically balance/route
the requests to a give ECS instance, so that requests targeted to a
particular ECS instance always goes to that instance, through the LB. If
you can find such a mechanism, you can try to utilize that in the grid node
registration process so that it will always route to the proper node.

As for the grid hub to LB side, the LB has a single hostname to map for
REMOTE_HOST, but if you can find a solution for the above, perhaps you can
add additional aliases to the LB hostname like via AWS Route53, and map a
new alias for each ECS instance's grid node instance (unmapping the entry
when tearing down instances) - so you register with the hub using the
Route53 hostname entry for the ECS's LB rather than the original LB
hostname. That should have the grid see more than 1 node.

On a related note, maybe you might want to use something like this
instead: https://aws.amazon.com/blogs/devops/ui-testing-at-scale-with-aws-lambda/

Post by manoj mandava
So we're using ECS to manage our node containers, and the only way to use
dynamic port mapping on ECS is through an ELB. So, if I define my container
mapping ports to be 4444:4444, ECS can't scale my containers on the same
EC2 instance since port 4444 is already being used by that first container.
So, we have to use dynamic port mapping where ECS assigns a random port
mapping to each container and then connects that port to the LB. So,
ideally traffic would hit the LB DNS and the load balancer would distribute
the load accordingly. But with the Grid, that's an issue because I can't
register each one of my nodes when using the load balancer.

Post by David
Interesting situation. May I ask why an LB is used here? Don't have
control over the infrastructure setup (maintained by IT/Ops)? Selenium Grid
in a way is meant to be like an LB, so you shouldn't want to use an LB with
Grid I'd think.
If anything, it would probably make more sense in a case of running
multiple Grid hubs & LB the hubs, with the Grid nodes unmanaged without any
LB. Then the selenium driver clients can use single grid URL with LB
routing to any of the available Grid hubs. But that only makes sense when
you need to run tests at high scale with many grids + nodes.

Post by manoj mandava
We have a few AWS EC2 instances spinning up multiple Chrome node Docker
containers. These instances are behind the Application Load Balancer, and
when we're registering the nodes to the hub, we're passing in the DNS name
of the Load Balancer as the REMOTE_HOST parameter. When doing this, I am
only seeing one Chrome web driver registered in the hub, which makes sense
since we're using the same REMOTE_HOST for all the instances. But there are
multiple EC2 instances behind that load balancer and multiple Chrome nodes
on each instance. So, my question is, would the Selenium hub know that
there are multiple Chrome nodes behind that load balancer and send more
requests to the load balancer DNS name or will the hub think there's only
one Chrome node?