62

I'm trying to setup a basic web application, which has an associated database, in AWS ECS. Locally I have these setup in different containers, and on ECS, I'd like to have separate task definitions so that I may scale the two separately.

  1. I registered my first task definition as david_mongodb successfully in ECS. It has a container named david_mongodb in it.
  2. Then I attempted to register my second task definition as david_web, which has a container named david_web that links the database via david_mongodb:db.

  3. When I click 'Create', it returns an error:

    Unable to create Task Definition
Linked container 'david_mongodb:db' doesn't exist.

It seems like task definitions can't see container names in other task definitions? I'm thinking putting both david_web and david_mongodb containers in the same task definition would work, but I don't want to do that: it would prevent me from scaling either web app or database separately. This overview seems to confirm that my architecture is recommended...

So how do I link containers that live in different task definitions? Or is there another clever way of handling this?

David Elner
  • 5,091
  • 6
  • 33
  • 49
  • [Amazon ECS Now Supports Networking Modes and Memory Reservation](https://forums.aws.amazon.com/ann.jspa?annID=3980).There is no need to do inter instance networking using ELB – RajDev Aug 19 '16 at 17:27
  • 1
    There is no connection between the question and this answer, having control over the network modes is not directly related to link your tasks running in different containers. – Felipe Plets Jan 15 '17 at 22:08
  • I don't believe the networking mode solves this. You'd still need to discover which node (and the IP) where the task lands. – CashIsClay Nov 14 '16 at 17:51
  • Since Nov. 2017, we can also use private IP address to connect to containers in other tasks, as long as the tasks are defined to use `awsvpc` network mode. Announced here: https://aws.amazon.com/blogs/compute/introducing-cloud-native-networking-for-ecs-containers/ . – kanji Jul 07 '19 at 06:18

2 Answers2

90

Links in an ECS task definition are analogous to Docker links and only work when the containers are part of the same task definition (containers that are part of a single task definition are placed together on the same host). In order to communicate between containers in different task definitions, you'll need a mechanism for discovering where the containers are located (what host) as well as the port for communication.

ECS has integration with Elastic Load Balancing (Application Load Balancers, Network Load Balancers, and Classic Load Balancers) through the service feature, where tasks will be automatically registered in the ELB and deregistered in the ELB appropriately.

ECS also has integration with Route 53 Auto Naming for DNS-based service discovery using A and SRV records. Your service's tasks can be automatically entered into and removed from DNS records.

Service Discovery for Amazon ECS Using DNS describes a different approach where a Lambda function listens to the ECS event stream through CloudWatch Events and updates Route 53 DNS records. This method has been superceded by the Route 53 Auto Naming feature described above.

If you want to avoid load balancers and DNS, another pattern might be an ambassador container (there's a sample called the ecs-task-kite that uses the ECS API) or you might be interested in an overlay network (Weave has a fairly detailed getting started guide for their solution).

Nathan Peck is keeping track of a number of different subjects related to ECS, including service discovery, here.

Samuel Karp
  • 4,373
  • 22
  • 34
  • Good to know. Is there a prescribed mechanism for that communcation? DNS+ELB comes to mind, but is there a solution that doesn't require an ELB? – David Elner Dec 30 '15 at 04:40
  • 1
    I've updated my answer with more details of possible approaches (ELB, ambassador containers, and overlay networks). – Samuel Karp Dec 30 '15 at 05:00
  • With the ELB example, would automatically registering the containers to the ELB have the same effect as it would by linking them in Docker? I believe linking creates env variables inside the container with IPs and ports for other containers. Will the ELB do a similar trick that will be transparent for containers that are already working by linking them? Or is there anything else that needs to be done to get the equivalent behavior? – Bastian Dec 31 '16 at 21:30
  • 1
    I'm not sure to understand but it looks like the ELB only works for http. I don't see how to register a database there. – Bastian Dec 31 '16 at 22:19
  • 1
    Using a service with an ELB does not automatically create environment variables in other tasks; you'll want to pass that in yourself or use Route 53 to have a well-known DNS name map to the load balancer. Application load balancers (ALB) only work with HTTP/S + websockets, but Classic load balancers (ELB) work with HTTP/S or arbitrary TCP. See: https://aws.amazon.com/elasticloadbalancing/ – Samuel Karp Jan 01 '17 at 23:37
  • ```"links": [ "my_backend" ],``` – mmell Oct 18 '19 at 17:11
0

you could now refer to this official best practices guide on networking between Amazon ECS services in a VPC, discussing on the considerations when adopting service discovery, ELB or service mesh for service-to-service communication with ECS.

once
  • 1,369
  • 4
  • 22
  • 32