2

My docker pushes with the latest version appear to be failing almost all the time.

This is what happens with gcloud docker -- push

The push refers to a repository [us.gcr.io/rndb-140315/dream_closet]

80a8fcd2cfa1: Layer already exists

3bca4b8aadf9: Layer already exists

1f3202a15537: Layer already exists

5549f40f950c: Layer already exists

5f70bf18a086: Layer already exists

34d126c63186: Layer already exists

52b4a81ec16b: Layer already exists

6db3819d35cd: Layer already exists

46ecc6c4543a: Layer already exists

dial tcp 108.177.9.82:443: i/o timeout

Here is the docker log:

"Calling POST /v1.24/images/us.gcr.io/rndb-140315/dream_closet/push?tag=" 16-11-15T00:44:46.219497295Z" level=debug msg="hostDir: /etc/docker/certs.d/us.gcr.io"

16-11-15T00:44:46.250418329Z" level=debug msg="hostDir: /etc/docker/certs.d/us.gcr.io"

16-11-15T00:44:46.250519209Z" level=debug msg="Trying to push us.gcr.io/rndb-140315/dream_closet to https://us.gcr.io v2" 16-11-15T00:44:46.446481694Z" level=debug msg="Pushing repository: us.gcr.io/rndb-140315/dream_closet:latest"

16-11-15T00:45:16.709508684Z" level=error msg="Upload failed, retrying: dial tcp 108.177.10.82:443: i/o timeout"

16-11-15T00:45:16.709791097Z" level=error msg="Upload failed, retrying: dial tcp 108.177.10.82:443: i/o timeout"

16-11-15T00:45:16.710004220Z" level=error msg="Upload failed, retrying: dial tcp 108.177.10.82:443: i/o timeout"

16-11-15T00:45:51.711554093Z" level=error msg="Upload failed, retrying: dial tcp 108.177.10.82:443: i/o timeout"

16-11-15T00:46:31.712779966Z" level=error msg="Upload failed, retrying: dial tcp 108.177.10.82:443: i/o timeout"

16-11-15T00:47:17.355180162Z" level=error msg="Attempting next endpoint for push after error: Post https://us.gcr.io/v2/rndb-140315/dream_clos 16-11-15T00:47:17.355302561Z" level=debug msg="Skipping v1 endpoint https://us.gcr.io because v2 registry was detected"

Any ideas on how to resolve? This worked on a previous version.

Community
  • 1
  • 1
BeatingToADifferentRobot
  • 394
  • 3
  • 17
  • _almost all the time_ ? So this means that sometime the upload works properly. If this is the case I would think of a network/traffic problem. I also had sometime difficulty to upload to docker hub. – Marcs Nov 14 '16 at 21:59
  • I've had it work once I believe. It gives me an i/o timeout at different random points. The issue I have is my internet connection tends to be reliable so I don't believe it's my network. – BeatingToADifferentRobot Nov 14 '16 at 23:14
  • Have you checked Docker's log ? [Where is the Docker daemon log?](http://stackoverflow.com/questions/30969435/where-is-the-docker-daemon-log#30970134) – Marcs Nov 14 '16 at 23:23
  • Updated the question with the docker log. It did help in that I was behind the docker client version (1.12.2 vs 1.12.3), but that didn't seem to resolve it. – BeatingToADifferentRobot Nov 15 '16 at 00:51
  • How big is the layer you're trying to push? Can you split it into multiple smaller layers? – Software Engineer Nov 15 '16 at 00:54
  • Google lists it at 125.9 mb, but all the layers are there so it's the last step that seems to fail. – BeatingToADifferentRobot Nov 15 '16 at 01:21

3 Answers3

2

I've just had this same problem, which was caused by a network problem.

I had been using a VPN, it seemed that the specific VPN node through which I was connected couldn't cope with the bandwidth I was pushing at it.

When I disconnected from the VPN, it worked fine.

Rachel
  • 2,858
  • 2
  • 26
  • 30
0

A workaround that resolved the need for this was to do a gcloud app deploy instead of a docker push with a gcloud app deploy --image-url="xxxxxxxxx". In the past gcloud app deploy used to timeout because of the size of the image and the work around was to do a docker push. Now it seems to work fine.

Oddly, as part of the gcloud app deploy it does do a docker push to a different path in the gcloud container registry which seems to work.

BeatingToADifferentRobot
  • 394
  • 3
  • 17
-1

If you have two images with same tag (which i was pushing to cloud).

Solution is:

Inspect your images and find two or more with the same tag: docker images Delete them: docker rmi --force 'image id'

Ref: Docker - What is proper way to rebuild and push updated image to docker cloud?

Community
  • 1
  • 1
Senthil
  • 2,156
  • 1
  • 14
  • 19
  • This isn't really the issue at all. It's a new image. The "layer already exists" is because I've been trying to push it multiple times and the last step fails. – BeatingToADifferentRobot Nov 14 '16 at 22:48
  • it is indeed possible to push to the gcr.io repos without going through gcloud, e.g: docker login -e dummy@example.com -p $(gcloud auth print-access-token) -u _token https://gcr.io docker push [your-image] Ref: http://stackoverflow.com/questions/35601542/gcloud-docker-push-reliability – Senthil Nov 15 '16 at 03:27