6

I'd like to measure the number of times a Docker image has been downloaded from a Google Artifact registry repository in my GCP project.

Is this possible?

Natan Yellin
  • 6,063
  • 5
  • 38
  • 57

2 Answers2

7

Interesting question.

I think this would be useful too.

I think there aren't any Monitoring metrics (no artifactregistry resource type is listed nor metrics are listed)

However, you can use Artifact Registry audit logs and you'll need to explicitly enable Data Access logs see e.g. Docker-GetManifest.

enter image description here

NOTE I'm unsure whether this can be achieved from gcloud.

Monitoring Developer tools, I learned that Audit Logs are configured in Project Policies using AuditConfig's. I still don't know whether this functionality is available through gcloud (anyone?) but evidently, you can effect these changes directly using API calls e.g. projects.setIamPolicy:

gcloud projects get-iam-policy ${PROJECT}
auditConfigs:
- auditLogConfigs:
  - logType: DATA_READ
  - logType: DATA_WRITE
  service: artifactregistry.googleapis.com
bindings:
- members:
  - user:me
  role: roles/owner
etag: BwXanQS_YWg=

Then, pull something from the repo and query the logs:

PROJECT=[[YOUR-PROJECT]]
REGION=[[YOUR-REGION]]
REPO=[[YOUR-REPO]]

FILTER="
logName=\"projects/${PROJECT}/logs/cloudaudit.googleapis.com%2Fdata_access\"
protoPayload.methodName=\"Docker-GetManifest\"
"

gcloud logging read "${FILTER}" \
--project=${PROJECT} \
--format="value(timestamp,protoPayload.methodName)"

Yields:

2022-03-20T01:57:16.537400441Z  Docker-GetManifest

You ought to be able to create a logs-based metrics for these too.

DazWilkin
  • 32,823
  • 5
  • 47
  • 88
  • This was the first thing I tried, but no luck so far. For some reason there are no `Docker-GetManifest`s in the audit logs. I see logs for `ListRepositories`, possibly only for logged in users in my organization and not other users of the docker registry. (Didn't verify this - just eyeballed the first ten entries or so.) I don't think it's an IAM issue as I seem to have appropriate permissions. Any ideas for troubleshooting this? – Natan Yellin Mar 19 '22 at 23:53
  • It works for me. You must enable Audit Logs for Artifact Registry for Data Read|Write. I'll add an example log query to my answer. – DazWilkin Mar 20 '22 at 02:04
  • No luck, still doesn't work. Is it possible this doesn't work for public images? Is there support I can reach out to? – Natan Yellin Mar 24 '22 at 10:43
  • It would be straightforward for you to test your hypothesis; create a non-public image and see whether the audit logs are reported for it alone. If you Google "Google Cloud Support", the first result is [Google Cloud Support](https://cloud.google.com/support-hub). You can also file an issue using Google's public [issue tracker](https://issuetracker.google.com/issues/new?component=187156&template=1163179). – DazWilkin Mar 24 '22 at 15:30
  • Yeah, true. Will do thanks. – Natan Yellin Mar 25 '22 at 16:18
2

We do not yet have platform logs for Artifact Registry unfortunately, so using the CALs is the only way to do this today. You can also turn the CALs into log-based metrics and get graphs and metrics that way too.

The recommendation to filter by 'Docker-GetManifest' is also correct - it's the only request type for which a Docker Pull always has exactly one. There will be a lot of other requests that are related but don't match 1:1. The logs will have all requests (Docker-Token, 0 or more layer pulls), including API requests like ListRepositories which is called by the UI in every AR region when you load the page.

Unfortunately, the theory about public requests not appearing is correct. CALs are about logging authentication events, and when a request has no authentication whatsover, CALs are not generated.

Lindsay Smith
  • 76
  • 1