I'm using GCP compute Engine with Container Optimized OS. I enabled Logging and Monitoring on that instance.
Logging and Monitoring has write permissions
But it always show Not applicable for agents in monitoring dashboard.
If I SSH in my instance and run docker images then I see output.
These two images and one which is mine application images.
At first look I see there is only stackdriver-logging-agent not any monitoring agent automatically installed. So I searched a little bit and find image for monitoring gcr.io/stackdriver-agents/stackdriver-monitoring-agent I installed that image in my docker and run it as a container, but still there is no monitoring for that instance same "not applicable** message.
Here is the logs for stackdriver-monitoring-agent.
option = Hostname; value = ;
Created new plugin context.
write_gcm: inside module_register for stackdriver_agent/5.5.2-365+docker3.xenial
plugin_load: plugin "write_gcm" successfully loaded.
plugin_load: plugin "stackdriver_agent" successfully loaded.
type = stackdriver_agent, key = Hostname, value = container.
Found a configuration for the `stackdriver_agent' plugin, but the plugin isn't loaded or didn't register a configuration callback.
Done parsing `/opt/stackdriver/collectd/share/collectd/types.db'
interval_g = 60.000;
timeout_g = 2;
hostname_g = ;
write_gcm: Doing GET request: url http://169.254.169.254/computeMetadata/v1beta1/instance/hostname, body (null), num_headers 1
write_gcm: Elapsed time for curl operation was 0.00183108 seconds.
write_gcm: Unsuccessful HTTP request 404: <!DOCTYPE html>
<html lang=en>
<meta charset=utf-8>
<meta name=viewport content="initial-scale=1, minimum-scale=1, width=device-width">
<title>Error 404 (Not Found)!!1</title>
<style>
*{margin:0;padding:0}html,code{font:15px/22px arial,sans-serif}html{background:#fff;color:#222;padding:15px}body{margin:7% auto 0;max-width:390px;min-height:180px;padding:30px 0 15px}* > body{background:url(//www.google.com/images/errors/robot.png) 100% 5px no-repeat;padding-right:205px}p{margin:11px 0 22px;overflow:hidden}ins{color:#777;text-decoration:none}a img{border:0}@media screen and (max-width:772px){body{background:none;margin-top:0;max-width:none;padding-right:0}}#logo{background:url(//www.google.com/images/branding/googlelogo/1x/googlelogo_color_150x54dp.png) no-repeat;margin-left:-5px}@media only screen and (min-resolution:192dpi){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat 0% 0%/100% 100%;-moz-bord
write_gcm: wg_get_from_metadata_server failed fetching http://169.254.169.254/computeMetadata/v1beta1/instance/hostname
write_gcm: Doing GET request: url http://169.254.169.254/latest/meta-data/hostname, body (null), num_headers 0
write_gcm: Elapsed time for curl operation was 0.000778606 seconds.
write_gcm: Unsuccessful HTTP request 404: <!DOCTYPE html>
<html lang=en>
<meta charset=utf-8>
<meta name=viewport content="initial-scale=1, minimum-scale=1, width=device-width">
<title>Error 404 (Not Found)!!1</title>
<style>
*{margin:0;padding:0}html,code{font:15px/22px arial,sans-serif}html{background:#fff;color:#222;padding:15px}body{margin:7% auto 0;max-width:390px;min-height:180px;padding:30px 0 15px}* > body{background:url(//www.google.com/images/errors/robot.png) 100% 5px no-repeat;padding-right:205px}p{margin:11px 0 22px;overflow:hidden}ins{color:#777;text-decoration:none}a img{border:0}@media screen and (max-width:772px){body{background:none;margin-top:0;max-width:none;padding-right:0}}#logo{background:url(//www.google.com/images/branding/googlelogo/1x/googlelogo_color_150x54dp.png) no-repeat;margin-left:-5px}@media only screen and (min-resolution:192dpi){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat 0% 0%/100% 100%;-moz-bord
write_gcm: wg_get_from_metadata_server failed fetching http://IP_ADDRESS/latest/meta-data/hostname
Unable to contact metadata server to detect cloud provider
write_gcm: Cloud provider not specified and autodetect failed.
write_gcm: wg_monitored_resource_create failed.
write_gcm: Tearing down context.
write_gcm: wg_init: wg_context_create failed.
Initialization of plugin `write_gcm' failed with status -1. Plugin will be unloaded.
plugin_read_thread: Handling `stackdriver_agent'.
Created new plugin context.
Created new plugin context.
plugin_dispatch_values: No write callback has been registered. Please load at least one output plugin, if you want the collected data to be stored.
plugin_dispatch_values: time = 1656741908.379; interval = 60.000; host = ; plugin = agent; plugin_instance = ; type = derive; type_instance = uptime;
uc_insert: Added /agent/derive-uptime to the cache.
Filter subsystem: Built-in target `write': Dispatching value to all write plugins failed with status 2 (ENOENT). Most likely this means you didn't load any write plugins.
Available write targets: [none]
Created new plugin context.
plugin_dispatch_values: time = 1656741908.379; interval = 60.000; host = ; plugin = agent; plugin_instance = ; type = gauge; type_instance = memory_usage;
uc_insert: Added /agent/gauge-memory_usage to the cache.
Available write targets: [none]
plugin_read_thread: read-function of the `stackdriver_agent' plugin took 0.000378 seconds.
plugin_read_thread: Effective interval of the `stackdriver_agent' plugin is 60.000 seconds.
plugin_read_thread: Next read of the `stackdriver_agent' plugin at 1656741968.379.
Error: one or more plugin init callbacks failed.
Initialization complete, entering read-loop.
I don't know I'm doing it write or wrong, I just want to monitoring my container in GCP. Please let me know what I'm missing.
