If Tomcat runs as a Windows service (created using the Windows Tomcat installer) you can't simply enable JMX using -Dcom.sun.management.jmxremote. You also need to set -Dcom.sun.management.jmxremote.port=<port>. This is, among other places, explained here: Unable to use JConsole with Tomcat running as windows service
However, I haven't found an explanation anywhere WHY this is the way it is.
I was a bit lost and confused when I wrote the question. Here's what I've learned in the meantime. Usually the Tomcat Windows service runs under the local system account. That's the main reason for all the fuss.
If you simply set -Dcom.sun.management.jmxremote in the service config's JVM settings JMX will be enabled indeed. If you have a JMX client that runs in the same JVM instance (e.g. because it's baked into the application itself) you can get a hold of the MBeanServer like so: MBeanServerFactory.findMBeanServer(<specific-agent-ID-or-null>). However, JConsole won't list this Tomcat process. It's all explained here: https://blogs.oracle.com/nbprofiler/entry/monitoring_java_processes_running_as.
As explained elsewhere, if this is not good enough you also need to set -Dcom.sun.management.jmxremote.port=<port>. Then you can connect to JMX with JConsole using localhost:<port>. From Java code this can be achieved using:
JMXServiceURL target = new JMXServiceURL("service:jmx:rmi:///jndi/rmi://localhost:<port>/jmxrmi");
JMXConnector connector = JMXConnectorFactory.connect(target);
connector.getMBeanServerConnection();