A HashMap is not sorted by either keys or values. What you are looking for is a TreeMap.
For a HashMap, the only guarantee is, that the keys are hashed and put in an array, based on their hash.
The LinkedHashMap, according to the Javadoc, creates an internal LinkedList, and tracks the original insertion order of entries. In other words, if you use LinkedHashMap, you won't, necessariely receive a 'sorted' list at all.
You have two options to work around this: Either use a TreeMap (or derivate thereof), or sort every time, you want to output the values. TreeMaps have an internal sorting, based on their keys. If the keys are compared to each other the way you'd expect (by comparing the Strings) then you get a properly ascending sorting, based on the keys. However this does not solve your problem, that you want to sort the values.
To solve your original problem, use a bidirectional TreeMap. Apache Commons4 implements such a map (https://commons.apache.org/proper/commons-collections/javadocs/api-4.3/org/apache/commons/collections4/bidimap/AbstractDualBidiMap.html#values--)
It allows you to access both a key and a value set. But be aware that this map will not work for you, if your values are not unique. Like keys, all values in a bidirectional map need to be unique, because they need to serve as keys themselves.
From the Javadoc:
This map enforces the restriction that there is a 1:1 relation between keys and values, meaning that multiple keys cannot map to the same value. This is required so that "inverting" the map results in a map without duplicate keys. See the put(K, V) method description for more information.