I see your point. Generally speaking, a class with final fields is thread-safe just because you cannot write code which modifies those fields directly in runtime. So, no any other thread can modify an instance of such immutable class unexpectedly.
Now, about the cost. There is no specific java bytecode to read/write final fields. Moreover, you can change any final field via Reflection, JNI calls, generated bytecodes (besides the "normal" constructor route). That means that the final fields are not effectively final, they seem to be just a compilation time restriction, a kind of syntax sugar. But, there is one specific aspect of final fields which MAY affect performance, because JMM defines specific semantic for them (usually specialty costs, doesn't it? :) ): http://docs.oracle.com/javase/specs/jls/se7/html/jls-17.html#jls-17.5
It defines synthetic synchronize action "Freeze action" for constructors which contain final fields. That action is a part of "Freeze action - Read of reference to the created object" HB edge which really guarantees that all threads see correctly initialized final fields. But we see that in the real life there is no any significant difference between frozen and non-frozen constructors: http://shipilev.net/blog/2014/all-fields-are-final/
So, the bottom line is that final fields cost almost the same as non-final ones. Local final variables is a pure syntax sugar and cost nothing over non-final locals.
And finally, if you develop a high-performance/low-latency application, don't use immutable objects (don't allocate memory all the time) in the main path of it, use mutable objects (with buffers, pools, etc) instead, because typically even an empty constructor costs about 30ns + minor GC also isn't for free + cache locality is worst in case of intensive memory (re)allocation...