This was the exact question I had and JayakrishnaGunnam-MT's answer was helpful.
I feel like putting it all together was the one missing piece that I needed.
So, the below statement is what gave me the exact result I needed--and I think is the most direct solution to the original question.
az keyvault secret show --name $SecretName --vault-name $KeyVaultName --query value -o tsv
az keyvault secret show --name $SecretName --vault-name $KeyVaultName
get the secret
--query value
query for only the value of the secret
-o tsv
format that query result to its raw value (no double quotes)
If I omit #2, I get more than just the secret value.
If I omit #3, I get just the secret value but with double quotes.
If I have that full statement, I get just the secret value and without double quotes--which is the result I need.