How to restrict access to specific secrets in POD ( POD Should access only to specific secrets used by the given application)

Question

I want to access specific secret from POD , I dont want POD to have access to other secrets(which are not needed).

I have created an service account with no access to secrets ( not even "get").

apiVersion: v1
kind: ServiceAccount
metadata:
  name: no-access-to-secrets
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: no-access-to-secrets
rules:
- apiGroups:  [""] 
  verbs:  [""]
  resources:
    - ""
  resourceNames:
    - ""
      
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: no-access-to-secrets
subjects:
- kind: ServiceAccount
  name: no-access-to-secrets
roleRef:
  kind: Role
  name: no-access-to-secrets
  apiGroup: rbac.authorization.k8s.io
 

Expectation: I have assigned this service account to POD, so POD should complain or failed to start due to lack of access to the secret.

...
template:
    metadata:
      labels:
        app: app-name
    spec:
      serviceAccountName: no-access-to-secrets
envFrom:
  - secretRef:
     name: my-secret

...

but to my surprise, the pod is started with reading a secret as env vars. So confused, help me here.

I dont know where I am doing wrong here.