…still lets me delete the span (but not edit it).
You can't achieve this via the contenteditable attribute alone. It is working as intended. The contenteditable attribute does what it says: it allows someone to edit the content of the element.
Consider a jar with jam, inside a box which says you can change the content of the box. The jar has a label which says "you may not change the jam in this jar". This rule does not forbid you from throwing said jar into the trashcan because you never actually change the content of the jar by doing so.
I suggest, like Noel Widmer did, that you try to find another solution to your problem. If you can provide a high fidelity example we might suggest some alternatives.