No, I would not consider this harmful.
The "Zen of Admin" as described in Apress's djangobook seemed to imply an assumption of trust as part of the admin's "philosophy", and paired with the often-repeated "admin is not your app" advice, I too was scared at first and think the Django documentation could point out intended, viable use cases.
Please see my almost identical question Django AdminSite/ModelAdmin for end users?
From Jordan's answer (who I gave the bounty):
There is nothing inherently special
about admin. It behaves just like any
other view. So if it is using
permissions to determine access (for
example, if you set a user's .is_staff
to True but give them access only to
specific permissions) then it will be
equally secure to any view you might
create that uses permissions to
determine access.
...
The people who wrote
django.contrib.admin did not write it
with the assumption that anyone with
an is_staff = True could be trusted as
much as a superuser, or was stupid
enough to never take a look at the
source code of a web page. Although
writing your own views is encouraged,
it is still a robust interface.
Also note Django's relatively recent security update http://www.djangoproject.com/weblog/2010/dec/22/security/
regarding querystring parameters in object lists.
Such an update (quote: "an attacker with access to the admin [...]") is a clear indication that the admin's implementation of the permission system is being constantly scrutinized.