Is it possible to create a SWC file without using the Flex framework?
I just want bare-bone AS3.
Yes we are not forced into using flex, in fact Adobe doesn't even support Flex as their product officially anymore as it is now an open-source apache project. http://blogs.apache.org/flex/
The compiler itself for flash is open-source and free to use, that is why there are many third party IDEs and development tools that can also produce SWC libraries. The compiler just requires a special xml file in a zip in order to make a swc. So if you want to avoid doing this manually to the spec its just a matter of choosing a gui way to do this.
One of the most popular one open-source gui ways atm I believe is Flash Develop http://www.flashdevelop.org/ which has a plugin to do what you want. http://www.flashdevelop.org/community/viewtopic.php?f=4&t=2987
This IDE is highly recommended but if you need something more cross platform, I recommend Intellij Idea which is a great as3 and java ide, since you know what jar files are.
"only have access/knowledge of the classes I've declared as public?"
The classes in your swc will be no different to being part of your main project so if you create a swc with public or private it will be public or private the same way. To be honest though most code shared by blogs and repositories in the community are just raw *.as files, swc is handy however for shared libraries in a team and can make this more organised.