There had been a race condition which was causing svgedit to err, evident in Chrome when loading with file:// URLs, and now fixed in the master branch on Github.
You won't be able to load svg-editor-es.html locally from a file:// URL--svg-editor-es.html being the original source which relies on ES6 Modules to load its files but problematic as they are not permitted to load locally, causing origin errors to show in the console), but the svg-editor.html file (which is the backward compatible way to use svgedit) appears to be working now after the fix--at least for some basic functionality like making drawings.
Some functionality may not be possible to work, however, due to limitations related to limited permissions with file:// URLs, e.g., loading some images. (I seemed to recall browsers previously preventing files outside of their directory or child directories from loading files in parent directories, but this restriction does not seem to apply now, though there are some warnings I see about Ajax not being able to load some images which svgedit attempts to load.)
As such, even with the above-mentioned recent fix, it might not be possible to fully work offline, unless perhaps you opt to disable the security restrictions on your browser, something one should not do lightly. But it does appear to work for some basic drawings at least.
While I figure this may address your direct question about why it doesn't work without a server, there is also another approach to working "offline" which, though it would need a server to initially serve the files, may allow svgedit to store the application files to work completely offline the next time you visit that URL in the browser--and not run into problems with browser security restrictions. Browsers nowadays can work offline even when served from a server (done by something called "service workers"--see https://caniuse.com/#feat=serviceworkers for the browsers that support this).
Service workers are, however, not all that easy to cobble together, and though you should be able to track any future progress on this by subscribing to the issue at https://github.com/SVG-Edit/svgedit/issues/243 (as it is already a requested feature), there is no one currently undertaking to implement this at this time. Hopefully someone will be inspired to implement this.
By the way, if you install svgedit using "npm" (a tool which becomes available if you install Node), svgedit has a start script which you can invoke from the command line with npm start from within the svgedit folder, and that will run a local (Node) server for you, specifically a simple static file server which will simply allow you to load svgedit from http URLs (i.e., http://localhost:8000/editor/svg-editor.html or http://127.0.0.1:8000/editor/svg-editor.html; you can also use the ES6 Modules file if you are on a modern browser: http://localhost:8000/editor/svg-editor-es.html )--without your needing to install any other server.
