92

I noticed when a file is executed on Windows (.exe or .dll), it is locked and cannot be deleted, moved or modified.

Linux, on the other hand, does not lock executing files and you can delete, move, or modify them.

Why does Windows lock when Linux does not? Is there an advantage to locking?

Giovanni Galbo
  • 12,963
  • 13
  • 59
  • 78
David Lenihan
  • 921
  • 1
  • 7
  • 3
  • 7
    There is a utility called [WhoLockMe](http://www.dr-hoiby.com/WhoLockMe/index.php) which adds a menu entry to the context menu in the explorer which can display the process(es) locking a given file. Extremely useful when you get weird file-locking errors. Edit: I know that this is does not answer the question, but I thought it was useful enough in the context to warrant a separate answer (as oppposed to just a comment). – JesperE Oct 13 '08 at 07:22

8 Answers8

114

Linux has a reference-count mechanism, so you can delete the file while it is executing, and it will continue to exist as long as some process (Which previously opened it) has an open handle for it. The directory entry for the file is removed when you delete it, so it cannot be opened any more, but processes already using this file can still use it. Once all processes using this file terminate, the file is deleted automatically.

Windows does not have this capability, so it is forced to lock the file until all processes executing from it have finished.

I believe that the Linux behavior is preferable. There are probably some deep architectural reasons, but the prime (and simple) reason I find most compelling is that in Windows, you sometimes cannot delete a file, you have no idea why, and all you know is that some process is keeping it in use. In Linux it never happens.

Oren Shemesh
  • 1,538
  • 1
  • 8
  • 6
  • woah - almost word for word what I was about to write before it showed up saying that there were new answers! – Mez Oct 13 '08 at 07:22
  • What mechanism does Windows use to count the number of processes that have locked the file :) ? – xtofl Oct 13 '08 at 07:38
  • Windows uses "Handles", though the semantics of what a locked file "feels" like is much different. – Eric Tuttleman Oct 13 '08 at 08:53
  • 2
    Note that you can use a tool like Process Explorer in Windows to see what process is using a file/folder. – J c Oct 13 '08 at 09:02
  • 18
    A practical reason in favor of the Linux behavior is that you can update the OS and other software on the system while it's running, and never/rarely reboot (you can even switch the running kernel without reboot, it's just difficult enough that it's only called for in uptime-critical applications). – joelhardi Oct 13 '08 at 09:58
  • 7
    "Windows does not have this capability"... are you sure? The NT kernel is based on refcounting objects with handles and references. – Adam Mitz Oct 13 '08 at 11:38
  • 1
    While it's possible that Windows may technically *have* the capability in current versions, it still doesn't *use* it - my occasional forays into Windows still (in XP, which is NT-based) get the occasional "you can't delete this file because it's in use" error. – Dave Sherohman Oct 13 '08 at 12:31
  • 1
    Windows derived this behavior from its DOS-FAT file system. Each file-content has exactly one file-name and vice versa; there are no hard-linked files; there are no symbolic-linked files; there is one Microsoft-.lnk file format to emulate symbolic links (but files only) with extensions like icons and parameters - it is more like a binary script than a file-link. In the Linux/Unix world exist even unix-sockets, pipes, and other special files. Unix-sockets allow programs to realize similar mutex behavior like files om Windows. – comonad Apr 02 '11 at 18:53
  • 14
    Windows actually has 3 bits you can set which define what another process can do to a file while it's open. A file can be deleted while it is open if the `FILE_SHARE_DELETE` bit is set. I don't think the PE loader (which loads EXEs and DLLs) sets this bit. The handles are reference counted and on a delete the file goes away when the last handle is dropped, but the difference between that and Unix is that NT will block a new file from being created with the same name when this happens. – asveikau Feb 25 '13 at 17:19
  • 2
    What comonad says is completely wrong, NTFS of course uses Hardlinks and always did, symlinks have been added in Windows Vista. It's as well completely wrong that Windows doesn't use ref copunting, it does, just read APIs like CreateFile wher eit's clearly said under which circumstances files are deletable and such. And that's the proper place for the real answer to the question as well: CreateFile has an argument claled dwShareMode which controls mandatory locking of opened files and lets applications decide. The default value is an exclusive lock... – Thorsten Schöning May 16 '15 at 08:20
  • 1
    @asveikau But when you delete a file opened with `FILE_SHARE_DELETE`, a tombstone file entry remains in the directory which you can no longer rename or remove until the file is actually closed. Thus, deleting such a file gets you into quite a bind. It’s better to rename the file to a different location prior to deleting it. (edit: I see that you said that in your comment, sorry for the noise) – binki May 19 '19 at 20:27
  • @joelhardi, any link, resources on how to switch the running kernel without reboot? – Turkhan Badalov Apr 11 '20 at 21:23
32

As far as I know, linux does lock executables when they're running -- however, it locks the inode. This means that you can delete the "file" but the inode is still on the filesystem, untouched and all you really deleted is a link.

Unix programs use this way of thinking about the filesystem all the time, create a temporary file, open it, delete the name. Your file still exists but the name is freed up for others to use and no one else can see it.

Neil Williams
  • 12,318
  • 4
  • 43
  • 40
  • "all the time"? Any examples? – Mez Oct 13 '08 at 07:22
  • 4
    Ask google about "unix secure temporary file" and you'll find enough descriptions of the technique to show that it's well-known and commonly-used. While I don't have any specific examples to give, I dare say any security-conscious app that uses temp files does this. – Dave Sherohman Oct 13 '08 at 12:30
29

Linux does lock the files. If you try to overwrite a file that's executing you will get "ETXTBUSY" (Text file busy). You can however remove the file, and the kernel will delete the file when the last reference to it is removed. (If the machine wasn't cleanly shutdown, these files are the cause of the "Deleted inode had zero d-time" messages when the filesystem is checked, they weren't fully deleted, because a running process had a reference to them, and now they are.)

This has some major advantages, you can upgrade a process thats running, by deleting the executable, replacing it, then restarting the process. Even init can be upgraded like this, replace the executable, and send it a signal, and it'll re-exec() itself, without requiring a reboot. (THis is normally done automatically by your package management system as part of it's upgrade)

Under windows, replacing a file that's in use appears to be a major hassle, generally requiring a reboot to make sure no processes are running.

There can be some problems, such as if you have an extremely large logfile, and you remove it, but forget to tell the process that was logging to that file to reopen the file, it'll hold the reference, and you'll wonder why your disk didn't suddenly get a lot more free space.

You can also use this trick under linux for temporary files. open the file, delete it, then continue to use the file. When your process exits (for no matter what reason -- even power failure), the file will be deleted.

Programs like lsof and fuser (or just poking around in /proc//fd) can show you what processes have files open that no longer have a name.

6

I think linux / unix doesn't use the same locking mechanics because they are built from the ground up as a multi-user system - which would expect the possibility of multiple users using the same file, maybe even for different purposes.

Is there an advantage to locking? Well, it could possibly reduce the amount of pointers that the OS would have to manage, but now a days the amount of savings is pretty negligible. The biggest advantage I can think of to locking is this: you save some user-viewable ambiguity. If user a is running a binary file, and user b deletes it, then the actual file has to stick around until user A's process completes. Yet, if User B or any other users look on the file system for it, they won't be able to find it - but it will continue to take up space. Not really a huge concern to me.

I think largely it's more of a question on backwards compatibility with window's file systems.

Eric Tuttleman
  • 1,320
  • 1
  • 9
  • 16
  • "Windows" in this context is the Windows NT line. This was designed as a multi-user successor to single-user Windows 3.11. Compare to Unix, which is the single-user successor to Multics. – MSalters Oct 13 '08 at 09:36
6

I think you're too absolute about Windows. Normally, it doesn't allocate swap space for the code part of an executable. Instead, it keeps a lock on the excutable & DLLs. If discarded code pages are needed again, they're simply reloaded. But with /SWAPRUN, these pages are kept in swap. This is used for executables on CD or network drives. Hence, windows doesn't need to lock these files.

For .NET, look at Shadow Copy.

MSalters
  • 173,980
  • 10
  • 155
  • 350
2

If executed code in a file should be locked or not is a design decision and MS simply decided to lock, because it has clear advantages in practice: That way you don't need to know which code in which version is used by which application. This is a major problem with Linux default behaviour, which is simply ignored by most people. If system wide libs are replaced, you can't easily know which apps use code of such libs, most of the times the best you can get is that the package manager knows some users of those libs and restarts them. But that only works for general and well know things like maybe Postgres and its libs or such. The more interesting scenarios are if you develop your own application against some 3rd party libs and those get replaced, because most of the times the package manager simply doesn't know your app. And that's not only a problem of native C code or such, it can happen with almost everything: Just use httpd with mod_perl and some Perl libs installed using a package manager and let the package manager update those Perl libs because of any reason. It won't restart your httpd, simply because it doesn't know the dependencies. There are plenty of examples like this one, simply because any file can potentially contain code in use in memory by any runtime, think of Java, Python and all such things.

So there's a good reason to have the opinion that locking files by default may be a good choice. You don't need to agree with that reasons, though.

So what did MS do? They simply created an API which gives the calling application the chance to decide if files should be locked or not, but they decided that the default value of this API is to provide an exclusive lock to the first calling application. Have a look at the API around CreateFile and its dwShareMode argument. That is the reason why you might not be able to delete files in use by some application, it simply doesn't care about your use case, used the default values and therefore got an exclusive lock by Windows for a file.

Please don't believe in people telling you something about Windows doesn't use ref counting on HANDLEs or doesn't support Hardlinks or such, that is completely wrong. Almost every API using HANDLEs documents its behaviour regarding ref counting and you can easily read in almost any article about NTFS that it in deed does support Hardlinks and always did. Since Windows Vista it has support for Symlinks as well and the Support for Hardlinks has been improved by providing APIs to read all of those for a given file etc.

Additionally, you may simply want to have a look at the structures used to describe a file in e.g. Ext4 compared to those of NTFS, which have a lot in common. Both work with the concept of extents, which separates data from attributes like file name, and inodes are pretty much just another name for an older, but similar concept of that. Even Wikipedia lists both file systems in its article.

There's really a lot of FUD around file locking in Windows compared to other OSs on the net, just like about defragmentation. Some of this FUD can be ruled out by simply reading a bit on the Wikipedia.

Thorsten Schöning
  • 3,501
  • 2
  • 25
  • 46
  • The ramblings about version management of shared dependencies are irrelevant. You have to deal with those problems even if updating them requires a reboot. On Windows, it even has a name: DLL hell. Further, the runtime behaviors around it you describe are fully handled by Linux's case where it already has the file loaded into memory and it remains available; the application will keep running with the old version until it restarts, just as when Windows forces a reboot to unlock the file. There's no advantage on that front. – jpmc26 Jan 08 '19 at 14:01
  • DLL hell is of course completely missing the point and we don't have the 90s anymore, just read about things like `WinSxS`and stuff. Additionally, it's not about loading things into memory and keeping them there, Windows does exactly that as well if necessary, it's about knowing and deciding if files should be replaced or not and who is in charge to decide that. The Windows-API simply lets the first user of the file decide and that makes a lot of sense. – Thorsten Schöning Jan 08 '19 at 18:40
  • But that's my point: deciding which version of a DLL to use is part of DLL hell. Call it "dependency hell" if you want to distinguish from some of Windows' old idiosyncratic behaviors. Regardless, defaulting to locking executable files *does not help* with managing shared dependencies. At all. The things that depend on a particular file might not even be running when you try to upgrade it; there's no extra safety. There are 2 choices with shared dependencies: free for all, which risks stuff breaking when you try to run it, or package manager, which risks preventing you from installing things. – jpmc26 Jan 09 '19 at 02:13
  • This question is not about deciding which EXE or DLL to use at all, it's about what happens afterwards by default and why. You are discussing a completely different topic. Locking is used after the decision which EXE or DLL to execute to get some level of additional control by the first user, which is Windows itself in this example, and to tell others about that control. And not letting "others" delete or write to files Windows needs for some reason and therefore locks them, of course is a mechanism for additional coordination. – Thorsten Schöning Jan 09 '19 at 09:32
  • The point is that "others" don't know about a lot of things and are simply ignoring the complexity of a lot of questions. Your example with code from some DLL that might not be running at all currently is such an example, simply because nobody knows. One can either ignore such questions and simply put new files in place or care about the complexity and MS decided to care. That is an arbitrary decision you don't need to follow, but it's not wrong in general as well only because you favour to ignore things. – Thorsten Schöning Jan 09 '19 at 09:36
  • But MS's decision *does not reduce the complexity*. Once a binary is in memory, holding a lock on the file isn't solving any problems, and it simultaneously creates problems by making it difficult for the user to make changes. – jpmc26 Jan 09 '19 at 10:05
  • 1
    Some EXE or DLL is not "in memory" most likely anyway, but mapped into it by default. Mapping requires the file content to be available, so arbitrarily replacing it is considered unsafe by default and one should know what one's doing. Which is obviously not the case if one is surprised by locked EXEs or DLLs. OTOH, all other files are only locked by default, not necessarily, so apps can decide if they allow you write or delete operations, depending on their use case. App's developers should know better than arbitrary users how they use their files and which operations are safe when. – Thorsten Schöning Jan 09 '19 at 14:51
  • Now see, that part about mapping to the file directly should be in your answer, but it's not. Instead, your answer is wrongly asserting that it helps with dependency version management. Linux does not appear to work that way. Although, looks like [MSalters covers it](https://stackoverflow.com/a/197133/1394393). – jpmc26 Jan 09 '19 at 17:25
  • My answer clearly states that locking is a design decision of users of files, because those know best. Mapping a file's content into memory is only one arbitrary aspect of many, many different and in fact can easily be considered an implementation detail which might change at any given time with any version of Windows. So of course I focussed on describing the more general concept behind the reasons to lock sometimes by someone. – Thorsten Schöning Jan 10 '19 at 06:56
  • The first program to launch does not "know best" about locking its executable files. I'm not even sure Windows gives you control over whether binaries are locked at load time to change this, and even if it does, the dev still doesn't "know best" about how a share binary is used. It doesn't know any better than any other program that uses it or the OS or anything. Your reason is invalid and doesn't even make sense. – jpmc26 Jan 10 '19 at 06:59
  • I wasn't talking about first programs and their own executables, I was talking about users of files in general and in case of executing EXEs and DLLs, that first user by default is the loader of Windows itself. Apps don't magically load themselfs obviously. And that concept, let the first user of a file decide always, is a valid strategy in general and, surprise, supported by the already mentioned Windows-API. That doesn't mean all apps need to provide you the choice you want of course as well, it simply depends. You should be more careful trying to follow my arguments. – Thorsten Schöning Jan 10 '19 at 07:53
  • No, you need to say something that answers the question. The question isn't whether it's a "valid approach." The question is, "What problems does it solve?" You have not demonstrated that locking executable files solves any. This is much more often a source of consternation than it helps anything. As for *general* file locking, that's totally irrelevant and not what the question is asking about. Of course a program writing to a file locks it to avoid write conflicts. That solves a problem. You're not discussing any problems that locking binary files while they execute solves. – jpmc26 Jan 10 '19 at 10:22
  • `Of course a program writing to a file locks it to avoid write conflicts.` You mean IF some user of a file decides that concurrent writes are bad at all and such a decision is exactly what MS made for the Windows loader and what is exactly what I told in the answer and to you in too many comments already over and over again? I think I'm deciding this bores me too much now. :-) – Thorsten Schöning Jan 10 '19 at 13:08
  • No. Windows pretty much forces you to acquire a lock to be able to write. But the point is *that's moot* because programs don't rewrite their own binaries during execution. The only realistic scenarios for writing an executable while it's running are self updates and rebuilding during development, and the Windows model *actively hinders* those use cases. That's why your answer is *not relevant*. – jpmc26 Jan 10 '19 at 13:49
  • Simply look at the linked `CreateFile` and `dwShareMode`, no one forces you to aquire a lock IF the first user of a file allows you to write. Rewriting EXEs or DLLs by the same app is not the only use case to protects against, arbitrary OTHER apps writing to the files even by accident is a valid one as well. You simply don't seem to understand that you don't need to come to the same conclusion as MS did, but that doesn't make their claim necessarily wrong. It's simply that you have different priorities. – Thorsten Schöning Jan 10 '19 at 17:31
0

NT variants have the

openfiles

command, which will show which processes have handles on which files. It does, however, require enabling the system global flag 'maintain objects list'

openfiles /local /?

tells you how to do this, and also that a performance penalty is incurred by doing so.

spender
  • 117,338
  • 33
  • 229
  • 351
0

Executables are progressively mapped to memory when run. What that means is that portions of the executable are loaded as needed. If the file is swapped out prior to all sections being mapped, it could cause major instability.

Ken Netherland
  • 1