16

I have seen many posts about creating a unique filename from the naive %TIME% to the plausible (but insufficient) %RANDOM%. Using wmic os get localdatetime is much better, but it can still fail on multiple CPU/core machines. The following script will eventually fail when run in 5+ shells on a multple core machine.

@ECHO OFF
SETLOCAL ENABLEDELAYEDEXPANSION

FOR /L %%i IN (0, 1, 1000) DO (
    FOR /F "usebackq" %%x IN (`wmic os get localdatetime ^| find "."`) do (set MYDATE=%%x)
    ECHO MYDATE is now !MYDATE!
    IF EXIST testuniq_!MYDATE!.txt (
        ECHO FAILED ON !MYDATE!
        GOTO TheEnd
    )
    COPY NUL >testuniq_!MYDATE!.txt
)

:TheEnd
EXIT /B 0

Does anyone have a reliable way to create a unique file name in a shell script?

lit
  • 14,456
  • 10
  • 65
  • 119
  • 2
    It's not often we get a question in [tag:batch-file] land that warrants over 5 answers. +1 interesting question! – rojo Jan 06 '15 at 21:50
  • You could always use %RANDOM%%RANDOM% to make a longer, less likely number(?) – mattb Jul 21 '23 at 14:17
  • Step 1: Generate a unique ID. See https://stackoverflow.com/questions/4313422/generate-guid-in-windows-with-batch-file Step 2: Use the unique ID as filename – Johan Witters Aug 21 '23 at 10:51
  • @JohanWitters, there are even stronger answers than using a GUID? – lit Aug 21 '23 at 11:34

12 Answers12

13

Any system that relies on a random number can theoretically fail, even one using a GUID. But the world seems to have accepted that GUIDs are good enough. I've seen code posted somewhere that uses CSCRIPT JScript to generate a GUID.

There are other ways:

First I will assume that you are trying to create a unique temporary file. Since the file will be deleted once your process ends, all you must do is establish an exclusive lock on a resource whose name is a derivative of your temp file name. (actually I go in reverse - the temp name is derived from the locked resource name).

Redirection establishes an exclusive lock on the output file, so I simply derive a name from the time (with 0.01 second preciscion), and attempt to lock a file with that name in the user's temp folder. If that fails than I loop back and try again until I succeed. Once I have success, I am guaranteed to have sole ownership of that lock, and all derivitives (unless someone intentionally breaks the system).

Once my process terminates, the lock will be released, and a temp file with the same name could be reused later on. But the script normally deletes the temp file upon termination.

@echo off
setlocal

:getTemp
set "lockFile=%temp%\%~nx0_%time::=.%.lock"
set "tempFile=%lockFile%.temp"
9>&2 2>nul (2>&9 8>"%lockFile%" call :start %*) || goto :getTemp

:: Cleanup
2>nul del "%lockFile%" "%tempFile%"
exit /b


:start
:: Your code that needs the temp file goes here. This routine is called with the
:: original parameters that were passed to the script. I'll simply write some
:: data to the temp file and then TYPE the result.
>"%tempFile%" echo The unique tempfile for this process is "%tempfile%"
>>"%tempFile%" echo(%*
type "%tempFile%"
exit /b

Looping due to name collision should be rare unless you are really stressing your system. If so, you can reduce the chance of looping by a factor of 10 if you use WMIC OS GET LOCALDATETIME instead of %TIME%.


If you are looking for a persistent unique name, then the problem is a bit more difficult, since you cannot maintain the lock indefinitely. For this case I recommend the WMIC OS LocalDateTime approach, coupled with two checks for name collision.

The first check simply verifies the file does not already exist. But this is a race condition - two processes could make the check at the same time. The second check creates the file (empty) and establishes a temporary exclusive lock on it. The trick is to make sure that the lock is maintained for a period of time that is longer than it takes for another process to check if the file exists. I'm lazy, so I simply use TIMEOUT to establish a 1 second wait - way more than should be necessary.

The :getUniqueFile routine expects three arguments - a base name, an extension, and the variable name where the result is to be stored. The base name can include drive and path information. Any path information must be valid, otherwise the routine will enter an infinite loop. That issue could be fixed.

@echo off
setlocal

:: Example usage
call :getUniqueFile "d:\test\myFile" ".txt" myFile
echo myFile="%myFile%"
exit /b

:getUniqueFile  baseName  extension  rtnVar
setlocal
:getUniqueFileLoop
for /f "skip=1" %%A in ('wmic os get localDateTime') do for %%B in (%%A) do set "rtn=%~1_%%B%~2"
if exist "%rtn%" (
  goto :getUniqueFileLoop
) else (
  2>nul >nul (9>"%rtn%" timeout /nobreak 1) || goto :getUniqueFileLoop
)
endlocal & set "%~3=%rtn%"
exit /b

The above should be guaranteed to return a new unique file name for the given path. There is a lot of room for optimization to establish some command to execute during the lock check that takes "long enough" but not "too long"

dbenham
  • 127,446
  • 28
  • 251
  • 390
  • 1
    Your lock file idea seems to be the most foolproof solution. Forcing each instance to establish an exclusive lock guarantees that no two concurrent instances will ever be able to use the same file. It's immune to race conditions. If I were the O.P. this would win. – rojo Jan 08 '15 at 17:55
  • I agree. This seems to be the most industrial strength solution. – lit Jan 09 '15 at 19:50
  • 2
    @dbenham - I am revisiting this topic. The problem is that my "Your code" needs to return an exit code. If it returns a non-zero value, control transfers to the loop to get a unique name. – lit May 12 '15 at 13:29
  • @dbenham - This is working well for me. I had to add one line to remove any space characters because the TIME format might be a 12-hour clock with a leading space. `set "lockFile=%lockFile: =%` – lit May 13 '15 at 13:07
  • @paul - I don't see any reason why you cannot have spaces within the lock file name (someplace other than the end). But it certainly doesn't hurt to remove them. – dbenham May 13 '15 at 16:08
  • @dbenham - You are right that having a space in the lock filename can work. However, the application temp filename is construed from the lock filename. That would be ok to contain a space as well, IFF all of the code that uses the temp filename variable is quoted correctly. Sometimes that is not the case. – lit May 13 '15 at 16:19
  • I found a [description of your way of redirecting](http://stackoverflow.com/a/20484234/2932052), but I'd be happy to understand what `9>` and `8>` means. Sorry for bothering you, but maybe you can point me into the right direction? Thanks :) – Wolf Feb 13 '17 at 15:21
  • @Wolf - I am simply redirecting to unused (undefined) file handles 8 and 9 (at which point they obviously become defined). – dbenham Feb 13 '17 at 16:25
  • @lit - But there is no guarantee that `%TEMP%` does not also contain spaces or other problem characters. So the temp filename path should be properly quoted wherever it is used. – dbenham Feb 13 '17 at 16:27
  • Here is an improvement of 1st solution to allow the script to return a value: `:start CALL :start2 %* SET saved_errorlevel=%ERRORLEVEL% EXIT /B 0 :start` – A. Richard Jun 22 '21 at 08:12
  • Note that these two methods do not seem to work if you are going to read stdin from your script. – Florent Angly Sep 05 '22 at 15:56
  • @FlorentAngly - Not true. Neither technique touches stdin (file handle 0). There should be no problem using either technique in a script that reads from stdin. – dbenham Sep 06 '22 at 13:38
  • 1
    @dbenham Actually, only the second method fails to work with stdin: `script.cmd < input.txt` causes it to enter an infinite loop. But maybe if you can reproduce it, you will also be able to fix it for this use case. – Florent Angly Sep 06 '22 at 14:06
  • @FlorentAngly - Ahh - found the problem. The stupid TIMEOUT command does not work if stdin is redirected. It works if you revert to the old school PING method to sleep for 1 second. So the 4th line from the bottom becomes `2>nul >nul (9>"%rtn%" ping (192.0.2.1 -n 1 -w 1000 & (call ))) || goto :getUniqueFileLoop` – dbenham Sep 08 '22 at 16:14
8

You could use certutil to base64 encode %date% %time% with a %random% seed like this:

@echo off
setlocal

:: generate unique ID string
>"%temp%\~%~n0.%username%.a" echo %username%%date%%time%%random%
>NUL certutil -encode "%temp%\~%~n0.%username%.a" "%temp%\~%~n0.%username%.b"
for /f "usebackq EOL=- delims==" %%I in ("%temp%\~%~n0.%username%.b") do set "unique_id=%%I"
del "%temp%\~%~n0.%username%.a" "%temp%\~%~n0.%username%.b"

echo %unique_id%

In case the same script is being run from the same directory by multiple users, I added %username% to the temp files to avoid further conflict. I suppose you could replace %random% with %username% for the same effect. Then you'd only get a conflict if a single user executes the same code block twice concurrently.

(Edit: added %username% as a seed for uniqueness.)

rojo
  • 24,000
  • 5
  • 55
  • 101
  • Will certutil always produce the same output it the input it the same? If so, then if the original text, %date%%time%%random%, is not unique, then encoding it is not going to help. In my experimentation, the original text is not always unique. – lit Jan 06 '15 at 18:10
  • How many threads are likely to run within the same centisecond by the same user? You could add `%username%` as a seed as well. Or if you want to increase the randomness of `%random%`, then `set /a "rand = (%random% + 1) * (%random% + 1) * %random%"` should result in a random integer in the full signed 32-bit range of -2147483648 to 2147483647. (Because when a number exceeds 32-bits, the cmd interpreter rolls over to negative numbers.) True that there's some weighting and this won't result in derivatives of prime numbers, but you're still getting more randomness nevertheless. – rojo Jan 06 '15 at 18:24
4

The Batch-JScript hybrid script below uses WSH's fso.GetTempName() method that was designed precisely for this purpose:

@if (@CodeSection == @Batch) @then

@echo off

for /F "delims=" %%a in ('cscript //nologo //E:JScript "%~F0"') do set "fileName=%%a"
echo Created file: "%fileName%"
goto :EOF

@end

var fso = new ActiveXObject("Scripting.FileSystemObject"), fileName;
do { fileName = fso.GetTempName(); } while ( fso.FileExists(fileName) );
fso.CreateTextFile(fileName).Close();
WScript.Echo(fileName);
Aacini
  • 65,180
  • 12
  • 72
  • 108
  • This is looking pretty good. I will probably choose this as the answer after some more testing in integration into a larger project. – lit Jan 06 '15 at 19:46
  • 2
    At first I thought that the name generated by GetTempName() was guaranteed to be unique. But based on my reading of various unofficial sources, it appears that the returned name is simply a random number, without any true verification that it is unique. I think you would be better of using JScript to generate a GUID. – dbenham Jan 06 '15 at 20:03
  • 1
    @dbenham: It doesn't matter if `GetTempName()` generate an unique name or not because the additional `FileExists(fileName)` test guarantee that the file is unique. `GetTempName()` is just a method simpler to use than the combination of random/time values. – Aacini Jan 06 '15 at 20:38
  • 2
    But you have a race condition. Two processes could make the check with the same name at the same time, and both would succeed and return duplicate names. (Unlikely, but it is possible) – dbenham Jan 06 '15 at 21:14
  • Re: using JScript to generate a GUID, I added a simpler [powershell example](http://stackoverflow.com/a/27807315/1683264) already. But I've got to say, although it pains me to post 2 solutions while recommending someone else's, I think [dbenham's solution](http://stackoverflow.com/a/27807260/1683264) is the most bullet-proof. Damn it. – rojo Jan 08 '15 at 18:00
4

I like Aacini's JScript hybrid solution. As long as we're borrowing from other runtime environments, how about using .NET's System.GUID with PowerShell?

@echo off
setlocal

for /f "delims=" %%I in ('powershell -command "[string][guid]::NewGuid()"') do (
    set "unique_id=%%I"
)

echo %unique_id%
rojo
  • 24,000
  • 5
  • 55
  • 101
3
@echo off
:: generate a tempfilename in %1 (default TEMPFILE)
:loop
set /a y$$=%random%+100000
set y$$=temp%y$$:~1,2%.%y$$:~-3%
if exist "%temp%\%y$$%" goto loop
SET "y$$=%temp%\%y$$%"&copy nul "%temp%\%y$$%" >nul 2>NUL
:: y$$ now has full tempfile name
if "%1"=="" (set "tempfile=%y$$%") else (set "%1=%y$$%")

Here's a cut-down version of my tempfile generator to create a file named tempnn.nnn in the %temp% directory.

Once tempnn.nnn has been created, then it's simple to create as many further tempfiles as you like for the process by appending a suffix to %y$$%, eg %y$$%.a etc. Of course, that presumes that some other process doesn't randomly create filenames without using this procedure.

Magoo
  • 77,302
  • 8
  • 62
  • 84
  • I like that your method makes damn sure the file doesn't exist with `if exist` and a loop. – rojo Jan 06 '15 at 17:35
  • 1
    I believe there are two issues here. First, %random% is often not random. Secondly, the time between the test (if exist) and the set (copy nul) leaves an opportunity for failure. – lit Jan 06 '15 at 18:18
  • @Paul: The looping on file-exists takes care of the rndom-isn't-random issue. Certainly, there is a theoretical possibility that a file may appear between the test and copy, but there's no way batch can deal with that. It can't create and then test since it may then kill a valid file created by another process, so we're left with the test-and-create method. Limitation of the language - and unless you can guarantee that your process isn't interrupted, a limitation of any language. – Magoo Jan 06 '15 at 18:26
  • I agree with Paul. See [my answer](http://stackoverflow.com/a/27807260/1012053) for a fool proof way to "deal with that". – dbenham Jan 06 '15 at 21:17
1

A long time ago in a galax..newsgroup called alt.msdos.batch, a contributor insisted on posting a QBASIC solution to each issue raised, wrapped in a batch shell and claiming it was batch because it only used standard Microsoft-supplied software.

After a long time, he was cured of his errant ways, but I've been severely allergic to hybrid methods ever since. Sure - if it cures the problem, blah,blah - and sometimes there's no escaping that course (run batch silently, for instance.) Apart from that, I really don't want thebother of learning a new language or two...So that's why I eschew *script solutions. YMMV.

So - here's another approach...

@ECHO OFF
SETLOCAL
:rndtitle
SET "progtitle=My Batch File x%random%x"
TITLE "%progtitle%"
SET "underline="
SET "targetline="
FOR /f "delims=" %%a IN ('TASKLIST /v^|findstr /i /L /c:"======" /c:"%progtitle%"') DO (
 IF DEFINED underline (
  IF DEFINED targetline GOTO rndtitle
  SET "targetline=%%a"
 ) ELSE (SET "underline=%%a")
)

:: Here be a trap. The first column expands to fit the longest image name...
:pidloop
IF "%underline:~0,1%"=="=" SET "underline=%underline:~1%"&SET "targetline=%targetline:~1%"&GOTO pidloop
FOR /f %%a IN ("%targetline%") DO SET /a pid=%%a
ECHO %pid%

GOTO :EOF

Essentially, set the title to something random. Note that x%random%x is used, not simply %random% so that a search for "x123x" won't detect "x1234x".

Next, get a verbose tasklist, locating the line underlining the heading and the title. The first line returned by findstr will be the underline, the next will set targetline and any further returns indicate that the title is not unique, so go back, change it and try again until it is unique.

Finally, get the process ID which is in the second column, noting that the first is of unknown width (hence why the underline is grabbed.)

Result : this process's PID which must be unique and hence can be used as the basis for a unique tempfile name - build it in a directory reserved for the purpose.

Magoo
  • 77,302
  • 8
  • 62
  • 84
  • While I did try to get the PID before, I never came to a workable solution. Screen scraping the output of tasklist.exe looks like it will work. But, I think Powershell might be easier and more direct. – lit May 11 '15 at 02:37
1

I would like to submit another method and find out if there are any holes is it. Please let me know. Thanks.

C:>title my shell a80en333xyb

C:>type getppid.ps1
(Get-WmiObject -Class Win32_Process -Filter "processid='$pid'").ParentProcessId

C:>powershell getppid.ps1
2380

Or, to run it without creating a .ps1 file:

powershell -NoProfile -Command "(Get-WmiObject -Class Win32_Process -Filter "processid=`'$pid`'").ParentProcessId"

To verify that the correct task is found, the title is set to an unlikely value and tasklist.exe is used to search for the value returned by getppid.ps1.

C:>tasklist /v | find "2380"
cmd.exe                       2380 RDP-Tcp#0                  7      8,124 K Running         PHSNT\pwatson                                           0:00:03 my shell a80en333xyb
lit
  • 14,456
  • 10
  • 65
  • 119
1

you can try with guid.bat

for /f "tokens=* delims={}" %%# in ('guid.bat') do set "unique=%%#"
echo %unique%

for usage directly from console:

for /f "tokens=* delims={}" %# in ('guid.bat') do @set "unique=%#"
npocmaka
  • 55,367
  • 18
  • 148
  • 187
0

make the file contents an object, use the pointer memaddress as the first part of your file name and a Rand() as your second part. the memory address will be unique for all objects even with multiple instances running.

  • Nice ideas if I was using a language such as C/C++, Python, Perl, Java, etc. This is a DOS/Windows batch file script. I can write those other things, but trying not to. – lit Jan 06 '15 at 18:13
  • I don't see how this could be guaranteed to be unique if there are multiple machines writing to a shared device. – dbenham Jan 06 '15 at 20:07
  • if you are using a batch script then perhaps "machine name - process ID - Timestamp - random number" should be sufficiently unique. – Bryan Devaney Jan 07 '15 at 13:19
  • This will only work for temporary files, as long as the memory is reserved throughout the life of the temporary file. True, no other process could have that mem address at a given point in time, but the same address could be returned at different points in time for unrelated processes, leading to name collision if the file is persistent (not temporary). – dbenham Jan 09 '15 at 20:05
  • true. OK to be certain of universal uniqueness accross mutiple machines, multiple processes per machine with persistant filenames, you would proably need to go: MachineName (or better: GUID) /ProcessID/TimeStamp/InternalMEMaddress and you can stick a RAND() on the end if you want, but it'd probably be overkill. Also I'm pretty sure you are going to hit the max filename size on windows systems with that.but truncating the memaddress to least significat 8 digits should be enough alongside the timestamp and PID/UID unless your files are very large and you are generating hundreds per second. – Bryan Devaney Jan 15 '15 at 11:21
0

At first some remarks for using a PID as a part of a unique file name:

When you just look for a parent PID this might be ambiguous because when you run that code in a for loop or in a call within your batch file always a new cmd.exe is created and you get the PID of this temporary cmd.exe process.

So you should rather use the PID of your batch "root" cmd.exe process. In order to get that - instead of searching by title in the the task list, which is slow and might also be ambiguous, there is another approach.

You can get the PID of your batch "root" cmd.exe process by using Windows API functions:

  • GetConsoleWindow provides the console window handle
  • GetWindowThreadProcessId provides the PID by using this console window handle

You can run these Windows API functions by using PowerShell:

Add-Type -MemberDefinition @"
  // HWND WINAPI GetConsoleWindow(void)
  [DllImport("kernel32.dll", EntryPoint = "GetConsoleWindow")]
  public static extern IntPtr GetConsoleWindow();

  // DWORD GetWindowThreadProcessId(HWND hWnd, LPDWORD lpdwProcessId)
  [DllImport("user32.dll", EntryPoint = "GetWindowThreadProcessId")]
  public static extern uint GetWindowThreadProcessId(IntPtr hWnd, out uint lpdwProcessId);

  public static uint GetPID()
  {
    IntPtr h = GetConsoleWindow();
    if ((uint)h==0) return 0;
    uint rc = 0;
    GetWindowThreadProcessId(h, out rc);
    return rc;
  }
"@ -Name Win32 -NameSpace System

[System.Win32]::GetPID();

Within your batch file you can run the PowerShell code as an encoded command:

set MyPID=
for /f %%a in ('powershell -NoLogo -NoProfile -NonInteractive -EncodedCommand QQBkAGQALQBUAHkAcABlACAALQBNAGUAbQBiAGUAcgBEAGUAZgBpAG4AaQB0AGkAbwBuACAAQAAiAA0ACgBbAEQAbABsAEkAbQBwAG8AcgB0ACgAIgBrAGUAcgBuAGUAbAAzADIALgBkAGwAbAAiACwAIABFAG4AdAByAHkAUABvAGkAbgB0ACAAPQAgACIARwBlAHQAQwBvAG4AcwBvAGwAZQBXAGkAbgBkAG8AdwAiACkAXQANAAoAcAB1AGIAbABpAGMAIABzAHQAYQB0AGkAYwAgAGUAeAB0AGUAcgBuACAASQBuAHQAUAB0AHIAIABHAGUAdABDAG8AbgBzAG8AbABlAFcAaQBuAGQAbwB3ACgAKQA7AA0ACgBbAEQAbABsAEkAbQBwAG8AcgB0ACgAIgB1AHMAZQByADMAMgAuAGQAbABsACIALAAgAEUAbgB0AHIAeQBQAG8AaQBuAHQAIAA9ACAAIgBHAGUAdABXAGkAbgBkAG8AdwBUAGgAcgBlAGEAZABQAHIAbwBjAGUAcwBzAEkAZAAiACkAXQANAAoAcAB1AGIAbABpAGMAIABzAHQAYQB0AGkAYwAgAGUAeAB0AGUAcgBuACAAdQBpAG4AdAAgAEcAZQB0AFcAaQBuAGQAbwB3AFQAaAByAGUAYQBkAFAAcgBvAGMAZQBzAHMASQBkACgASQBuAHQAUAB0AHIAIABoAFcAbgBkACwAIABvAHUAdAAgAHUAaQBuAHQAIABsAHAAZAB3AFAAcgBvAGMAZQBzAHMASQBkACkAOwANAAoAcAB1AGIAbABpAGMAIABzAHQAYQB0AGkAYwAgAHUAaQBuAHQAIABHAGUAdABQAEkARAAoACkADQAKAHsADQAKACAASQBuAHQAUAB0AHIAIABoACAAPQAgAEcAZQB0AEMAbwBuAHMAbwBsAGUAVwBpAG4AZABvAHcAKAApADsADQAKACAAaQBmACAAKAAoAHUAaQBuAHQAKQBoAD0APQAwACkAIAByAGUAdAB1AHIAbgAgADAAOwANAAoAIAB1AGkAbgB0ACAAcgBjACAAPQAgADAAOwANAAoAIABHAGUAdABXAGkAbgBkAG8AdwBUAGgAcgBlAGEAZABQAHIAbwBjAGUAcwBzAEkAZAAoAGgALAAgAG8AdQB0ACAAcgBjACkAOwANAAoAIAByAGUAdAB1AHIAbgAgAHIAYwA7AA0ACgB9AA0ACgAiAEAAIAAtAE4AYQBtAGUAIABXAGkAbgAzADIAIAAtAE4AYQBtAGUAUwBwAGEAYwBlACAAUwB5AHMAdABlAG0ADQAKAFsAUwB5AHMAdABlAG0ALgBXAGkAbgAzADIAXQA6ADoARwBlAHQAUABJAEQAKAApADsADQAKAA^=^= 2^>nul') do (
  set "MyPID=%%~a"
)

if defined MyPID echo %MyPID%
Konrad
  • 321
  • 2
  • 4
0

generate unique number (datetime stamp)

@echo off
for /f "tokens=2 delims==" %%a in ('wmic OS Get localdatetime /value') do set "dt=%%a"
set "YY=%dt:~2,2%" & set "YYYY=%dt:~0,4%" & set "MM=%dt:~4,2%" & set "DD=%dt:~6,2%"
set "HH=%dt:~8,2%" & set "Min=%dt:~10,2%" & set "Sec=%dt:~12,2%"
rem set "datestamp=%YY%%MM%%DD%" & set "timestamp=%HH%%Min%%Sec%"
set "datestamp=%YYYY%%MM%%DD%" 
set "timestamp=%HH%%Min%%Sec%"
set unique_number=%datestamp%%timestamp%
echo %unique_number%
Alper Ebicoglu
  • 8,884
  • 1
  • 49
  • 55
0

Here is another way. Note that the newly created file will be created with zero (0) length. Your code is responsible for removing it if needed.

C:>TYPE CreateTempFile.bat
FOR /F "delims=" %%A IN ('powershell.exe -NoLogo -NoProfile -Command "(New-TemporaryFile).FullName"') DO (SET "NEWFILE=%%~A")
ECHO New file is "%NEWFILE%"
DIR "%NEWFILE%"
IF EXIST "%NEWFILE%" (DEL "%NEWFILE%")
lit
  • 14,456
  • 10
  • 65
  • 119