string sanitizer for filename

Question

I'm looking for a php function that will sanitize a string and make it ready to use for a filename. Anyone know of a handy one?

( I could write one, but I'm worried that I'll overlook a character! )

Edit: for saving files on a Windows NTFS filesystem.

Answer 1

Making a small adjustment to Tor Valamo's solution to fix the problem noticed by Dominic Rodger, you could use:

// Remove anything which isn't a word, whitespace, number
// or any of the following caracters -_~,;[]().
// If you don't need to handle multi-byte characters
// you can use preg_replace rather than mb_ereg_replace
// Thanks @Łukasz Rysiak!
$file = mb_ereg_replace("([^\w\s\d\-_~,;\[\]\(\).])", '', $file);
// Remove any runs of periods (thanks falstro!)
$file = mb_ereg_replace("([\.]{2,})", '', $file);

Answer 2

This is how you can sanitize filenames for a file system as asked

function filter_filename($name) {
    // remove illegal file system characters https://en.wikipedia.org/wiki/Filename#Reserved_characters_and_words
    $name = str_replace(array_merge(
        array_map('chr', range(0, 31)),
        array('<', '>', ':', '"', '/', '\\', '|', '?', '*')
    ), '', $name);
    // maximise filename length to 255 bytes http://serverfault.com/a/9548/44086
    $ext = pathinfo($name, PATHINFO_EXTENSION);
    $name= mb_strcut(pathinfo($name, PATHINFO_FILENAME), 0, 255 - ($ext ? strlen($ext) + 1 : 0), mb_detect_encoding($name)) . ($ext ? '.' . $ext : '');
    return $name;
}

Everything else is allowed in a filesystem, so the question is perfectly answered...

... but it could be dangerous to allow for example single quotes ' in a filename if you use it later in an unsafe HTML context because this absolutely legal filename:

 ' onerror= 'alert(document.cookie).jpg

becomes an XSS hole:

<img src='<? echo $image ?>' />
// output:
<img src=' ' onerror= 'alert(document.cookie)' />

Because of that, the popular CMS software Wordpress removes them, but they covered all relevant chars only after some updates:

$special_chars = array("?", "[", "]", "/", "\\", "=", "<", ">", ":", ";", ",", "'", "\"", "&", "$", "#", "*", "(", ")", "|", "~", "`", "!", "{", "}", "%", "+", chr(0));
// ... a few rows later are whitespaces removed as well ...
preg_replace( '/[\r\n\t -]+/', '-', $filename )

Finally their list includes now most of the characters that are part of the URI rerserved-characters and URL unsafe characters list.

Of course you could simply encode all these chars on HTML output, but most developers and me too, follow the idiom "Better safe than sorry" and delete them in advance.

So finally I would suggest to use this:

function filter_filename($filename, $beautify=true) {
    // sanitize filename
    $filename = preg_replace(
        '~
        [<>:"/\\\|?*]|            # file system reserved https://en.wikipedia.org/wiki/Filename#Reserved_characters_and_words
        [\x00-\x1F]|             # control characters http://msdn.microsoft.com/en-us/library/windows/desktop/aa365247%28v=vs.85%29.aspx
        [\x7F\xA0\xAD]|          # non-printing characters DEL, NO-BREAK SPACE, SOFT HYPHEN
        [#\[\]@!$&\'()+,;=]|     # URI reserved https://www.rfc-editor.org/rfc/rfc3986#section-2.2
        [{}^\~`]                 # URL unsafe characters https://www.ietf.org/rfc/rfc1738.txt
        ~x',
        '-', $filename);
    // avoids ".", ".." or ".hiddenFiles"
    $filename = ltrim($filename, '.-');
    // optional beautification
    if ($beautify) $filename = beautify_filename($filename);
    // maximize filename length to 255 bytes http://serverfault.com/a/9548/44086
    $ext = pathinfo($filename, PATHINFO_EXTENSION);
    $filename = mb_strcut(pathinfo($filename, PATHINFO_FILENAME), 0, 255 - ($ext ? strlen($ext) + 1 : 0), mb_detect_encoding($filename)) . ($ext ? '.' . $ext : '');
    return $filename;
}

Everything else that does not cause problems with the file system should be part of an additional function:

function beautify_filename($filename) {
    // reduce consecutive characters
    $filename = preg_replace(array(
        // "file   name.zip" becomes "file-name.zip"
        '/ +/',
        // "file___name.zip" becomes "file-name.zip"
        '/_+/',
        // "file---name.zip" becomes "file-name.zip"
        '/-+/'
    ), '-', $filename);
    $filename = preg_replace(array(
        // "file--.--.-.--name.zip" becomes "file.name.zip"
        '/-*\.-*/',
        // "file...name..zip" becomes "file.name.zip"
        '/\.{2,}/'
    ), '.', $filename);
    // lowercase for windows/unix interoperability http://support.microsoft.com/kb/100625
    $filename = mb_strtolower($filename, mb_detect_encoding($filename));
    // ".file-name.-" becomes "file-name"
    $filename = trim($filename, '.-');
    return $filename;
}

And at this point you need to generate a filename if the result is empty and you can decide if you want to encode UTF-8 characters. But you do not need that as UTF-8 is allowed in all file systems that are used in web hosting contexts.

The only thing you have to do is to use urlencode() (as you hopefully do it with all your URLs) so the filename საბეჭდი_მანქანა.jpg becomes this URL as your <img src> or <a href>: http://www.maxrev.de/html/img/%E1%83%A1%E1%83%90%E1%83%91%E1%83%94%E1%83%AD%E1%83%93%E1%83%98_%E1%83%9B%E1%83%90%E1%83%9C%E1%83%A5%E1%83%90%E1%83%9C%E1%83%90.jpg

Stackoverflow does that, so I can post this link as a user would do it:
http://www.maxrev.de/html/img/საბეჭდი_მანქანა.jpg

So this is a complete legal filename and not a problem as @SequenceDigitale.com mentioned in his answer.

Answer 3

SOLUTION 1 - simple and effective

$file_name = preg_replace( '/[^a-z0-9]+/', '-', strtolower( $url ) );

• strtolower() guarantees the filename is lowercase (since case does not matter inside the URL, but in the NTFS filename)
• [^a-z0-9]+ will ensure, the filename only keeps letters and numbers
• Substitute invalid characters with '-' keeps the filename readable

Example:

URL:  http://stackoverflow.com/questions/2021624/string-sanitizer-for-filename
File: http-stackoverflow-com-questions-2021624-string-sanitizer-for-filename

---

SOLUTION 2 - for very long URLs

You want to cache the URL contents and just need to have unique filenames. I would use this function:

$file_name = md5( strtolower( $url ) )

this will create a filename with fixed length. The MD5 hash is in most cases unique enough for this kind of usage.

Example:

URL:  https://www.amazon.com/Interstellar-Matthew-McConaughey/dp/B00TU9UFTS/ref=s9_nwrsa_gw_g318_i10_r?_encoding=UTF8&fpl=fresh&pf_rd_m=ATVPDKIKX0DER&pf_rd_s=desktop-1&pf_rd_r=BS5M1H560SMAR2JDKYX3&pf_rd_r=BS5M1H560SMAR2JDKYX3&pf_rd_t=36701&pf_rd_p=6822bacc-d4f0-466d-83a8-2c5e1d703f8e&pf_rd_p=6822bacc-d4f0-466d-83a8-2c5e1d703f8e&pf_rd_i=desktop
File: 51301f3edb513f6543779c3a5433b01c

Answer 4

What about using rawurlencode() ? http://www.php.net/manual/en/function.rawurlencode.php

Here is a function that sanitize even Chinese Chars:

public static function normalizeString ($str = '')
{
    $str = strip_tags($str); 
    $str = preg_replace('/[\r\n\t ]+/', ' ', $str);
    $str = preg_replace('/[\"\*\/\:\<\>\?\'\|]+/', ' ', $str);
    $str = strtolower($str);
    $str = html_entity_decode( $str, ENT_QUOTES, "utf-8" );
    $str = htmlentities($str, ENT_QUOTES, "utf-8");
    $str = preg_replace("/(&)([a-z])([a-z]+;)/i", '$2', $str);
    $str = str_replace(' ', '-', $str);
    $str = rawurlencode($str);
    $str = str_replace('%', '-', $str);
    return $str;
}

Here is the explaination

1. Strip HTML Tags
2. Remove Break/Tabs/Return Carriage
3. Remove Illegal Chars for folder and filename
4. Put the string in lower case
5. Remove foreign accents such as Éàû by convert it into html entities and then remove the code and keep the letter.
6. Replace Spaces with dashes
7. Encode special chars that could pass the previous steps and enter in conflict filename on server. ex. "中文百强网"
8. Replace "%" with dashes to make sure the link of the file will not be rewritten by the browser when querying th file.

OK, some filename will not be releavant but in most case it will work.

ex. Original Name: "საბეჭდი-და-ტიპოგრაფიული.jpg"

Output Name: "-E1-83-A1-E1-83-90-E1-83-91-E1-83-94-E1-83-AD-E1-83-93-E1-83-98--E1-83-93-E1-83-90--E1-83-A2-E1-83-98-E1-83-9E-E1-83-9D-E1-83-92-E1-83-A0-E1-83-90-E1-83-A4-E1-83-98-E1-83-A3-E1-83-9A-E1-83-98.jpg"

It's better like that than an 404 error.

Hope that was helpful.

Carl.

Answer 5

Instead of worrying about overlooking characters - how about using a whitelist of characters you are happy to be used? For example, you could allow just good ol' a-z, 0-9, _, and a single instance of a period (.). That's obviously more limiting than most filesystems, but should keep you safe.

Answer 6

Well, tempnam() will do it for you.

http://us2.php.net/manual/en/function.tempnam.php

but that creates an entirely new name.

To sanitize an existing string just restrict what your users can enter and make it letters, numbers, period, hyphen and underscore then sanitize with a simple regex. Check what characters need to be escaped or you could get false positives.

$sanitized = preg_replace('/[^a-zA-Z0-9\-\._]/','', $filename);

Answer 7

safe: replace every sequence of NOT "a-zA-Z0-9_-" to a dash; add an extension yourself.

$name = preg_replace('/[^a-zA-Z0-9_-]+/', '-', strtolower($name)).'.'.$extension;

so a PDF called

"This is a grüte test_service +/-30 thing"

becomes

"This-is-a-gr-te-test_service-30-thing.pdf"

Answer 8

preg_replace("[^\w\s\d\.\-_~,;:\[\]\(\]]", '', $file)

Add/remove more valid characters depending on what is allowed for your system.

Alternatively you can try to create the file and then return an error if it's bad.

Answer 9

PHP provides a function to sanitize a text to different format

filter_var() with second parameter FILTER_SANITIZE_URL

How to use:

echo filter_var(
   "Lorem Ipsum has been the industry's", FILTER_SANITIZE_URL
); 

Sample output:

LoremIpsumhasbeentheindustry's

Answer 10

Making a small adjustment to Sean Vieira's solution to allow for single dots, you could use:

preg_replace("([^\w\s\d\.\-_~,;:\[\]\(\)]|[\.]{2,})", '', $file)

Answer 11

The following expression creates a nice, clean, and usable string:

/[^a-z0-9\._-]+/gi

Turning today's financial: billing into today-s-financial-billing

Answer 12

These may be a bit heavy, but they're flexible enough to sanitize whatever string into a "safe" en style filename or folder name (or heck, even scrubbed slugs and things if you bend it).

1) Building a full filename (with fallback name in case input is totally truncated):

str_file($raw_string, $word_separator, $file_extension, $fallback_name, $length);

2) Or using just the filter util without building a full filename (strict mode true will not allow [] or () in filename):

str_file_filter($string, $separator, $strict, $length);

3) And here are those functions:

// Returns filesystem-safe string after cleaning, filtering, and trimming input
function str_file_filter(
    $str,
    $sep = '_',
    $strict = false,
    $trim = 248) {

    $str = strip_tags(htmlspecialchars_decode(strtolower($str))); // lowercase -> decode -> strip tags
    $str = str_replace("%20", ' ', $str); // convert rogue %20s into spaces
    $str = preg_replace("/%[a-z0-9]{1,2}/i", '', $str); // remove hexy things
    $str = str_replace(" ", ' ', $str); // convert all nbsp into space
    $str = preg_replace("/&#?[a-z0-9]{2,8};/i", '', $str); // remove the other non-tag things
    $str = preg_replace("/\s+/", $sep, $str); // filter multiple spaces
    $str = preg_replace("/\.+/", '.', $str); // filter multiple periods
    $str = preg_replace("/^\.+/", '', $str); // trim leading period

    if ($strict) {
        $str = preg_replace("/([^\w\d\\" . $sep . ".])/", '', $str); // only allow words and digits
    } else {
        $str = preg_replace("/([^\w\d\\" . $sep . "\[\]\(\).])/", '', $str); // allow words, digits, [], and ()
    }

    $str = preg_replace("/\\" . $sep . "+/", $sep, $str); // filter multiple separators
    $str = substr($str, 0, $trim); // trim filename to desired length, note 255 char limit on windows

    return $str;
}


// Returns full file name including fallback and extension
function str_file(
    $str,
    $sep = '_',
    $ext = '',
    $default = '',
    $trim = 248) {

    // Run $str and/or $ext through filters to clean up strings
    $str = str_file_filter($str, $sep);
    $ext = '.' . str_file_filter($ext, '', true);

    // Default file name in case all chars are trimmed from $str, then ensure there is an id at tail
    if (empty($str) && empty($default)) {
        $str = 'no_name__' . date('Y-m-d_H-m_A') . '__' . uniqid();
    } elseif (empty($str)) {
        $str = $default;
    }

    // Return completed string
    if (!empty($ext)) {
        return $str . $ext;
    } else {
        return $str;
    }
}

So let's say some user input is: .....&lt;div&gt;&lt;/div&gt;<script></script>&amp; Weiß Göbel 中文百强网File name %20 %20 %21 %2C Décor \/. /. . z \... y \...... x ./ “This name” is & 462^^ not &nbsp;&nbsp;&nbsp;&nbsp;&nbsp; = that grrrreat -][09]()1234747) საბეჭდი-და-ტიპოგრაფიული

And we wanna convert it to something friendlier to make a tar.gz with a file name length of 255 chars. Here is an example use. Note: this example includes a malformed tar.gz extension as a proof of concept, you should still filter the ext after string is built against your whitelist(s).

$raw_str = '.....&lt;div&gt;&lt;/div&gt;<script></script>&amp; Weiß Göbel 中文百强网File name  %20   %20 %21 %2C Décor  \/.  /. .  z \... y \...... x ./  “This name” is & 462^^ not &nbsp;&nbsp;&nbsp;&nbsp;&nbsp; = that grrrreat -][09]()1234747) საბეჭდი-და-ტიპოგრაფიული';
$fallback_str = 'generated_' . date('Y-m-d_H-m_A');
$bad_extension = '....t&+++a()r.gz[]';

echo str_file($raw_str, '_', $bad_extension, $fallback_str);

The output would be: _wei_gbel_file_name_dcor_._._._z_._y_._x_._this_name_is_462_not_that_grrrreat_][09]()1234747)_.tar.gz

You can play with it here: https://3v4l.org/iSgi8

Or a Gist: https://gist.github.com/dhaupin/b109d3a8464239b7754a

EDIT: updated script filter for   instead of space, updated 3v4l link

Answer 13

Use this to accept just words (unicode support such as utf-8) and "." and "-" and "_" in string :

$sanitized = preg_replace('/[^\w\-\._]/u','', $filename);

Answer 14

The best I know today is static method Strings::webalize from Nette framework.

BTW, this translates all diacritic signs to their basic.. š=>s ü=>u ß=>ss etc.

For filenames you have to add dot "." to allowed characters parameter.

/**
 * Converts to ASCII.
 * @param  string  UTF-8 encoding
 * @return string  ASCII
 */
public static function toAscii($s)
{
    static $transliterator = NULL;
    if ($transliterator === NULL && class_exists('Transliterator', FALSE)) {
        $transliterator = \Transliterator::create('Any-Latin; Latin-ASCII');
    }

    $s = preg_replace('#[^\x09\x0A\x0D\x20-\x7E\xA0-\x{2FF}\x{370}-\x{10FFFF}]#u', '', $s);
    $s = strtr($s, '`\'"^~?', "\x01\x02\x03\x04\x05\x06");
    $s = str_replace(
        array("\xE2\x80\x9E", "\xE2\x80\x9C", "\xE2\x80\x9D", "\xE2\x80\x9A", "\xE2\x80\x98", "\xE2\x80\x99", "\xC2\xB0"),
        array("\x03", "\x03", "\x03", "\x02", "\x02", "\x02", "\x04"), $s
    );
    if ($transliterator !== NULL) {
        $s = $transliterator->transliterate($s);
    }
    if (ICONV_IMPL === 'glibc') {
        $s = str_replace(
            array("\xC2\xBB", "\xC2\xAB", "\xE2\x80\xA6", "\xE2\x84\xA2", "\xC2\xA9", "\xC2\xAE"),
            array('>>', '<<', '...', 'TM', '(c)', '(R)'), $s
        );
        $s = @iconv('UTF-8', 'WINDOWS-1250//TRANSLIT//IGNORE', $s); // intentionally @
        $s = strtr($s, "\xa5\xa3\xbc\x8c\xa7\x8a\xaa\x8d\x8f\x8e\xaf\xb9\xb3\xbe\x9c\x9a\xba\x9d\x9f\x9e"
            . "\xbf\xc0\xc1\xc2\xc3\xc4\xc5\xc6\xc7\xc8\xc9\xca\xcb\xcc\xcd\xce\xcf\xd0\xd1\xd2\xd3"
            . "\xd4\xd5\xd6\xd7\xd8\xd9\xda\xdb\xdc\xdd\xde\xdf\xe0\xe1\xe2\xe3\xe4\xe5\xe6\xe7\xe8"
            . "\xe9\xea\xeb\xec\xed\xee\xef\xf0\xf1\xf2\xf3\xf4\xf5\xf6\xf8\xf9\xfa\xfb\xfc\xfd\xfe"
            . "\x96\xa0\x8b\x97\x9b\xa6\xad\xb7",
            'ALLSSSSTZZZallssstzzzRAAAALCCCEEEEIIDDNNOOOOxRUUUUYTsraaaalccceeeeiiddnnooooruuuuyt- <->|-.');
        $s = preg_replace('#[^\x00-\x7F]++#', '', $s);
    } else {
        $s = @iconv('UTF-8', 'ASCII//TRANSLIT//IGNORE', $s); // intentionally @
    }
    $s = str_replace(array('`', "'", '"', '^', '~', '?'), '', $s);
    return strtr($s, "\x01\x02\x03\x04\x05\x06", '`\'"^~?');
}


/**
 * Converts to web safe characters [a-z0-9-] text.
 * @param  string  UTF-8 encoding
 * @param  string  allowed characters
 * @param  bool
 * @return string
 */
public static function webalize($s, $charlist = NULL, $lower = TRUE)
{
    $s = self::toAscii($s);
    if ($lower) {
        $s = strtolower($s);
    }
    $s = preg_replace('#[^a-z0-9' . preg_quote($charlist, '#') . ']+#i', '-', $s);
    $s = trim($s, '-');
    return $s;
}

Answer 15

It seems this all hinges on the question, is it possible to create a filename that can be used to hack into a server (or do some-such other damage). If not, then it seems the simple answer to is try creating the file wherever it will, ultimately, be used (since that will be the operating system of choice, no doubt). Let the operating system sort it out. If it complains, port that complaint back to the User as a Validation Error.

This has the added benefit of being reliably portable, since all (I'm pretty sure) operating systems will complain if the filename is not properly formed for that OS.

If it is possible to do nefarious things with a filename, perhaps there are measures that can be applied before testing the filename on the resident operating system -- measures less complicated than a full "sanitation" of the filename.

Answer 16

function sanitize_file_name($file_name) { 
 // case of multiple dots
  $explode_file_name =explode('.', $file_name);
  $extension =array_pop($explode_file_name);
  $file_name_without_ext=substr($file_name, 0, strrpos( $file_name, '.') );    
  // replace special characters
  $file_name_without_ext = preg_quote($file_name_without_ext);
  $file_name_without_ext = preg_replace('/[^a-zA-Z0-9\\_]/', '_', $file_name_without_ext);
  $file_name=$file_name_without_ext . '.' . $extension;    
  return $file_name;
}

Answer 17

/ and .. in the user provided file name can be harmful. So you should get rid of these by something like:

$fname = str_replace('..', '', $fname);
$fname = str_replace('/',  '', $fname);

Answer 18

one way

$bad='/[\/:*?"<>|]/';
$string = 'fi?le*';

function sanitize($str,$pat)
{
    return preg_replace($pat,"",$str);

}
echo sanitize($string,$bad);

Answer 19

$fname = str_replace('/','',$fname);

Since users might use the slash to separate two words it would be better to replace with a dash instead of NULL