55

I have user entries as filenames. Of course this is not a good idea, so I want to drop everything except [a-z], [A-Z], [0-9], _ and -.

For instance:

my§document$is°°   very&interesting___thisIs%nice445.doc.pdf

should become

my_document_is_____very_interesting___thisIs_nice445_doc.pdf

and then ideally

my_document_is_very_interesting_thisIs_nice445_doc.pdf

Is there a nice and elegant way for doing this?

marcgg
  • 65,020
  • 52
  • 178
  • 231

7 Answers7

66

I'd like to suggest a solution that differs from the old one. Note that the old one uses the deprecated returning. By the way, it's anyway specific to Rails, and you didn't explicitly mention Rails in your question (only as a tag). Also, the existing solution fails to encode .doc.pdf into _doc.pdf, as you requested. And, of course, it doesn't collapse the underscores into one.

Here's my solution:

def sanitize_filename(filename)
  # Split the name when finding a period which is preceded by some
  # character, and is followed by some character other than a period,
  # if there is no following period that is followed by something
  # other than a period (yeah, confusing, I know)
  fn = filename.split /(?<=.)\.(?=[^.])(?!.*\.[^.])/m

  # We now have one or two parts (depending on whether we could find
  # a suitable period). For each of these parts, replace any unwanted
  # sequence of characters with an underscore
  fn.map! { |s| s.gsub /[^a-z0-9\-]+/i, '_' }

  # Finally, join the parts with a period and return the result
  return fn.join '.'
end

You haven't specified all the details about the conversion. Thus, I'm making the following assumptions:

  • There should be at most one filename extension, which means that there should be at most one period in the filename
  • Trailing periods do not mark the start of an extension
  • Leading periods do not mark the start of an extension
  • Any sequence of characters beyond A–Z, a–z, 0–9 and - should be collapsed into a single _ (i.e. underscore is itself regarded as a disallowed character, and the string '$%__°#' would become '_' – rather than '___' from the parts '$%', '__' and '°#')

The complicated part of this is where I split the filename into the main part and extension. With the help of a regular expression, I'm searching for the last period, which is followed by something else than a period, so that there are no following periods matching the same criteria in the string. It must, however, be preceded by some character to make sure it's not the first character in the string.

My results from testing the function:

1.9.3p125 :006 > sanitize_filename 'my§document$is°°   very&interesting___thisIs%nice445.doc.pdf'
 => "my_document_is_very_interesting_thisIs_nice445_doc.pdf"

which I think is what you requested. I hope this is nice and elegant enough.

Anders Sjöqvist
  • 3,372
  • 4
  • 21
  • 22
  • Getting an "undefined (?...) sequence..." when I attempt to use the code. Any limitations with ruby version? – JP. Dec 12 '12 at 15:37
  • @JP. Sorry for the extremely late reply, and you've probably figured it out yourself by now. Haven't tested it, but I believe that look-behinds (which is what the question mark indicates) appeared in Ruby 1.9. So yes, there are limitations. See for example http://stackoverflow.com/q/7605615/1117365 – Anders Sjöqvist Nov 13 '13 at 15:56
  • 1
    Is better with this code `fn[0] = fn[0].parameterize` and after `return fn.join '.'` – Adriano Resende Mar 08 '17 at 03:16
32

From http://web.archive.org/web/20110529023841/http://devblog.muziboo.com/2008/06/17/attachment-fu-sanitize-filename-regex-and-unicode-gotcha/:

def sanitize_filename(filename)
  filename.strip.tap do |name|
   # NOTE: File.basename doesn't work right with Windows paths on Unix
   # get only the filename, not the whole path
   name.gsub!(/^.*(\\|\/)/, '')

   # Strip out the non-ascii character
   name.gsub!(/[^0-9A-Za-z.\-]/, '_')
  end
end
max pleaner
  • 26,189
  • 9
  • 66
  • 118
miku
  • 181,842
  • 47
  • 306
  • 310
  • Thanks for the link! BTW, in the article you linked, the poster says that this function has problem. – marcgg Dec 21 '09 at 10:33
  • 4
    the `name.gsub!(/[^0-9A-Za-z.\-]/, '_')` is the only part I have used after 5 years :D – Aleks Aug 21 '14 at 15:08
  • 4
    Won't the use of `gsub!` cause the function to return `nil` if no replacement is performed? If so, won't this now create a need to assign the value of the gsub'd string to a new variable and test for nil before returning anything? – wgp Aug 22 '14 at 17:35
  • looks like this matches filenames with numbers such as '2015-03-09-ruby-block-procs-and-method-call.md' – random-forest-cat Mar 10 '15 at 01:23
  • 1
    `gsub!` will return `nil` if there are no matches but it is done within the context of the block passed to `strip`. That block finishes and strip returns to `filename`, which is what the method returns. Funky but just fine. – Huliax Mar 10 '16 at 16:34
26

In Rails you might also be able to use ActiveStorage::Filename#sanitized:

ActiveStorage::Filename.new("foo:bar.jpg").sanitized # => "foo-bar.jpg"
ActiveStorage::Filename.new("foo/bar.jpg").sanitized # => "foo-bar.jpg"
Joshua Pinter
  • 45,245
  • 23
  • 243
  • 245
morgler
  • 1,669
  • 1
  • 18
  • 26
  • 2
    This is sexy. I was using `parameterize` before but it was a little too heavy handed and would remove "safe" characters like spaces and the ampersand. But this doesn't do that. e.g. `ActiveStorage::Filename.new("foo:bar & baz.jpg").sanitized #=> "foo-bar & baz.jpg"`. Nice! Also, you can easily add an initializer that monkey patches `String` and adds a `String#sanitized` method that essentially just calls this, so you can do something like `"foo:bar & baz.jpg".sanitized #=> "foo-bar & baz.jpg"`. Dead sexy. – Joshua Pinter Jan 10 '22 at 19:34
  • This should be the accepted and most popular answer. – Kalsan Jan 16 '23 at 12:44
17

If you use Rails you can also use String#parameterize. This is not particularly intended for that, but you will obtain a satisfying result.

"my§document$is°°   very&interesting___thisIs%nice445.doc.pdf".parameterize
albandiguer
  • 2,327
  • 1
  • 18
  • 14
  • 1
    This isn't technically accurate because it will also remove the decimal character, which is somewhat essential in preserving extensions. Fortunately, the code behind parameterize is [relatively simple](http://apidock.com/rails/ActiveSupport/Inflector/parameterize) and can be implemented with just a few `gsub` calls. – Rob Yurkowski Jun 09 '14 at 18:08
3

For Rails I found myself wanting to keep any file extensions but using parameterize for the remainder of the characters:

filename = "my§doc$is°° very&itng___thsIs%nie445.doc.pdf"
cleaned = filename.split(".").map(&:parameterize).join(".")

Implementation details and ideas see source: https://github.com/rails/rails/blob/master/activesupport/lib/active_support/inflector/transliterate.rb

def parameterize(string, separator: "-", preserve_case: false)
  # Turn unwanted chars into the separator.
  parameterized_string.gsub!(/[^a-z0-9\-_]+/i, separator)
  #... some more stuff
end
Blair Anderson
  • 19,463
  • 8
  • 77
  • 114
2

If your goal is just to generate a filename that is "safe" to use on all operating systems (and not to remove any and all non-ASCII characters), then I would recommend the zaru gem. It doesn't do everything the original question specifies, but the filename produced should be safe to use (and still keep any filename-safe unicode characters untouched):

Zaru.sanitize! "  what\ēver//wëird:user:înput:"
# => "whatēverwëirduserînput"
Zaru.sanitize! "my§docu*ment$is°°   very&interes:ting___thisIs%nice445.doc.pdf" 
# => "my§document$is°° very&interesting___thisIs%nice445.doc.pdf"
David
  • 1,359
  • 1
  • 13
  • 20
0

There is a library that may be helpful, especially if you're interested in replacing weird Unicode characters with ASCII: unidecode.

irb(main):001:0> require 'unidecoder'
=> true
irb(main):004:0> "Grzegżółka".to_ascii
=> "Grzegzolka"
Jan Warchoł
  • 1,063
  • 1
  • 9
  • 22