How do i pick file names with specified pattern in scala

Question

OTC_omega_20210302.csv
CH_delta_20210302.csv
MD_omega_20210310.csv
CD_delta_20210310.csv

val hdfsPath = "/development/staging/abcd-efgh"
val fs = org.apache.hadoop.fs.FileSystem.get(spark.sparkContext.hadoopConfiguration)

val files = fs.listStatus(new Path(s"${hdfsPath}")).filterNot(_.isDirectory).map(_.getPath)
val regX = "OTC_*[0-9].csv|CH_*[0-9].csv".stripMargin.r
val filteredFiles = files.filter(fName => regX.findFirstMatchIn(fName.getName).isDefined)

What is regex do i need to give if i need any file name that starts with either (OTC_ or CH_ ) and ends with YYYYMMDD.csv ?

As per the above files i need two outputs OTC_omega_20210302.csv CH_delta_20210302.csv

Please help

@wiktor, Yes, This works, but what is the use of ?: here , also why \\. in the end , it works like this also ^(OTC|CH)_.*[0-9]{8}.csv — Surender Raja, Mar 15 '21 at 17:35
`.` matches any char other than a line break char. You must escape `.` to make it match a dot. — Wiktor Stribiżew, Mar 15 '21 at 17:40
The `(?:...)` is a [non-capturing group](https://stackoverflow.com/questions/3512471/what-is-a-non-capturing-group-in-regular-expressions), it is used for alternatives or to quantify sequences of patterns. Since it does not cause memory allocation for submatches, it is better in this case than a capturing group. — Wiktor Stribiżew, Mar 15 '21 at 17:50

score 0 · Accepted Answer · answered Mar 15 '21 at 17:41

You can use

val regX = "^(?:OTC|CH)_.*[0-9]{8}\\.csv$".r
val regX = """^(?:OTC|CH)_.*[0-9]{8}\.csv$""".r

See the regex demo.

Details:

^ - start of string
(?:OTC|CH) - a non-capturing group matching either OTC or CH char sequences
_ - a _ char
.* - any zero or more chars other than line break chars, as many as possible
[0-9]{8} - eight digits
\. - a literal dot (note . matches any char other than a line break char, you must escape . to make it match a dot)
csv - a csv string
$ - end of string.

How do i pick file names with specified pattern in scala

1 Answers1