There are several ways of storing persistent data in Android.
The way you are currently doing it is through the device storage, and you are quite right it would probably not scale well in addition to being directly accessible to the user meaning they could edit or delete your metadata.
Using SharedPreferences would be one way of storing the metadata which has the advantage of being completely hidden from the user, as well as being relatively easy to set up. The main disadvantages I can see are that it may not scale well if a user has a large number of files, and it is much more difficult to retrieve files with certain criteria, a certain tag for instance, as you mention in the comments.
The best way to store data that will scale well, be persistent, and let you run queries on the data would be an on device SQLite database. SQLite will usually have more overhead in terms of setup time, but is far more robust and featured than any of the other options besides perhaps network based storage, which based on the information you have given is probably not something you are interested in. Based on your problem the SQLite database is probably the way to go and has the bonus of being expandable in case you ever decide to add more information, or even store the files in the SQLite database.