2

What is the maximum number of data sets that can be created in SAS library? Is there a difference in limit (if any) for permanent and temporary libraries like work library?

I can find e.g. information that up to SAS 9.1 the maximum number of variables per data set was 32,767 and that from SAS 9.1 it is only limited by system resources, but I cannot find any info about the number of data sets per library

kristof
  • 52,923
  • 24
  • 87
  • 110
  • Base SAS? Or in a metadata server environment? I suspect in the former case there is no formal limit, though performance will degrade with thousands of datasets. – Joe Oct 16 '15 at 16:01
  • 2
    Well, you can only have 32 characters in a names so that would set a formal limit on the order of 10^50. – Jeff Oct 16 '15 at 16:26
  • 1
    We have >10k in one library. It is one of our most-used libraries and has been running fine without any issues for us. SAS 9.3TS1M2, Windows 7x64. It's on the list of things to clean up as I don't particularly like having that many files in a single folder (although technically libnames can be spread across folders). =/ – Robert Penridge Oct 16 '15 at 17:03
  • 1
    However much space your disk drive has. – Stu Sztukowski Oct 16 '15 at 17:23
  • @Joe, you are right I was referring to Base SAS. You also mention metadata server, are there some formal limitations there? – kristof Oct 19 '15 at 09:00

1 Answers1

1

I suspect that any such limit is more likely to stem from the underlying file system than from SAS itself, as described in this answer:

How many files can I put in a directory?

On a somewhat pedantic note, it is perhaps worth pointing out that a sas dataset can consist of multiple files - so as well as a primary .sas7bdat file you might have numerous connected files, e.g. for indexes, audit trails, and generation data sets (up to 1000!). If you have lot of these then you will obviously hit the file system ceiling somewhat sooner. Depending on the file system, you might find that you get some sort of performance degradation when listing all the files in the folder. However, for anything more recent than FAT32 I doubt you'd ever hit any practical limit based on the number of files (as opposed to their total size).

Also, a standard cautionary point: it isn't clear what you're doing with SAS that would need so many files; quite often in such cases it is much neater and more efficient to put everything in one big dataset and take advantage of by-group processing.

Community
  • 1
  • 1
user667489
  • 9,501
  • 2
  • 24
  • 35
  • thank you, we inherited a code that can potentially create thousands of small data sets in work library and I wanted to check if it might run into any sas limitations. – kristof Oct 19 '15 at 08:55