1

As we know, in NTFS we store our metadata in structures called MFT record (which contains some runs).

The book asks the next question

Can it be that a short file uses more MFT records than a longer file?

I think that the answer is no - because if we have a file, whose metadata is more than one MFT's size, then we have to use runs. Any other suggestions?

phuclv
  • 37,963
  • 15
  • 156
  • 475
Adam Sh
  • 8,137
  • 22
  • 60
  • 75

3 Answers3

5

The answer is yes.

MFT record (up to NT4 it had 4KB, now it has 1KB) contains file attributes (such as file name, security info, small file data, etc.); some of these attributes can be resident (= attribute header AND attribute data are both located in MFT record), or non-resident (only attrbite header is in MFT, attribute data are stored elsewhere on the disk - eg., data attribute is usually non-resident). Some attributes must remain resident - e.g., file name attribute; some attributes can be resident, or nonresidet - e.g., data attribute.

If you have large file, it means that MFT record contains only attribute header of data attribute, but data are located elsewhere - so even if file is large, it only uses one MFT record; on the other hand, if you have even very small file, but this file has more file names (links), NTFS must allocate more MFT records to accomodate all file name attributes and it can span across many MFT records (using special attribute called attribute list).

Robert Goldwein
  • 5,805
  • 6
  • 33
  • 38
  • Thank you, but the data is not located also in MFT? – Adam Sh Jul 24 '12 at 13:34
  • 1
    Only up to ~300 bytes, file data are in MFT (as a resident $DATA attribute) - if file data are larger, $DATA attribute becomes non-resident and data are moved outside MFT to 4KB (usually) clusters on disk. If you'd like to play with MFT records, check out this page: http://support.microsoft.com/kb/253066 and look specifically for utility called NFI (NTFS File Sector Information Utility) - e.g., it dumps contents of file record. – Robert Goldwein Jul 25 '12 at 20:35
  • @RobertGoldwein [*The figure MFT Entry with Resident Record shows the contents of an MFT record for a small file or folder. Small files and folders (typically, **900 bytes or smaller**) are entirely contained within the file’s MFT record*](https://technet.microsoft.com/en-us/library/cc781134(WS.10).aspx). For example on my system files ~640-960 bytes are still stored as residents – phuclv Jul 13 '18 at 02:55
  • @phuclv You are right, ~300 bytes is rather conservative, 600 - 900 bytes is more appropriate under standard conditions. By that I mean e.g. one hard link, moderate number of security descriptors, etc. – Robert Goldwein Jul 14 '18 at 23:24
2

A small file can use more MFT records if its attribute list is too large to fit in a single record.

When there is no more space for storing attributes in the file record segment, additional file record segments are allocated and inserted in the first (or base) file record segment in an attribute called the attribute list. The attribute list indicates where each attribute associated with the file can be found. This includes all attributes in the base file record, except for the attribute list itself. For more information, see ATTRIBUTE_LIST_ENTRY.

https://learn.microsoft.com/en-us/windows/desktop/devnotes/master-file-table

An example of this is when it has a large number of ADS. I created a file that consumes most of the MFT record space (1000/1024 bytes as you can see below)

PS D:\> fsutil file queryoptimizemetadata .\test.txt
File metadata optimization                  : None
Attribute list size                         : 0 (0)
File metadata space used                    : 1000 (0x3e8)
File metadata space allocated               : 1024 (0x400)
File metadata space usage                   : 97%
File records count(s)                       : 1
Number of resident attribute(s)             : 3
Number of nonresident attribute(s)          : 8
Total number of attributes                  : 11
Total active file metadata optimization(s)  : 0
Total pending file metadata optimization(s) : 0

After I added another stream you can see that the File metadata space allocated has been doubled to 2048 bytes, i.e. 2 records, and ATTRIBUTE_LIST has to be used which takes 448 bytes

PS D:\> cat .\test.txt | Set-Content -path .\test.txt -Stream stream8
PS D:\> fsutil file queryoptimizemetadata .\test.txt
File metadata optimization                  : None
Attribute list size                         : 448 (0x1c0)
File metadata space used                    : 1624 (0x658)
File metadata space allocated               : 2048 (0x800)
File metadata space usage                   : 79%
File records count(s)                       : 2
Number of resident attribute(s)             : 4
Number of nonresident attribute(s)          : 9
Total number of attributes                  : 13
Total active file metadata optimization(s)  : 0
Total pending file metadata optimization(s) : 0

This can also happen if the file is too fragmented (or too many holes in a sparse file) and the record doesn't have enough space to store the extent list. Other possibilities are too complex permissions, or multiple hard links like others mentioned

$ATTRIBUTE_LIST may be needed if the file:

  • has a large number of hard links (lots of file name attributes present).
  • becomes very fragmented, so the data runs overflow the MFT record.
  • has a complex security descriptor (not applicable to NTFS v3.0+)
  • has many named streams, e.g. data streams.

https://flatcap.org/linux-ntfs/ntfs/attributes/attribute_list.html

More information can be found in Can the NTFS $MFT file have child records?

phuclv
  • 37,963
  • 15
  • 156
  • 475
1

Generally, a newly created file (with no data yet) will be made resident. Meaning that any new data will be placed in its MFT record as long as there's enough space available. As data grows and no longer fit into available MFT record space, the data is made non resident by moving it out of MFT and into block of cluster(s) (and has a run list). One a file is non resident, it'll never be made back to resident if the data is later shrinked to one or zero byte.

A larger data isn't always made non resident if the file is of sparse type and no data has been written yet. You can have a sparse file of 1MB size and still resident as long as the written data can fit into its availale MFT record space.

A zero byte file can even use more than one MFT records if it has multiple hardlinks. In this case, its other MFT records store the remaining hardlink names.

Jay
  • 4,627
  • 1
  • 21
  • 30