How to implement IAmsiStream to support running a malware scan on windows on a stream

Question

When implementing an IAmsiStream to perform a scan with Windows Defender, on files larger than ~20MBs it fails with Value does not fall within the expected range..

What is missing from this implementation?

    public class AmsiStream : IAmsiStream
    {
        private readonly Stream _input;
        private readonly string _name;
        private static readonly byte[] _nullPtr = new byte[Marshal.SizeOf(IntPtr.Zero)];

        public AmsiStream(Stream input, string name)
        {
            _input = input ?? throw new ArgumentNullException(nameof(input));
            _name = name ?? throw new ArgumentNullException(nameof(name));
        }

        public int GetAttribute(AMSI_ATTRIBUTE attribute, int dataSize, byte[] data, out int retData)
        {
            const int E_NOTIMPL = unchecked((int)0x80004001);
            const int E_NOT_SUFFICIENT_BUFFER = unchecked((int)0x8007007A);

            byte[] bytes = { };
            int retValue = 0;

            switch (attribute)
            {

                case AMSI_ATTRIBUTE.AMSI_ATTRIBUTE_APP_NAME:
                    bytes = Encoding.Unicode.GetBytes("TestAmsi" + "\0");
                    break;
                case AMSI_ATTRIBUTE.AMSI_ATTRIBUTE_CONTENT_NAME:
                    bytes = Encoding.Unicode.GetBytes(_name + "\0");
                    break;
                case AMSI_ATTRIBUTE.AMSI_ATTRIBUTE_CONTENT_SIZE:
                    bytes = BitConverter.GetBytes((ulong)_input.Length);
                    break;
                case AMSI_ATTRIBUTE.AMSI_ATTRIBUTE_SESSION:
                    bytes = _nullPtr;
                    break;
                case AMSI_ATTRIBUTE.AMSI_ATTRIBUTE_CONTENT_ADDRESS:
                    retValue = E_NOTIMPL;
                    break;
                default:
                    retValue = E_NOTIMPL;
                    break;
            }

            retData = 0;
            if (retValue == 0)
            {
                retData = bytes.Length;
                if (dataSize < bytes.Length)
                    return E_NOT_SUFFICIENT_BUFFER;

                Array.Copy(bytes, data, bytes.Length);
            }

            return retValue;

        }

        public int Read(long position, int size, byte[] buffer, out int readSize)
        {
            _input.Seek(position, SeekOrigin.Begin);
            readSize = _input.Read(buffer, 0, size);
            return 0;
        }
    }

A test case is :

        [Fact]
        public void TestWithLargeFile()
        {
            const long k = 1024;
            const long m = 1024 * k;
            const long fileSize = 21 * m;
            var c = new Random(42);
            var fileBuffer = new byte[fileSize];

            c.NextBytes(fileBuffer);
            Equal(AMSI_RESULT.AMSI_RESULT_CLEAN, ScanInternal(new MemoryStream(fileBuffer), "test.file.txt"));
        }

        private AMSI_RESULT ScanInternal(Stream streamToCheck, string fileName)
        {
            var scanner = (IAntiMalware)new CAntiMalware();
            var stream = new AmsiStream(streamToCheck, fileName);
            var result = scanner.Scan(stream, out AMSI_RESULT scanResult, out IAntiMalwareProvider _);

            if (result != (ulong)HResult.S_OK)
            {
                throw new InvalidOperationException($"Malware scan returned not OK: {result}");
            }

            return scanResult;
        }

Full source code for this test at this github repo

You should handle `AMSI_ATTRIBUTE_CONTENT_ADDRESS` when the content is fully loaded into memory, see this [docs](https://learn.microsoft.com/en-us/windows/win32/api/amsi/nf-amsi-iamsistream-getattribute#remarks). — weichch, Nov 10 '21 at 00:25
IAmsiStream implementations should not load the entire content into memory, as it might not fit. The goal is to stream data. — Bruno Lopes, Nov 10 '21 at 11:54

weichch · Answer 1 · 2021-11-11T06:26:20.177

I don't think you can stream a large file to Windows Defender provider. As per this github thread, there seems to be a limit implemented by the platform that if you don't handle AMSI_ATTRIBUTE_CONTENT_ADDRESS in GetAttribute, the maximum buffer size passed in the Read method is 16MB. And the purpose of the Read method is not streaming, instead it

Requests a buffer-full of content to be read.

The sample code (C++) provided by Microsoft can't handle files that are larger than this size limit either.

The Microsoft developer docs declared:

AMSI is designed in particular to combat "fileless malware"

which

exists exclusively as a computer memory-based artifact

as per Wikipedia.

To me, if the file size is larger than 16MB, you could load it into memory and return the buffer address by handling AMSI_ATTRIBUTE_CONTENT_ADDRESS. And for large files, you could consider using a memory mapped file:

public unsafe int GetAttribute(AMSI_ATTRIBUTE attribute, int dataSize,
    [Out]byte[] data, [Out]out int retData)
{
    ...
    case AMSI_ATTRIBUTE_CONTENT_ADDRESS:

        retData = sizeof(IntPtr);

        if (dataSize < retData)
        {
            return E_NOT_SUFFICIENT_BUFFER;
        }

        // This is an example
        // Better to open in constructor and close in Dispose
        var mappedFile = MemoryMappedFile.CreateFromFile(_name, FileMode.Open);
        var mappedFileAccessor = mappedFile.CreateViewAccessor(0, fileSize);

        byte* fileContent = null;
        mappedFileAccessor.SafeMemoryMappedViewHandle.AcquirePointer(ref fileContent);

        fixed (byte* pData = data)
        {
            *(IntPtr*) pData = (IntPtr)fileContent;
        }

        return 0;
    ...
}

Danut Radoaica · Answer 2 · 2021-11-13T11:33:17.100

Instead use MalwareScanner from: https://github.com/NewOrbit/MalwareScan.AMSI

var scanner = new MalwareScanner("MyApplications Viruscanner"); var result = scanner.HasVirus(stream, filename);

or for scanning an incoming large file bytes segment by bytes segment see:

WindowsCOMAntiMalware: https://github.com/bsmg/BeatSaber-IPA-Reloaded/blob/master/IPA.Loader/AntiMalware/_HideInNet3/WindowsCOMAntiMalware.cs

with 2 IAmsiStream implementations:

AmsiMemoryStream: https://github.com/bsmg/BeatSaber-IPA-Reloaded/blob/master/IPA.Loader/AntiMalware/_HideInNet3/ComAPI/AmsiMemoryStream.cs AmsiFileStream: https://github.com/bsmg/BeatSaber-IPA-Reloaded/blob/master/IPA.Loader/AntiMalware/_HideInNet3/ComAPI/AmsiFileStream.cs

P.S.: the 16mb limit is nowhere to be found in the official docs, I don't think is a IAmsiStream limitation, the IAmsiStream is passed to the IAntimalwareProvider which is a vendor implementation and maybe the problem lays there; you should try another vendor to see if you get the same error:

https://github.com/NewOrbit/MalwareScan.AMSI/blob/master/src/MalwareScan.AMSI/MalwareScanner.cs#L157 This copies the entire stream into a bytebuffer. For larger files this is a non-starter. — Bruno Lopes, Nov 08 '21 at 17:04
@BrunoLopes if I understand correctly, you want to scan an incoming large file bytes segment by bytes segment. I have updated the answer, please take a look — Danut Radoaica, Nov 08 '21 at 17:39

How to implement IAmsiStream to support running a malware scan on windows on a stream

2 Answers2