Yes, there is away to figure out each stream content. there is a signature for each file on this planet in addition to extension which is not reliable. it might be removed or falsely added.
So what is the signature?
In computing, a file signature is data used to identify or verify the
contents of a file. In particular, it may refer to:
File magic number: bytes within a file used to identify the
format of the file; generally a short sequence of bytes (most are
2-4 bytes long) placed at the beginning of the file; see list of file
signatures
File checksum or more generally the result of a hash function over the file contents: data used to verify the integrity of the file
contents, generally against transmission errors or malicious attacks.
The signature can be included at the end of the file or in a separate
file.
I used the magic number to define the magic number term I'm copying this from Wikipedia
In computer programming, the term magic number has multiple
meanings. It could refer to one or more of the following:
- Unique values with unexplained meaning or multiple occurrences which could (preferably) be replaced with named constants
- A constant numerical or text value used to identify a file format or protocol; for files, see List of file
signatures
- Distinctive unique values that are unlikely to be mistaken for other meanings(e.g., Globally Unique Identifiers)
in the second point it is a certain sequence of bytes like
PNG (89 50 4E 47 0D 0A 1A 0A)
or
BMP (42 4D)
So how to know the magic number of each file?
in this article "Investigating File Signatures Using PowerShell" we find the writer created a wonderful power shell function to get the magic number also he mentioned a tool and I'm copying this from his article
PowerShell V5 brings in Format-Hex, which can provide an alternative
approach to reading the file and displaying the hex and ASCII value to
determine the magic number.
form Format-Hex help I'm copying this description
The Format-Hex cmdlet displays a file or other input as hexadecimal
values. To determine the offset of a character from the output, add
the number at the leftmost of the row to the number at the top of the
column for that character.
This cmdlet can help you determine the file type of a corrupted file
or a file which may not have a file name extension. Run this cmdlet,
and then inspect the results for file information.
this tool is very good also to get the magic number of a file. Here is an example
another tool is online hex editor but to be onset I didn't understand how to use it.
now we got the magic number but how to know what type of data or is that file or stream?
and that is the most good question.
Luckily there are many database for these magic numbers. let me list some
- File Signatures
- FILE SIGNATURES TABLE
- List of file signatures
for example the first database has a search capability. just enter the magic number with no spaces and search
after you may find. Yes, may. There is a big possibility that you won't directly find the file type in question.
I faced this and solved it by testing the streams against specific types of signatures. Like PNG I was searching for in a stream
def GetPngStartingOffset(arr):
#targted magic Number for png (89 50 4E 47 0D 0A 1A 0A)
markerFound = False
startingOffset = 0
previousValue = 0
arraylength = range(0, len(arr) -1)
for i in arraylength:
currentValue = arr[i]
if (currentValue == 137): # 0x89
markerFound = True
startingOffset = i
previousValue = currentValue
continue
if currentValue == 80: # 0x50
if (markerFound and (previousValue == 137)):
previousValue = currentValue
continue
markerFound = False
elif currentValue == 78: # 0x4E
if (markerFound and (previousValue == 80)):
previousValue = currentValue
continue
markerFound = False
elif currentValue == 71: # 0x47
if (markerFound and (previousValue == 78)):
previousValue = currentValue
continue
markerFound = False
elif currentValue == 13: # 0x0D
if (markerFound and (previousValue == 71)):
previousValue = currentValue
continue
markerFound = False
elif currentValue == 10: # 0x0A
if (markerFound and (previousValue == 26)):
return startingOffset
if (markerFound and (previousValue == 13)):
previousValue = currentValue
continue
markerFound = False
elif currentValue == 26: # 0x1A
if (markerFound and (previousValue == 10)):
previousValue = currentValue
continue
markerFound = False
return 0
Once this function found the magic number
I split the stream and save the png file
arr = stream.read()
a = list(arr)
B = a[GetPngStartingOffset(a):len(a)]
bytesString = bytes(B)
image = Image.open(io.BytesIO(bytesString))
image.show()
At the end this is not an end to end solution but it is a way to figure out streams content
Thanks for reading and Thanks for @Robert Columbia for his patience
