9

I am trying to implement a soap client for a server that uses NTLM authentication. The libraries that I use (requests-ntlm2 which relies on ntlm-auth) implement the MD4 algorithm that lies in the core of the NTLM protocol via the standard library's hashlib.

Although hashlib seems to support MD4:

>>> import hashlib
>>> hashlib.algorithms_available
{'md5-sha1', 'md4', 'shake_128', 'md5', 'blake2s', 'sha3_512', 'ripemd160', 'sha512', 'mdc2', 'blake2b', 'sha3_256', 'sha3_224', 'sha512_224', 'sha1', 'sha384', 'sha256', 'sha224', 'whirlpool', 'sha512_256', 'sha3_384', 'shake_256', 'sm3'}
>>>

and so does the openssl library in my system:

(victory) C:\code\python\services>openssl
help:
[...]
Message Digest commands (see the `dgst' command for more details)
blake2b512        blake2s256        md4               md5
mdc2              rmd160            sha1              sha224
sha256            sha3-224          sha3-256          sha3-384
sha3-512          sha384            sha512            sha512-224
sha512-256        shake128          shake256          sm3
[...]

when the authentication tries to run python produces an ValueError: unsupported hash type md4 error. Here is the relevant part of the traceback:

C:\ProgramData\Miniconda3\envs\victory\lib\site-packages\ntlm_auth\compute_hash.py in _ntowfv1(password)
    165         return nt_hash
    166 
--> 167     digest = hashlib.new('md4', password.encode('utf-16-le')).digest()
    168 
    169     return digest

C:\ProgramData\Miniconda3\envs\victory\lib\hashlib.py in __hash_new(name, data, **kwargs)
    161         # This allows for SHA224/256 and SHA384/512 support even though
    162         # the OpenSSL library prior to 0.9.8 doesn't provide them.
--> 163         return __get_builtin_constructor(name)(data)
    164 
    165 

C:\ProgramData\Miniconda3\envs\victory\lib\hashlib.py in __get_builtin_constructor(name)
    118         return constructor
    119 
--> 120     raise ValueError('unsupported hash type ' + name)
    121 
    122 

ValueError: unsupported hash type md4

Even when I try to merely call the MD4 from hashlib, I get the same result:

>>> import hashlib
>>> hashlib.new('md4')
Traceback (most recent call last):
  File "C:\ProgramData\Miniconda3\envs\victory\lib\hashlib.py", line 157, in __hash_new
    return _hashlib.new(name, data)
ValueError: [digital envelope routines] initialization error

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\ProgramData\Miniconda3\envs\victory\lib\hashlib.py", line 163, in __hash_new
    return __get_builtin_constructor(name)(data)
  File "C:\ProgramData\Miniconda3\envs\victory\lib\hashlib.py", line 120, in __get_builtin_constructor
    raise ValueError('unsupported hash type ' + name)
ValueError: unsupported hash type md4

Any insights about what's going on and/or any help would be immensely appreciated.

spitoglou
  • 131
  • 1
  • 1
  • 6
  • Following. This works fine for me. – Frank Yellin Nov 12 '21 at 06:28
  • @FrankYellin are you using the same stack (windows, python 3.8, requests-ntlm2)? `hashlib.new('md4')` works allright? – spitoglou Nov 12 '21 at 07:59
  • I have no specialized knowledge to add, but I'm on Windows, with a fresh Python 3.9, no OpenSSL available via PATH, and `hashlib.new('md4')` works for me. – Cory Petosky Nov 14 '21 at 02:59
  • If nothing else, there are pure Python implementations of md4. https://gist.github.com/kangtastic/c3349fc4f9d659ee362b12d7d8c639b6 – Frank Yellin Nov 14 '21 at 03:01
  • 1
    What version of openssl are you using? I had a similar issue, getting the same error message for code that had previously worked. After a lot of searching online for a solution, I finally figured out the culprit was the version of openssl being used. I rebuild my environment and explicitly specified openssl=1.1.1 - this solved the problem for me. – martino Dec 13 '21 at 08:25
  • I have this problem with openssl 3.x on Windows. If I revert back to 1.1.1 it works – Mark Thomas Mar 24 '22 at 20:21
  • @MarkThomas is it possible to create a virtual environment with customized packages? If I try to create a virtual environment with python 3.7, in packages I get openssl 3.x by default. – Ashish Dec 08 '22 at 20:55

5 Answers5

7

For ubuntu (jammy/focal).

Add this to your /etc/ssl/openssl.cnf to 're-enable' md4 to hashlib

[provider_sect]
default = default_sect
legacy = legacy_sect

[default_sect]
activate = 1

[legacy_sect]
activate = 1

Solution from this https://bugs.launchpad.net/ubuntu/+source/python3.10/+bug/1971580/comments/3

Depending on your version or distribuition the path to the openssl.cnf file can be /usr/lib/openssl.cnf or other.

imbr
  • 6,226
  • 4
  • 53
  • 65
4

Update as of May 2023

This has been addressed in requests-ntlm>=1.2.0 which now uses SPNEGO: https://github.com/requests/requests-ntlm/pull/126

OpenSSL configuration changes should no longer be required with this version.

Tip: Since this is one of the top results for "unsupported hash type md4" (an error I encountered when using Ansible to target Windows on a modern Linux distro) simply updating this library in your current Ansible install appears to be a solution:

python3 -m pip install -U requests-ntlm
Greg R
  • 206
  • 1
  • 2
2

Well, it seems that there was something corrupted in my conda environment. I created a new identical one, and it's been working ever since without having to change anything else.

spitoglou
  • 131
  • 1
  • 1
  • 6
  • That's really scary. I don't like the fact that a corrupted Conda environment can cause hashlib to generate the spurious results. – Frank Yellin Mar 26 '22 at 17:04
  • @spitoglou Did you by any chance find out what the reason for the corruption was? I just experienced the same, and environments breaking without warning is pretty scary indeed. – Saaru Lindestøkke May 13 '22 at 12:04
  • @SaaruLindestøkke Sorry for the delayed answer. Unfortunately, I have not been able to pinpoint the exact problem... – spitoglou Jul 02 '22 at 10:13
0

In my case, I was running a docker image python:3.7 and didn't specify the sub-version. The fix mentioned by @imbr did work. I found the openssl.cnf file location by running openssl version -d

But also, on my dev station, it was using python:3.7.15 and on the live server was python:3.7.17 From the release notes, you can see that openssl got an update on 17, https://www.python.org/downloads/release/python-3717/

Matt
  • 115
  • 12
-1
hashlib.new('md4', "test".encode()).hexdigest()
  • 4
    Remember that Stack Overflow isn't just intended to solve the immediate problem, but also to help future readers find solutions to similar problems, which requires understanding the underlying code. This is especially important for members of our community who are beginners, and not familiar with the syntax. Given that, **can you [edit] your answer to include an explanation of what you're doing** and why you believe it is the best approach? – Jeremy Caney Jul 18 '22 at 01:04