Errors running headless chromium in AWS Lambda

Question

I have been stuck trying to get this to work for literal weeks, so any help would be much appreciated.

I'm trying to run selenium tests written in Java in AWS Lambda. I turned on verbose logging in webdriver and here is the output (stack traces omitted):

[1615217980.576][INFO]: Launching chrome: /tmp/chrome_headless12660738644101056992/headless-

chromium --data-path=/tmp --disable-background-networking --disable-client-side-phishing-detection --disable-default-apps --disable-dev-profile --disable-dev-shm-usage --disable-extensions --disable-gpu --disable-hang-monitor --disable-popup-blocking --disable-prompt-on-repost --disable-sync --disable-web-security --disk-cache-dir=/tmp --enable-automation --enable-blink-features=ShadowDOMV0 --enable-logging --headless --homedir=/tmp --ignore-certificate-errors --log-level=0 --no-first-run --no-sandbox --no-service-autorun --password-store=basic --remote-debugging-port=0 --single-process --test-type=webdriver --use-mock-keychain --user-data-dir=/tmp --window-size=1366,768
[0308/153940.757451:WARNING:resource_bundle.cc(431)] locale_file_path.empty() for locale 
prctl(PR_SET_NO_NEW_PRIVS) failed
prctl(PR_SET_NO_NEW_PRIVS) failed
[0308/153942.834268:WARNING:discardable_shared_memory_manager.cc(194)] Less than 64MB of free space in temporary directory for shared memory files: 0
Fontconfig error: Cannot load default config file: No such file: (null)
DevTools listening on ws://127.0.0.1:41157/devtools/browser/563821ec-8b24-4a54-b834-d4d2ed23d8a2
[0308/153942.914879:ERROR:devtools_http_handler.cc(291)] Error writing DevTools active port to file
[0308/153942.932365:ERROR:egl_util.cc(70)] Failed to load GLES library: /tmp/chrome_headless12660738644101056992/swiftshader/libGLESv2.so: /tmp/chrome_headless12660738644101056992/swiftshader/libGLESv2.so: cannot open shared object file: No such file or directory
[0308/153942.933136:ERROR:simple_backend_impl.cc(81)] Failed to create directory: /tmp/Default/Code Cache/js
[0308/153942.933684:ERROR:simple_backend_impl.cc(81)] Failed to create directory: /tmp/Default/Code Cache/js
[0308/153942.933708:ERROR:simple_backend_impl.cc(757)] Simple Cache Backend: wrong file structure on disk: 1 path: /tmp/Default/Code Cache/js
[0308/153942.934435:ERROR:simple_backend_impl.cc(81)] Failed to create directory: /tmp/Default/Code Cache/wasm
[0308/153942.934494:ERROR:simple_backend_impl.cc(81)] Failed to create directory: /tmp/Default/Code Cache/wasm
[0308/153942.934503:ERROR:simple_backend_impl.cc(757)] Simple Cache Backend: wrong file structure on disk: 1 path: /tmp/Default/Code Cache/wasm
[0308/153942.955102:ERROR:disk_cache.cc(184)] Unable to create cache
[0308/153942.955146:ERROR:shader_disk_cache.cc(606)] Shader Cache Creation failed: -2
[0308/153942.972843:ERROR:gpu_channel_manager.cc(749)] ContextResult::kFatalFailure: Failed to create shared context for virtualization.
[0308/153942.972880:ERROR:shared_image_stub.cc(471)] SharedImageStub: unable to create context
[0308/153942.972890:ERROR:gpu_channel.cc(449)] GpuChannel: Failed to create SharedImageStub
[0308/153942.952932:FATAL:platform_font_skia.cc(97)] Check failed: InitDefaultFont(). Could not find the default font
Calling _exit(1). Core file will not be generated.
[1615217983.174][INFO]: [11c2ef49b3c2bcd0ed0121c5a7d97ab7] RESPONSE InitSession ERROR unknown error: Chrome failed to start: exited abnormally.
(unknown error: Devtools port number file contents <> were in an unexpected format)
(The process started from chrome location /tmp/chrome_headless12660738644101056992/headless-chromium is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
[1615217983.175][DEBUG]: Log type 'driver' lost 0 entries on destruction
[1615217983.175][DEBUG]: Log type 'browser' lost 0 entries on destruction
Failures: 2
Success: -1
org.openqa.selenium.WebDriverException: unknown error: Chrome failed to start: exited abnormally.

So there's a lot of errors about not being able to create directories (I've searched for these ones as well, but couldn't find out why it's not able to write)... but anyway the "fatal" one is complaining about the default font. I've read that Amazon Linux has no fonts installed so to get chrome to work on it, you need to install one. So I followed the procedure here: Include custom fonts in AWS Lambda. But here we still see the error:

Fontconfig error: Cannot load default config file: No such file: (null)

So this makes it sound like the path is null. But I'm setting both in the env vars:

And here is the folder structure:

And fonts.conf:

<?xml version="1.0"?>
<!DOCTYPE fontconfig SYSTEM "fonts.dtd">
<fontconfig>
   <dir>/var/task/lambda_fonts/</dir>
   <cachedir>/tmp/fonts-cache/</cachedir>
   <config></config>
</fontconfig>

So why is Lambda not finding the path?

Also here are the config options I'm passing to chromedriver, in case there's anything that jumps out at you:

        chromeOptions.addArguments("--no-sandbox");
        chromeOptions.addArguments("--headless");           
        chromeOptions.addArguments("--window-size=1366,768");
        chromeOptions.addArguments("--single-process");
        
        chromeOptions.addArguments("--user-data-dir=/tmp");
        chromeOptions.addArguments("--data-path=/tmp");
        chromeOptions.addArguments("--homedir=/tmp");
        chromeOptions.addArguments("--disk-cache-dir=/tmp");
        
        chromeOptions.addArguments("--disable-gpu");
        chromeOptions.addArguments("--disable-extensions");
        chromeOptions.addArguments("--disable-dev-shm-usage");
        chromeOptions.addArguments("--disable-dev-profile");
        chromeOptions.addArguments("--disable-web-security");