I am trying to perform matrix multiplication in a PHP script by using FFI to invoke a function cblas_dgemm, defined in a CBLAS library. I want to know why this works on my Ubuntu 20.04 workstation running php 8.2 but does not work on another Ubuntu 20.04 virtual machine running php 7.4 or yet another virtual machine running Ubuntu 22.04 with PHP 8.1.
First, I built CBLAS according to these instructions. This yields a shared library whose full path is:
/home/sneakyimp/cblas/CBLAS/lib/cblas_LINUX.so
I adapted the blas.h included in the ghostjat/np repo to point to that cblas_LINUX.so path and I removed some functions not defined in CBLAS to get this file, which I saved as cblas.h:
#define FFI_SCOPE "blas"
#define FFI_LIB "/home/sneakyimp/cblas/CBLAS/lib/cblas_LINUX.so"
typedef size_t CBLAS_INDEX_t;
size_t cblas_idamax(const int N, const float *X, const int incX);
double cblas_dsdot(const int N, const float *X, const int incX, const float *Y,
const int incY);
double cblas_ddot(const int N, const double *X, const int incX,
const double *Y, const int incY);
double cblas_dnrm2(const int N, const double *X, const int incX);
double cblas_dasum(const int N, const double *X, const int incX);
void cblas_dswap(const int N, double *X, const int incX,
double *Y, const int incY);
void cblas_dcopy(const int N, const double *X, const int incX,
double *Y, const int incY);
void cblas_daxpy(const int N, const double alpha, const double *X,
const int incX, double *Y, const int incY);
void cblas_drotg(double *a, double *b, double *c, double *s);
void cblas_drotmg(double *d1, double *d2, double *b1, const double b2, double *P);
void cblas_drot(const int N, double *X, const int incX,
double *Y, const int incY, const double c, const double s);
void cblas_drotm(const int N, double *X, const int incX,
double *Y, const int incY, const double *P);
void cblas_dscal(const int N, const double alpha, double *X, const int incX);
void cblas_dgemv(const enum CBLAS_ORDER order,
const enum CBLAS_TRANSPOSE TransA, const int M, const int N,
const double alpha, const double *A, const int lda,
const double *X, const int incX, const double beta,
double *Y, const int incY);
void cblas_dgbmv(const enum CBLAS_ORDER order,
const enum CBLAS_TRANSPOSE TransA, const int M, const int N,
const int KL, const int KU, const double alpha,
const double *A, const int lda, const double *X,
const int incX, const double beta, double *Y, const int incY);
void cblas_dtrmv(const enum CBLAS_ORDER order, const enum CBLAS_UPLO Uplo,
const enum CBLAS_TRANSPOSE TransA, const enum CBLAS_DIAG Diag,
const int N, const double *A, const int lda,
double *X, const int incX);
void cblas_dtbmv(const enum CBLAS_ORDER order, const enum CBLAS_UPLO Uplo,
const enum CBLAS_TRANSPOSE TransA, const enum CBLAS_DIAG Diag,
const int N, const int K, const double *A, const int lda,
double *X, const int incX);
void cblas_dtpmv(const enum CBLAS_ORDER order, const enum CBLAS_UPLO Uplo,
const enum CBLAS_TRANSPOSE TransA, const enum CBLAS_DIAG Diag,
const int N, const double *Ap, double *X, const int incX);
void cblas_dtrsv(const enum CBLAS_ORDER order, const enum CBLAS_UPLO Uplo,
const enum CBLAS_TRANSPOSE TransA, const enum CBLAS_DIAG Diag,
const int N, const double *A, const int lda, double *X,
const int incX);
void cblas_dtbsv(const enum CBLAS_ORDER order, const enum CBLAS_UPLO Uplo,
const enum CBLAS_TRANSPOSE TransA, const enum CBLAS_DIAG Diag,
const int N, const int K, const double *A, const int lda,
double *X, const int incX);
void cblas_dtpsv(const enum CBLAS_ORDER order, const enum CBLAS_UPLO Uplo,
const enum CBLAS_TRANSPOSE TransA, const enum CBLAS_DIAG Diag,
const int N, const double *Ap, double *X, const int incX);
void cblas_dsymv(const enum CBLAS_ORDER order, const enum CBLAS_UPLO Uplo,
const int N, const double alpha, const double *A,
const int lda, const double *X, const int incX,
const double beta, double *Y, const int incY);
void cblas_dsbmv(const enum CBLAS_ORDER order, const enum CBLAS_UPLO Uplo,
const int N, const int K, const double alpha, const double *A,
const int lda, const double *X, const int incX,
const double beta, double *Y, const int incY);
void cblas_dspmv(const enum CBLAS_ORDER order, const enum CBLAS_UPLO Uplo,
const int N, const double alpha, const double *Ap,
const double *X, const int incX,
const double beta, double *Y, const int incY);
void cblas_dger(const enum CBLAS_ORDER order, const int M, const int N,
const double alpha, const double *X, const int incX,
const double *Y, const int incY, double *A, const int lda);
void cblas_dsyr(const enum CBLAS_ORDER order, const enum CBLAS_UPLO Uplo,
const int N, const double alpha, const double *X,
const int incX, double *A, const int lda);
void cblas_dspr(const enum CBLAS_ORDER order, const enum CBLAS_UPLO Uplo,
const int N, const double alpha, const double *X,
const int incX, double *Ap);
void cblas_dsyr2(const enum CBLAS_ORDER order, const enum CBLAS_UPLO Uplo,
const int N, const double alpha, const double *X,
const int incX, const double *Y, const int incY, double *A,
const int lda);
void cblas_dspr2(const enum CBLAS_ORDER order, const enum CBLAS_UPLO Uplo,
const int N, const double alpha, const double *X,
const int incX, const double *Y, const int incY, double *A);
void cblas_dgemm(const enum CBLAS_ORDER Order, const enum CBLAS_TRANSPOSE TransA,
const enum CBLAS_TRANSPOSE TransB, const int M, const int N,
const int K, const double alpha, const double *A,
const int lda, const double *B, const int ldb,
const double beta, double *C, const int ldc);
void cblas_dsymm(const enum CBLAS_ORDER Order, const enum CBLAS_SIDE Side,
const enum CBLAS_UPLO Uplo, const int M, const int N,
const double alpha, const double *A, const int lda,
const double *B, const int ldb, const double beta,
double *C, const int ldc);
void cblas_dsyrk(const enum CBLAS_ORDER Order, const enum CBLAS_UPLO Uplo,
const enum CBLAS_TRANSPOSE Trans, const int N, const int K,
const double alpha, const double *A, const int lda,
const double beta, double *C, const int ldc);
void cblas_dsyr2k(const enum CBLAS_ORDER Order, const enum CBLAS_UPLO Uplo,
const enum CBLAS_TRANSPOSE Trans, const int N, const int K,
const double alpha, const double *A, const int lda,
const double *B, const int ldb, const double beta,
double *C, const int ldc);
void cblas_dtrmm(const enum CBLAS_ORDER Order, const enum CBLAS_SIDE Side,
const enum CBLAS_UPLO Uplo, const enum CBLAS_TRANSPOSE TransA,
const enum CBLAS_DIAG Diag, const int M, const int N,
const double alpha, const double *A, const int lda,
double *B, const int ldb);
void cblas_dtrsm(const enum CBLAS_ORDER Order, const enum CBLAS_SIDE Side,
const enum CBLAS_UPLO Uplo, const enum CBLAS_TRANSPOSE TransA,
const enum CBLAS_DIAG Diag, const int M, const int N,
const double alpha, const double *A, const int lda,
double *B, const int ldb);
I am now trying to run this PHP script, which loads that cblas.h, which refers to the just-compiled cblas_LINUX.so lib. This works on my workstation, but does not work on a couple of other virtual machines, even if I recompile CBLAS on those other vms.
<?php
$ffi_blas = FFI::load(__DIR__ . '/cblas.h');
$m1 = [
[1, 2, 3],
[4, 5, 6]
];
$m1t = array_map(null, ...$m1);
// lifted from ghostjat/np, converts matrix to obj with FFI\CData
function get_cdata($m) {
// this check should be more specific, but works for now
if (!is_array($m) || !is_array($m[0])) {
throw new Exception('param must be array of arrays');
}
$rowcount = sizeof($m);
$colcount = sizeof($m[0]);
$flat_m = [];
$size = $rowcount * $colcount;
$cdata = \FFI::cast('double *', \FFI::new("double[$size]"));
$i = 0;
foreach($m as $row) {
foreach($row as $val) {
$flat_m[$i] = $val;
$cdata[$i] = $val;
$i++;
}
}
return [
'rows' => $rowcount,
'cols' => $colcount,
'flat_m' => $flat_m,
'cdata' => $cdata
];
}
$m1_c = get_cdata($m1);
$m1t_c = get_cdata($m1t);
define('CBLAS_ROW_MAJOR', 101);
define('CBLAS_NO_TRANS', 111);
$mr_size = $m1_c['rows'] * $m1t_c['cols'];
$mr_rows = $m1_c['rows'];
$mr_cols = $m1t_c['cols'];
$mr_cdata = \FFI::cast('double *', \FFI::new("double[$mr_size]"));
$start = microtime(true);
// this fn works by pointer, returns some void object, I think?
$some_val = $ffi_blas->cblas_dgemm(CBLAS_ROW_MAJOR, CBLAS_NO_TRANS, CBLAS_NO_TRANS, $m1_c['rows'], $m1t_c['cols'], $m1_c['cols'], 1.0, $m1_c['cdata'], $m1_c['cols'], $m1t_c['cdata'], $m1t_c['cols'], 0.0, $mr_cdata, $mr_cols);
echo "cblas_dgemm elapsed time: " . (microtime(true) - $start) . "seconds\n";
On my workstation, it performs the correct calculation very quickly, even on very large matrices. On the machines where it doesn't work, they complain about an 'undefined symbol' like so:
php: symbol lookup error: /home/sneakyimp/cblas/CBLAS/lib/cblas_LINUX.so: undefined symbol: dgemm_
Can anyone tell me what 'undefined symbol' means? It refers to PHP but doesn't look like the usual PHP error. I suspect cblas is looking for some function/variable/constant/object, dgemm_, which it cannot find. I note that a grep search shows various binary files that refer to this, but its only appearance in the source code is in the file CBLAS/include/cblas_f77.h:
#define F77_dgemm dgemm_
Also, how do I fix this on the vms where the script won't run? I suspect my workstation might have some package installed (some variant of lapack?) which isn't on the vms.
EDIT: I listed all the lapack packages on my workstation:
ii liblapack3:amd64 3.9.0-1build1 amd64 Library of linear algebra routines 3 - shared version
ii liblapacke:amd64 3.9.0-1build1 amd64 Library of linear algebra routines 3 - C lib shared version
ii liblapacke-dev:amd64 3.9.0-1build1 amd64 Library of linear algebra routines 3 () - Headers
ii libtmglib-dev:amd64 3.9.0-1build1 amd64 test matrix generators of LAPACK - development files
ii libtmglib3:amd64 3.9.0-1build1 amd64 test matrix generators of LAPACK - shared version
and tried installing those on the uncooperative vms
# this installs libblas3 libgfortran5 liblapack3 liblapacke libquadmath0 libtmglib3
sudo apt install liblapacke
That did not solve the problem. I also tried installing gfortran using apt. This also did not help.
