In my node application, when I tried to use fake GCS bucket image from https://github.com/fsouza/fake-gcs-server. I have to use fake gcs bucket because we dont have access to our project GCS bucket publicly even with key. I faced couple of issues.
docker-compose.yml
version: '3.9'
networks:
storage-network:
driver: bridge
db-network:
driver: bridge
frontend-network:
driver: bridge
services:
api:
container_name: node_api
build:
context: ../node-app/
dockerfile: Dockerfile.dev
ports:
- 8080:8080
depends_on:
- postgres
restart: on-failure
volumes:
- ../node-app/:/app/
networks:
- storage-network
environment:
# Port
PORT: 8080
storage:
container_name: fake-gcs-server
image: fsouza/fake-gcs-server
restart: always
ports:
- 8083:8083
volumes:
- ./gcs_bucket:/data
command: ["-scheme", "http", "-port", "8083", "-external-url", "http://[::]:8083", "-backend", "memory"]
networks:
- storage-network
NodeJs function to upload file in this GCS bucket.
exports.testupload = async (req, res) => {
const bucketName = 'testBucket';
try {
await processFile(req, res);
const storage = new Storage({
apiEndpoint: "http://<ip_address_like_192.168..>:8083",
projectId: "test",
});
const bucket = storage.bucket(bucketName);
const blob = bucket.file(req.file.originalname);
const blobStream = blob.createWriteStream({
resumable: false,
});
blobStream.on("error", (err) => {
res.status(500).send({ message: err.message });
});
blobStream.on("finish", async (data) => {
res.status(200).send({
message: "Uploaded the file successfully: " + req.file.originalname,
url: publicUrl,
});
});
blobStream.end(req.file.buffer);
} catch (err) {
console.log(err);
if (err.code == "LIMIT_FILE_SIZE") {
return res.status(500).send({
message: "File size cannot be larger than 2MB!",
});
}
res.status(500).send({
message: `Could not upload the file: ${req.file.originalname}. ${err}`,
});
}
}
When I run the code and check. I always get http code 200 i.e. success. But when I look at mapped folder gcs_bucket I see there is no file in it.
Log in console from docker image of fake_gcs_server:
fake-gcs-server | time="2022-03-12T11:38:42Z" level=info msg="192.168.80.1 - - [12/Mar/2022:11:38:42 +0000] \"POST /upload/storage/v1/b/testbucket/o?uploadType=multipart&name=test_1647085120709.txt HTTP/1.1\" 200 476"
In another similar example I am able to read file from this fake GCS bucket. But I am facing below two problems.
- Uploading of document into mapped gcs_bucket/testBucket directory not working.
- In apiEndpoint, instead of localhost, I need to mention my ipv4 ipaddress starting 192.168.
Please let me know where I am doing mistakes.
