Stack Overflow
Stack Overflow

Questions tagged [gfs]

GFS can mean either Google File System, or "GFS2" which is a cluster file system available in Linux.

GFS can mean either Google File System, or "GFS2" which is a cluster file system available in Linux.

48 questions
149
votes
5 answers

What is an SSTable?

In BigTable/GFS and Cassandra terminology, what is the definition of a SSTable?
knorv
  • 49,059
  • 74
  • 210
  • 294
21
votes
2 answers

Is it possible to append to HDFS file from multiple clients in parallel?

Basically whole question is in the title. I'm wondering if it's possible to append to file located on HDFS from multiple computers simultaneously? Something like storing stream of events constantly produced by multiple processes. Order is not…
maximdim
  • 8,041
  • 3
  • 33
  • 48
8
votes
2 answers

Google File System Consistency Model

I was reading about GFS and its consistency model but I'm failing to grasp some of it. In particular, can someone provide me with a specific example scenario (or an explanation of why it cannot happen) of: concurrent record append that could…
Simone
  • 2,261
  • 2
  • 19
  • 27
7
votes
1 answer

Apache Hadoop vs Google Bigdata

Can any one explain me the key difference between Apache Hadoop vs Google Bigdata Which one is better(hadoop or google big data).
MST
  • 113
  • 1
  • 8
5
votes
0 answers

Shared storage FS (GFS2, GlusterFS, ?) comparison and test

Problem description: For our application (RHEL 5,6) we use shared storage (EVA) and need to find OCFS2 replacement (not supported on RHEL 6) for several FS shared between nodes (2-7). Current tips are GFS2 and GlusterFS. Usage: System receives…
Jan Kohout
  • 93
  • 1
  • 7
4
votes
0 answers

Node.JS Convert Base64 String into Binary and write to MongoDB GridFS

I have a Base64 string that I am converting to binary like this: var b64string = req.body.image.substr(23);//The base64 has a imageURL var buf = new Buffer(b64string, 'base64'); I need to insert this into MongoDB GridFS. The problem I am having…
Rob
  • 11,185
  • 10
  • 36
  • 54
3
votes
2 answers

How do I get access to the noaa rest api of the gfs data

I am interested in getting the GFS forecast data for Europe which is available for free from the following website: https://www.ncdc.noaa.gov/data-access/model-data/model-datasets/global-forcast-system-gfs According this site NOAA provides rest APIs…
McDizzy
  • 189
  • 11
3
votes
2 answers

White area on matplotlib plot with pygrib data between 359.5 and 360 degrees

What I try is to plot output of the gfs weather model with matplotlib using pygrib to save the data, which is saved in grib files. Nearly everything works fine, the output looks like this: It appears that the program isn't closing the gap between…
Dremet
  • 1,008
  • 1
  • 9
  • 23
3
votes
2 answers

Google File System block size

Why is block size in GFS 64Mb though it may lead to internal fragmentation when the file size is not in the multiple of 64?
naman wats
  • 31
  • 3
3
votes
0 answers

How to check whether a file is open anywhere in a cluster using GFS and lvm?

I wonder if it is possible to check if a file has already been opened by another node in the same GFS cluster. For example, the fuser command runs cluster-wide in TruCluster. Is it possible to query the lock manager's data via a command or API?
anonymous platypus
2
votes
1 answer

What are some examples of “Appending Operations” as to Random Write Operations?

I have just finished reading the Google File System (GFS) paper. The paper says that GFS is optimized for appending operations rather than random writes. Seeing that this characteristic is emphasized throughout the paper, I take it that it must…
Some Noob Student
  • 14,186
  • 13
  • 65
  • 103
2
votes
1 answer

Persistent Logging in Gnome-Shell-Extension development?

I'm trying around with the Javascript-based bindings to build an own Gnome-Shell-Extension which just embeds an webkitview. But the following 3 lines let completely crash the gnome-shell (top panel disappears, need to re-login to start it again).…
kraiz
  • 2,178
  • 1
  • 22
  • 22
2
votes
0 answers

How to properly parse NOAA GFS Grib2 files using GPS Coordinates?

I'm currently trying to download GFS files from NOAA to get irradiance forecasts. I've done this same process for NOAA's HRRR files, but for some reason, parsing out GFS files isn't giving me the proper result. For a given point on the West coast of…
arkipelago
  • 21
  • 1
2
votes
1 answer

how to use pygrib expand_reduce functionality?

I am using pygrib to open GFS data, I want the data to be in unstructured format (not the default option). in short, How can I set expand_reduce to False? grbs = pygrib.open(filename) print(grbs[1].expand_reduce) # this prints True (default) In the…
Lorenzo Donadio
  • 66
  • 6
2
votes
2 answers

Why doesn't Hadoop file system support random I/O?

The distributed file systems which like Google File System and Hadoop doesn't support random I/O. (It can't modify the file which were written before. Only writing and appending is possible.) Why did they design file system like this? What are the…
Benjamin
  • 10,085
  • 19
  • 80
  • 130
1
2 3 4