Why is ROSBag record only recording ten events per second from any sensor?

Question

The example ROSBags provided as part of Cartographer's tutorials (https://google-cartographer-ros.readthedocs.io/en/latest/demos.html) have a message frequency of about 1507 messages per second for LiDAR messages, and 250 messages per second from an IMU. I have an IMU broadcasting about 100 messages per second, and a Velodyne VLP-16 broadcasting messages using Velodyne's own driver (VLP16_points.launch). For both sensors, rosbag record only seems to record up to 10 messages per second. How can I increase this recording frequency?

Answer 1

I suspect that rather being a function of a time limiter (ex, a frequency passed to a timer callback that does recording, like you might be expecting), it's actually a buffer bottleneck.

It's only taking in so much data to write to disk, or data in the subscriber buffers, before it's dropping frames to write or process the data already there.

Aside from writing your own node using their C++ api and profiling the issue itself (also a valid & not too difficult option, if very necessary), there are some commandline built-in parameters that control the buffer sizes (below, emphasis added).

I also recommend calculating what your desired recording rate is in kB/s; if you know the size of one lidar msg, by how many you want to store, and the rate they're generated (rostopic hz/rostopic bw), to give you a good estimation for what you're actually asking.

-b SIZE, --buffsize=SIZE

Use an internal buffer of SIZE MB (Default: 256, 0 = infinite). This is the message queue of the recorder object, before messages are being passed on to the bag. Lowering this value might result in messages being dropped before they reach the recording process.

$ rosbag record -b 1024 /chatter

--chunksize=SIZE

Advanced. Record to chunks of SIZE KB (Default: 768). This is a buffer within the bag file object. Lowering this value will result in more writes to disk.

$ rosbag record --chunksize=1024 /chatter

http://wiki.ros.org/rosbag/Commandline#record

---

Edits from Comments:

The two options above affect the buffers of serialized messages, when the raw topic msg is converted to text and stored. If they're being exceeded, there should be ROS_WARN msgs in your terminal. (If not, you can enable them.)

If you do not see any ROS_WARN msgs about dropped frames, then the issue may be normal dropped ros msgs. Refering to previous questions, you can enable topic statistics to see if any msgs are being dropped because of queue size. From the source code, for the commandline tool, each subscribed topic only has a msg buffer of 100 (ops.queue_size = 100;). If this is the problem, the easiest solution is to copy the source package from github to your workspace, and modify the queue size. But I suspect that's not the real issue.

If you're recording more than 10 topics, there hypothetically could be another issue, as by default it "only" launches 10 'async spinner' threads; but all the other mentioned potential issues are more likely to be the root causes.