4

Sorry for the bad title, I really don't know how to describe that in short...

My scenario is that for a robotic application, we need a realtime loop to control the motor every 1ms. While at the same time we may want to do something that does not have realtime requirement such as path planning, image processing, object recognition, etc. Furthermore, some of the result from the non-realtime task would be sent to the realtime motor control loop to control the robot.

For the realtime part, I use Ubuntu with RT-Preempt Patch. Therefore, I can run the realtime control loop in the while loop just like the sample code here.

However, I have no idea for the non-realtime part. In my humble opinion, I would new a thread in the same process and run the non-realtime task in that thread.

Since I'm very new to realtime programming, I don't know what would be the problem for my design. Besides, I wonder if there are any paradigm for designing such kind of program?

---EDIT---

Some more details about my application.

The robot, to be more specific, is a robot arm.

For the realtime part, it calculate the forward kinematics, inverse kinematics, and jacobian. And then calculate the proper output command using a simple PID controller. At last, send the motor command using EtherCAT to each motor.

For the non-realtime part, for example, may take the PointCloud stream from kinect, do some preprocessing, calculate the pose of the object in the scene, determine a good grasp pose for the robot arm, and at last send the goal of each motor to the realtime part so that the robot arm will actually move to the goal and grasp the object. The whole process may take 10 sec or so. However at the same time the realtime loop should keep running and send the proper force command or position command to make the robot arm hold its original pose and stand still.

As for communication between these two parts, for most of the occasion, the command is generated by a novel algorithm from the non-realtime part and send it to the realtime part to make the robot arm move. However, sometimes the algorithm in the non-realtime part will need to know, for example, the current pose of the end effector. Therefore, the none-realtime part will need to get the information from the forward kinematics, which resides in the realtime part.

AlbertK
  • 231
  • 5
  • 13
  • Interesting question, but you should give much more details (what kind of robot).... I feel it might be more appropriate on http://programmers.stackexchange.com/ – Basile Starynkevitch Aug 07 '14 at 16:28
  • Ok, a robot arm. But again, constraints and consequences are different if it is for brain surgery and if it is for playing football. What are the consequence of losing a few milliseconds? – Basile Starynkevitch Aug 08 '14 at 07:52
  • For the non-realtime part, it's no big deal. even a few seconds(let's say within 5 seconds) is acceptable. The application is mainly about the pick and place, and the object classification. Of course we hope that the robot arm can response to the object and finish its task as fast as possible, but it's future improvement though, at least for now. One thing I can say for sure is that the miss of few milliseconds or even a few seconds won't cause any casualties or disastrous consequences. Well, my advisor may complain about the speed of the response, if you count that in :) – AlbertK Aug 09 '14 at 03:03
  • 1
    I'd recommend taking a look at the Orocos project. It's a framework for realtime robotics applications, and makes communication between realtime and non-realtime tasks relatively easy. – crosstalk Aug 09 '14 at 17:37
  • Okay...I've took a look at Orocos. That's really a giant monster! I don't mean anything negative. It looks good and the Real-Time Toolkit seems to be a good choice. And it also provided a kinematics and dynamics library, which is the function I need in the realtime part. However, it is just too big to find a good starting point. If you have any experience about that or any good documentations or tutorials, it would be greatly appreciated if you can share that with us. – AlbertK Aug 10 '14 at 09:22
  • 1
    I've worked on the ATRIAS project (at http://code.google.com/p/atrias/), which uses Orocos inside ROS. Feel free to look at the code (inside the software/ directory). I'm not sure how difficult it is to run Orocos outside of ROS. As for learning Orocos, unfortunately it is very complex and the tutorials aren't particularly helpful. In Orocos, you build "components". Each component has at least one thread, and components are somewhat isolated from each other. They communicate through "operations" (function calls that can cross thread boundaries) and "ports" (which are kind of like sockets). – crosstalk Aug 10 '14 at 16:08
  • Would you suggest using Orocos inside ROS, as compared with using standalone Orocos? The documentation I found about realtime control is [***ros_control***](http://wiki.ros.org/ros_control). Am I on the right direction? If so, well, there seems few tutorials or documentation about how to set it up. – AlbertK Aug 11 '14 at 05:17
  • The better documentation about Orocos that I could find is [**this one**](http://www.orocos.org/stable/documentation/rtt/v2.x/doc-xml/orocos-components-manual.html). This looks more like a book though. – AlbertK Aug 11 '14 at 05:23
  • The latter document is the only documentation I'm aware of on how to build a component. As for ros_control, this is the first I've heard of it. Thanks for linking it, that may become useful for myself in the future. – crosstalk Aug 11 '14 at 15:18
  • Then how do you learn to use the Orocos inside ROS? Is there any other good documents or tutorials that we can follow? Thanks. – AlbertK Aug 14 '14 at 08:34

2 Answers2

2

What is the real-time loop doing exactly? I am guessing (since you mentioned "every 1 millisecond do some motor control") do some very short computations, and output a few bytes to the motor device.

What is the non-realtime part doing? I imagine some user-interface???

What kind of robot?

Path planning may require strong real time, in particular if the robot is moving fast. A cruise missile or a Google Car is not the same as a small robot for the robocup. Losing a few milliseconds of communication may kill humans on a cruise missile or a Google Car, but is acceptable on the Robocup - you'll loose only a game and perhaps harm slightly your robot.

At 40 m/s (144 km/hour, which is slighty above the 130km/h highway road speed limit in France), two milliseconds mean 8 cm, and if that 8 cm is human flesh, it may mean killing someone.

How the two parts are interacting? Is the real-time part sending some information to the other part?

Perhaps the two parts could be some different processes (not threads) with some communication? Perhaps using shared memory with semaphores for synchronization. Then look at sem_overview(7) and shm_overview(7).

Notice that at the interface between the real-time and "non-real time" part the non-real time part might actually becomes "real-time" also.

The main point is to define the conceptual interface between the two (real-time vs non real-time) parts, and decide if you can afford losing some data and some synchronization (communicating between the two parts) or not.

If the role of the non-real time part is simply to display or set the speed (i.e. some low and high threshold) you probably might afford sometimes losing some data or synchronization. But the evil is in the details (if the robot is a real autonomous automobile driving on some highway à la Google Car, you probably should not afford losing some exchanges, and then both parts become real-time!).

Basile Starynkevitch
  • 223,805
  • 18
  • 296
  • 547
  • Why should I use two different process instead of tow different thread within a single process? As far as this scenario is concerned, what's the major difference, or what's the advantage over another? – AlbertK Aug 08 '14 at 02:40
  • Uh...sorry, should I open a new thread in programmers.stackexchange.com, instead of the discussion here? – AlbertK Aug 08 '14 at 02:43
  • OK, thanks for the advice. I just delete the copious comments and move them to the original post as an edit. – AlbertK Aug 09 '14 at 02:51
1

How you divide your overall system into the real time vs regular parts is very important, as stressed by @Basile Starynkevitch.

When it comes to the implementation in RT-Linux, the parts of your system with real time requirements will be made into real time threads, which run inside of the kernel. This allows them to run unpreempted, but has one giant drawback: there are no protections that one has come to expect in user-space. A real time thread crash means the system has crashed. For this reason (and others) you should limit the real time components as much as possible.

The non-real time components of your system will run as regular Linux processes. They can chug along with whatever expensive planning algorithms you want and the real time threads will preempt them and run as needed. The tricky part is communication, for which there are two mechanisms provided: FIFOs and shared memory.

FIFOs are the easiest, and allow unidirectional communication (use two if you want bidirectional). They are character devices and you don't have to worry about reading / writing overlaps. More from tldp.org.

For passing large amounts of data, shared memory is preferable. With it, two processes map the same section into memory. But you have to coordinate between the processes / threads to ensure that one does not read in middle of another one writing and vice versa. This is actually slightly easier to do when one of the parties is a real-time thread, because you know that the real-time thread will not be preempted. More from drdobbs.com.

You also need to be aware of priority inversion (when a high priority thread must wait for a low priority one to release a shared resource, for example).

superdesk
  • 1,162
  • 11
  • 24
  • I guess the RT-Linux you mention here is a little bit different from the RT-Preempt for my case. According to the documentation from the RT-Preempt wiki page, the patch seems to equipped the realtime process in the user space some kind of preemption capability that can preempt, for example, system call, critical section, etc. As you can see from the example code [here](https://rt.wiki.kernel.org/index.php/RT_PREEMPT_HOWTO#A_Realtime_.22Hello_World.22_Example), it is simply an ordinary C program, instead of a kernel module. – AlbertK Aug 09 '14 at 14:02
  • Apart from the different realtime approach, the FIFO looks good. Is that still suitable for the IPC between two user space process in my case? As for the shared memory approach, what will happen when the non-realtime process is reading the shared data, and in the half way being preempted by the realtime process. And then the realtime process modified the shared data. The similar problem for writing from the non-realtime part and reading from the realtime part. Or it depends on how much data I'm writing and reading?(Sorry I'm not really familiar with the data sync problem) – AlbertK Aug 09 '14 at 14:15
  • The more I look into this... the simpler it all seems. It looks like RT-Linux that I was referring to is either defunct / supplanted by the preempt patch. For FIFOs you can just use Linux pipes I think. – superdesk Aug 09 '14 at 15:33
  • Let's get a step back further. Since I already had some experiences about thread programming, what is the major drawback to program it as two different threads rather than two different processes? Or what is the advantage so that we had better separate them as two processes? – AlbertK Aug 10 '14 at 09:29
  • By the way, if it is indeed better to do that in two processes, I think it's a good starting point to dig into Linux process and IPC, and I'm glad to learn it. – AlbertK Aug 10 '14 at 09:29
  • So, you are going to have at least one process regardless. The question is if that process will create multiple threads or fork/exec a new process. The big difference is that threads share global state and address spaces. You need to check if this includes priority. – superdesk Aug 10 '14 at 21:17
  • My plan is to create two thread in that process. One for realtime and one for non-realtime. For the realtime thread, I would use ***sched_setscheduler()*** routine to set it as hard realtime thread. For the non-realtime thread, I guess a normal thread creation is enough? – AlbertK Aug 11 '14 at 05:06
  • Try it out. You need to make sure that you can actually have one thread as real time and one as a normal thread. – superdesk Aug 11 '14 at 13:36
  • I write a simple code [here](http://codepad.org/iM9Rd3Jm). And then use the command **ps -eLfc** to view the scheduling policy of each thread. The CLS column for the realtime thread is marked **FF**, which stand for SCHED_FIFO. The SCHED_FIFO is one of the scheduling policy for realtime thread, and that conforms to my setting in the code. As for the non-realtime thread, it is marked **TS**, which stand for SCHED_OTHER, stand for non realtime scheduling. – AlbertK Aug 14 '14 at 08:30
  • Is the verification enough that way? – AlbertK Aug 14 '14 at 08:31