Understanding per cpu structure in linux kernel

Question

I am studying linux kernel and sometimes I don't understand what kernel developers want in a particular piece of code. So I was reading through timers in kernel and a timer is created using a struct timer_list variable, that contains a per cpu pointer. I tried to understand a little better this per cpu variable so I was looking in linux kerenl, how things are getting created. So I have taken different structures from kernel and listed the #defines to integrate things and see a clear picture, what actually is happening.

structures from all this started

struct timer_list {
    /*
     * All fields that change during normal runtime grouped to the
     * same cacheline
     */
    struct list_head entry;
    unsigned long expires;
    struct tvec_base *base;//pointer to per cpu variable, so I checked what is inside

    void (*function)(unsigned long);
    unsigned long data;

    int slack;

#ifdef CONFIG_TIMER_STATS
    int start_pid;
    void *start_site;
    char start_comm[16];
#endif
#ifdef CONFIG_LOCKDEP
    struct lockdep_map lockdep_map;
#endif
};

the base pointer is a struct like this

struct tvec_base {
    spinlock_t lock;
    struct timer_list *running_timer;
    unsigned long timer_jiffies;
    unsigned long next_timer;
    unsigned long active_timers;
    struct tvec_root tv1;
    struct tvec tv2;
    struct tvec tv3;
    struct tvec tv4;
    struct tvec tv5;
} ____cacheline_aligned;//??why such a name __cacheline_aligned

struct tvec_base boot_tvec_bases;
EXPORT_SYMBOL(boot_tvec_bases);
static DEFINE_PER_CPU(struct tvec_base *, tvec_bases) = &boot_tvec_bases;//from here I am a little puzzeled as the way things are written been assigned.

DEFINE PER CPU is such a simple #define to understand, wish it was

    #define DEFINE_PER_CPU(type, name)                  \
        DEFINE_PER_CPU_SECTION(type, name, "")

#define DEFINE_PER_CPU_SECTION(type, name, sec)             \//?? what exactly we achieve here
    __PCPU_DUMMY_ATTRS char __pcpu_scope_##name;            \
    extern __PCPU_DUMMY_ATTRS char __pcpu_unique_##name;        \
    __PCPU_DUMMY_ATTRS char __pcpu_unique_##name;           \
    __PCPU_ATTRS(sec) PER_CPU_DEF_ATTRIBUTES __weak         \
    __typeof__(type) name

#define __PCPU_DUMMY_ATTRS                      \
    __attribute__((section(".discard"), unused))//i think it is a section in map file, but already kernel is built and I am builing a timer module, so what does it do?

If anyone have good experience in linux internal can you just point me in the right direction.

Some specific questions which if answered can make me understand whole thing,

`static DEFINE_PER_CPU(struct tvec_base *, tvec_bases) = &boot_tvec_bases;

1)what does this mean? this address '&boot_tvec_bases' is going where?

2) why a name ____cacheline_aligned; choosen. Does it do anything special?

3)what is

#define DEFINE_PER_CPU_SECTION(type, name, sec)

sec here?

This is not a question, please restate as a question. Feel free to ask as several sub questions, but the important thing about a question is that you can put a question mark at the end. Right now it's too general. — awiebe, Mar 07 '17 at 18:52
`____cacheline_aligned` as comes from the name is a special treatment of alignment to the structure. Depending on CPU type it might be 32 bytes, 64 bytes, or else. — 0andriy, Mar 07 '17 at 21:13
Initially all the pointers to the structures in the specific section has same value, i.e. address of `boot_tvec_base`. — 0andriy, Mar 07 '17 at 21:20

score 0 · Answer 1 · edited May 23 '17 at 10:29

timer.h:36: extern struct tvec_base boot_tvec_bases;
See ____cacheline_aligned_in_smp for structure in linux

____cacheline_aligned instructs the compiler to instantiate a struct or variable at an address corresponding to the beginning of an L1 cache line, for the specific architecture, i.e., so that it is L1 cache-line aligned. ____cacheline_aligned_in_smp is similar, but is actually L1 cache-line aligned only when the kernel is compiled in SMP configuration (i.e., with option CONFIG_SMP). These are defined in file include/linux/cache.h

3.http://www.makelinux.net/ldd3/chp-8-sect-5

Per-CPU variables are an interesting 2.6 kernel feature. When you create a per-CPU variable, each processor on the system gets its own copy of that variable. This may seem like a strange thing to want to do, but it has its advantages. Access to per-CPU variables requires (almost) no locking, because each processor works with its own copy.

Understanding per cpu structure in linux kernel

1 Answers1