5

I started to learn coding backwards: high level first. This has the obvious liability of missing some basic concepts that I should definitely know, and when I try to learn a low level language, it throws me.

I have tried many times to understand pointers, however the explanations rapidly go over my head, usually because all of the example code uses languages that use pointers, which I don't understand other things about, and then I spin.

I am the most (and very at that) fluent in Javascript.

How would you explain pointers to a sad Javascript developer like me? Could someone provide me a practical, real life example?

Maybe even showing how, if Javascript had pointers, you could do x, and a pointer is different than a raw variable because of y.

Community
  • 1
  • 1
dthree
  • 19,847
  • 14
  • 77
  • 106
  • They're simply addresses to memory locations, where the real data are stored – phuclv Jul 11 '14 at 00:54
  • 2
    The title of this question is really weird: it sounds like you understand pointers already but are having trouble explaining them to a javascript developer and want our help rewording things so you can explain them better. Instead of "how would you explain", I suggest just "Please explain", with similar changes throughout the question body. – amalloy Jul 11 '14 at 00:59
  • Got it. Made it a bit clearer. – dthree Jul 11 '14 at 01:00
  • 1
    There's so much stuff out there that explains how pointers work. What have you looked at already? Example: http://stackoverflow.com/questions/2144698/common-uses-for-pointers – Evan Trimboli Jul 11 '14 at 01:05
  • Pointers are like [call numbers](https://www.google.com.sg/search?q=librarian&sourceid=ie7&rls=com.microsoft:en-SG:IE-Address&ie=&oe=&gfe_rd=cr&ei=gzi_U5b9AYmL8Qfa3YGwCA&gws_rd=ssl#q=what%20is%20the%20library%20call%20number&rls=com.microsoft:en-SG:IE-Address) except much easier to understand and use. – Jake Jul 11 '14 at 01:22
  • To play devel's advocate: why *would* you care about pointers if you only write in a language that has no notion of pointers? It seems like you should learn about pointers when you have an actual reason to do so, and then you will see how they make sense when you see them in context. – Kerrek SB Jul 11 '14 at 01:39
  • @KerrekSB Spell devil right if you are going to be his advocate, please. I want (ul) to get into these other languages, but understanding pointers has been a major block for me. – dthree Jul 11 '14 at 01:45
  • @dc2: that seems circular reasoning: You don't understand pointers because you don't know those languages, and you can't know those languages because you don't understand pointers. Well, bite the bullet and dive in somewhere. Pick *one* language and just learn it. Eventually pointers will make sense. – Kerrek SB Jul 11 '14 at 01:51
  • 1
    Kerrek, that's exactly what I'm doing. It's called Go by the way. And I am mid learning it. And I got stuck on pointers. And I spent 1 hour looking around trying to understand it (on top of previous attempts in the past). And then I asked this question. And then you came along and were not helpful. And then fortunately someone else was helpful. And now I am learning pointers, at which "point" I will continue to learn Go, having grasped the fundamental concept of pointers. – dthree Jul 11 '14 at 02:38
  • 1
    To understand pointers, forget about Javascript entirely, and start over. Don't try to understand it in terms of Javascript. – Hot Licks Jul 11 '14 at 02:49
  • 1
    @HotLicks Tracking. I'll keep that in mind :) – dthree Jul 11 '14 at 02:58
  • 1
    @KerrekSB JS is super widespread and an entry point for many people. I learned graphics programming through webgl and JS. I wrote GLSL, experimented with quadtrees, octrees, DCELs, catmull-clark subdivision to name just a few things that people seldom do with JS, or is not something where a total beginner would start. All this time i never wrote anything using an actual explicit pointer. Given the nature of the language, many patterns are applied but bastardized. Many people pick up bad habits, or learn to think about something i na very specific to JS way. – pailhead Jun 29 '17 at 18:11
  • @KerrekSB With that being said, i think that there will be lots of talented, good developers who started with JS, and might struggle to wrap their heads around different concepts. They should not be denied the opportunity. The fact that the language is so widespread means that a lot of people can write **some** code. The concept of basics gets distorted here. If they understand **some** code, someone who truly understands CS should be able to explain any concept using that language. JS will pass by value and by reference, it has an `ArrayBuffer` etc. etc. – pailhead Jun 29 '17 at 18:15

2 Answers2

3

C++ rules are fairly simple and consistent. I actually find how Javascript handles object references and prototypes way more unintuitive.

Preface A: Why is Javascript A Bad Place To Start?

The first thing you need to fundamentally understand before you can tackle pointers is variables. You need to know what they are and how the computer keeps track of them.

Coming from a Javascript background you are used to every variable assigned to an object being a reference. That is, two variables can reference the same object. This is essentially pointers without any syntax to allow for more intricate use. You are also used to implicit copies of "basic" types like numbers. That is to say:

var a = MyObject;
var b = a;

Now if you change b you also change a. You would need to explicitly copy MyObject in order to have two variables pointing to different instances of it!

var a = 5;
var b = a;

Now if you change b, a is not actually changed. This is because assigning a to b when a is a simple type will copy it automatically for you. You cannot get the same behavior as objects with simple numbers and vise versa, so when you want two variables to refer to the same number you have to wrap it in an object. There is no explicit way to indicate how you want to handle references vs copies for primitive types.

You can see this inconsistent behavior with no variation on syntax (but an extreme variation on behavior) can make the relationship between variables and what they contain muddy. For this reason I highly suggest banishing this mental model for a moment as we continue on our journey to understand explicit pointers.

Preface B: YOLO: Variable Lifetime On The Stack

So, let's talk from here on out in C++ terms. C++ is one of the most explicit languages in terms of what a variable is vs a pointer. C++ is a good entry point because it is low level enough to talk in terms of memory and lifespan, but high level enough to understand things at a decent level of abstraction.

So, in C++ when you create any variable it exists in a certain scope. There are two ways to create a variable, on the stack, and on the heap.

The stack refers to the call stack of your application. Every brace pair pushes a new context onto the stack (and pops it when it runs out). When you create a local variable, it exists in that particular stack frame, when that stack frame is popped the variable is destroyed.

A simple example of scope:

#include <iostream>
#include <string>

struct ScopeTest{
    ScopeTest(std::string a_name):
        name(a_name){
        std::cout << "Create " << name << std::endl;
    }
    ~ScopeTest(){
        std::cout << "Destroy " << name << std::endl;
    }
    ScopeTest(ScopeTest &a_copied){
        std::cout << "Copy " << a_copied.name << std::endl;
        name = a_copied.name + "(copy)";
        a_copied.name += "(original)";
    }

    std::string name;
};

ScopeTest getVariable(){ //Stack frame push
    ScopeTest c("c"); //Create c
    return c; //Copy c + Destroy c(original)
}

int main(){
    ScopeTest a("a"); //Create a
    {
        ScopeTest b("b"); //Create b
        ScopeTest d = getVariable();
    } //Destroy c(copy) + Destroy b
} //Destroy a

Output:

Create a
Create b
Create c
Copy c
Destroy c(original)
Destroy c(copy)
Destroy b
Destroy a

This should illustrate explicitly how a variable ties its life to the stack, how it is copied around, and when it dies.

Preface C: YOLO Variable Lifetime on the Heap

So, that's interesting conceptually, but variables can also be allocated outside of the stack, this is called "heap" memory because it is largely structure-less. The issue with heap memory is that you don't really have automatic cleanup based on scope. So you need a way to tie it to some kind of "handle" to keep track of it.

I'll illustrate here:

{
    new ScopeTest("a"); //Create a
} //Whoa, we haven't destroyed it!  Now we are leaking memory!

So, clearly we can't just say "new X" without keeping track of it. The memory gets allocated, but doesn't tie itself to a lifespan so it lives forever (like a memory vampire!)

In Javascript you can just tie it to a variable and the object dies when the last reference to it dies. Later I'll talk about a more advanced topic in C++ which allows for that, but for now let's look at simple pointers.

In C++ when you allocate a variable with new, the best way to track it is to assign it to a pointer.

Preface D: Pointers and The Heap

As I suggested, we can track allocated memory on the heap with a pointer. Our previous leaky program can be fixed like so:

{
    ScopeTest *a = new ScopeTest("a"); //Create a
    delete a; //Destroy a
}

ScopeTest *a; creates a pointer, and assigning it to a new ScopeTest("a") gives us a handle we can actually use to clean up and refer to the variable which exists in heap memory. I know heap memory sounds kinda confusing, but it's basically a jumble of memory that you can point to and say "hey you, I want a variable with no lifespan, make one and let me point at it".

Any variable created with the new keyword must be followed by exactly 1 (and no more than 1) delete or it will live forever, using up memory. If you try to delete any memory address other than 0 (which is a no-op) more than one time, you could be deleting memory not under your program's control which results in undefined behavior.

ScopeTest *a; declares a pointer. From here on out, any time you say "a" you are referring to a specific memory address. *a will refer to the actual object at that memory address, and you can access properties of it (*a).name. a-> in C++ is a special operator that does the same thing as (*a).

{
    ScopeTest *a = new ScopeTest("a"); //Create a
    std::cout << a << ": " << (*a).name << ", " << a->name << std::endl;
    delete a; //Destroy a
}

Output for the above will look something like:

007FB430: a, a

Where 007FB430 is a hex representation of a memory address.

So in the purest sense, a pointer is literally a memory address and the ability to treat that address as a variable.

The Relationship Between Pointers and Variables

We don't just have to use pointers with heap allocated memory though! We can assign a pointer to any memory, even memory living on the stack. Just be careful your pointer doesn't out-live the memory it points to or you'll have a dangling pointer which could do bad things if you continue to try and use it.

It is always the programmer's job to make sure a pointer is valid, there are literally 0 checks in place in C++ to help you out when dealing with bare memory.

int a = 5; //variable named a has a value of 5.
int *pA = &a; //pointer named pA is now referencing the memory address of a (we reference "a" with & to get the address).

Now pA refers to the same value as &a, that is to say, it is the address of a.

*pA refers to the same value as a.

You can treat *pA = 6; the same as a = 6. Observe (continuing from the above two lines of code):

std::cout << *pA << ", " << a << std::endl; //output 5, 5
a = 6;
std::cout << *pA << ", " << a << std::endl; //output 6, 6
*pA = 7;
std::cout << *pA << ", " << a << std::endl; //output 7, 7

You can see why *pA is called a "pointer". It is literally pointing to the same address in memory as a. So far we have been using *pA to de-reference the pointer and access the value at the address it points to.

Pointers have a few interesting properties. One of those properties is that it can change the object it is pointing at.

int b = 20;
pA = &b;
std::cout << *pA << ", " << a << ", " << b << std::endl; //output 20, 7, 20
*pA = 25;
std::cout << *pA << ", " << a << ", " << b << std::endl; //output 25, 7, 25
pA = &a;
std::cout << *pA << ", " << a << ", " << b << std::endl; //output 7, 7, 25
*pA = 8;
std::cout << *pA << ", " << a << ", " << b << std::endl; //output 8, 8, 25
b = 30;
pA = &b;
std::cout << *pA << ", " << a << ", " << b << std::endl; //output 30, 8, 30

So you can see that a pointer is really just a handle to a point in memory. This can be exceptionally useful in many cases, do not write it off just because this sample is simplistic.


Now, the next thing you need to know about pointers is that you can increment them as long as the memory you are incrementing to belongs to your program. The most common example is C strings. In modern C++ strings are stored in a container called std::string, use that, but I will use an old C style string to demonstrate array access with a pointer.

Pay close attention to ++letter. What this does is increment the memory address the pointer is looking at by the size of the type it is pointing to.

Let's break this down a bit more, re-read the above sentence a few times then continue on.

If I have a type that is sizeof(T) == 4, every ++myPointerValue will shift 4 spaces in memory to point to the next "value" of that type. This is part of why the pointer "type" matters.

char text[] { 'H', 'e', 'l', 'l', 'o', '\0' }; //could be char text[] = "Hello"; but I want to show the \0 explicitly
char* letter = text;

for (char* letter = &text[0]; *letter != '\0';++letter){
    std::cout << "[" << *letter << "]";
}
std::cout << std::endl;

The above will loop over the string as long as there is no '\0' (null) character. Keep in mind this can be dangerous and is a common source of insecurity in programs. Assuming your array is terminated by some value, but then getting an array that overflows allowing you to read arbitrary memory. That's a high level description anyway.

For that reason it is much better to be explicit with string length and use safer methods such as std::string in regular use.


Alright, and as a final example to put things into context. Let's say I have several discreet "cells" that I want to link together into one coherent "list". The most natural implementation of this with non-contiguous memory is to use pointers to direct each node to the next one in the sequence.

With pointers you can create all sorts of complex data structures, trees, lists, and more!

struct Node {
    int value = 0;
    Node* previous = nullptr;
    Node* next = nullptr;
};
struct List {
    List(){
        head = new Node();
        tail = head;
    }
    ~List(){
        std::cout << "Destructor: " << std::endl;
        Node* current = head;
        while (current != nullptr){
            Node* next = current->next;
            std::cout << "Deleting: " << current->value << std::endl;
            delete current;
            current = next;
        }
    }
    void Append(int value){
        Node* previous = tail;
        tail = new Node();
        tail->value = value;
        tail->previous = previous;
        previous->next = tail;
    }

    void Print(){
        std::cout << "Printing the List:" << std::endl;
        Node* current = head;
        for (Node* current = head; current != nullptr;current = current->next){
            std::cout << current->value << std::endl;
        }
    }
    Node* tail;
    Node* head;
};

And putting it to use:

List sampleList;
sampleList.Append(5);
sampleList.Append(6);
sampleList.Append(7);
sampleList.Append(8);
sampleList.Print();

List may seem complicated at a glance, but I am not introducing any new concepts here. This is exactly the same things I covered above, just implemented with a purpose.

Homework for you to completely understand pointers would be to provide two methods in List:

  1. Node* NodeForIndex(int index)
  2. void InsertNodeAtIndex(int index, int value)

This list implementation is exceptionally poor. std::list is a much better example, but it most cases due to data locality you really want to stick with std::vector. Pointers are exceptionally powerful tools, and fundamental in computer science. You need to understand them to appreciate how the common data types you rely on every day are composed, and in time you will come to appreciate the explicit separation of value from pointer in C++.

Beyond simple pointers: std::shared_ptr

std::shared_ptr gives C++ the ability to deal with reference counted pointers. That is to say, it gives a similar behavior to Javascript object assignment (where an object is destroyed when the last reference to that object is set to null or destroyed).

std::shared_ptr is just like any other stack based variable. It ties its lifetime to the stack, and then holds a pointer to memory allocated on the heap. In this regard, it encapsulates the concept of a pointer in a safer manner than having to remember to delete.

Let's re-visit our earlier example that did leak memory:

{
    new ScopeTest("a"); //Create a
} //Whoa, we haven't destroyed it!  Now we are leaking memory!

With a shared_ptr we can do the following:

{
    std::shared_ptr<ScopeTest> a(new ScopeTest("a")); //Create a
}//Destroy a

And, a little more complex:

{
    std::shared_ptr<ScopeTest> showingSharedOwnership;
    {
        std::shared_ptr<ScopeTest> a(new ScopeTest("a")); //"Create a" (ref count 1)
        showingSharedOwnership = a; //increments a's ref count by 1. (now 2)
    } //the shared_ptr named a is destroyed, decrements ref count by 1. (now 1)
} //"Destroy a" showingSharedOwnership dies and decrements the ref count by 1. (now 0)

I won't go too much further here, but this should open your mind to pointers.

M2tM
  • 4,415
  • 1
  • 34
  • 43
  • 1
    Thank you for your generous answer! I'm mid digesting this now. – dthree Jul 11 '14 at 02:34
  • @dc2 No problem! Hope it helps, ask me if you have any questions. I have a full source listing which should compile for you here: http://pastebin.com/6hs4hrW9 – M2tM Jul 11 '14 at 02:36
  • @dc2 one last comment on the answer here. Pointers are very important to understand, but this is not how modern C++ is written. Nowadays you would lean more on the standard library, and instead of new and delete you would use scoped pointers like "shared_ptr" and "unique_ptr" where appropriate (on top of preferring stack based variables over heap allocation). When implementing low level datastructures it is still important, however. – M2tM Jul 11 '14 at 02:38
  • Thanks. Question 1: I guess dereference is called that because it temporarily eliminates the "referencing" aspect of the pointer and jumps straight to the actual memory location? – dthree Jul 11 '14 at 02:44
  • @dc2 Yes. A pointer references something by memory. The process of grabbing the value of the memory the pointer accesses is referred to as dereferencing. The * used in combination with a pointer (as opposed to multiplication) is actually called a dereference operator. The & is called the address or reference operator (though it can also be "bitwise and" when used on numbers). – M2tM Jul 11 '14 at 02:46
  • 1
    You're really trying to scare the guy away, aren't you?? – Hot Licks Jul 11 '14 at 02:50
  • @HotLicks No, just trying to be helpful. :C It's C++. – M2tM Jul 11 '14 at 02:54
  • @dc2 apologies, in my answer I forgot the "Node" struct. I had it in my pastebin, but just recently updated my answer to represent it there too. – M2tM Jul 11 '14 at 02:56
  • @HotLicks I'm actually picking it up pretty well. I'm not too pathetic - I've been developing for 8 years and work on full stack projects. Just somehow magically avoided anything too low level that whole time. :D – dthree Jul 11 '14 at 02:56
  • Okay @M2tM in your second section, if you didn't check for the 0\, would that throw a famous `null pointer exception`? Is that what that saying refers to? – dthree Jul 11 '14 at 02:57
  • Without that check it might loop forever, it might crash your program, it might blow up your home town. Dereferencing a pointer which points to unallocated memory is undefined behaviour. C++ has no safety checks in place, if you are running in debug mode you will might get "could not read 0xDEADBEEF" or some other debug hex code address. Anything could happen though. – M2tM Jul 11 '14 at 03:04
  • It could also say null pointer exception if you try dereferencing a pointer pointing at nullptr or 0 note: C++ does not auto initialize stack variables to 0. Only globals and static and a few other exceptions to the rule. – M2tM Jul 11 '14 at 03:07
  • @dc2 I have created multiple prefaces to frame things better for you from a javascript perspective and to offer more understanding. – M2tM Jul 11 '14 at 18:46
  • @dc2 I added a bunch to the beginning and a little to the end, you can skip the "The Relationship Between Pointers and Variables" section, but everything else is new. – M2tM Jul 11 '14 at 18:55
  • 1
    Thanks a ton for all of your hard work in helping me understand this. I'm mid chewing through it all!! – dthree Jul 12 '14 at 17:51
3

Here's an attempt at a self-contained answer from first principles.

Pointers are part of a type system that permit the implementation of reference semantics. Here's how. We suppose that our language has a type system, by which every variable is of a certain type. C is a good example, but many languages work like this. So we can have a bunch of variables:

int a = 10;
int b = 25;

Further, we assume that function arguments are always copied from the caller scope into the function scope. (This is also true for many real languages, though the details can quickly become subtle when the type system gets 'hidden' from the user (e.g. such as in Java)). So let's have a function:

void foo(int x, int y);

When calling foo(a, b), the variables a and b are copied into local variables x and y corresponding to the formal parameters, and those copies are visible within the function scope. Whatever the function does with x and y has no effect on the variables a and b at the call site. The entire function call is opaque to the caller.

Now let's move on to pointers. A language with pointers contains, for every object type T, a related type T *, which is the type "pointer to T". Values of type T * are produced by taking the address of an existing object of type T. So a language that has pointers also needs to have a way to produce pointers, which is "taking the address of something". The purpose of a pointer is to store the address of an object.

But that's only one half of the picture. The other half is what to do with the address of an object. The main reason for caring about the address of an object is to be able to refer to the object whose address is being stored. This object is obtained by a second operation, suitably called dereferencing, which when applied to a pointer produces the object which is being "pointed to". Importantly, we do not a copy of the object, but we get the actual object.

In C, the address-of operator is spelled &, and the dereference operator is spelled *.

int * p = &a;    // p stores the address of 'a'

*p = 12;         // now a == 12

The first operand of the final assignment, *p, is the object a itself. Both a and *p are the same object.

Now why is this useful? Because we can pass pointers to functions to allow functions to change things outside the function's own scope. Pointers allow for indirection, and thus for referencing. You can tell the function about "something else". Here's the standard example:

void swap(int * p, int * q)
{
    int tmp = *p;
    *p = *q;
    *q = tmp;
}

We can tell the function swap about our variables a and b by giving it the addresses of those variables:

swap(&a, &b);

In this way, we are using pointers to implement reference semantics for the function swap. The function gets to refer to variables elsewhere and can modify them.

The fundamental mechanism of reference semantics can thus be summarized thus:

  • The caller takes the address of the object to be refered to:

     T a;
     mangle_me(&a);
    
  • The callee takes a pointer parameter and dereferneces the pointer to access the refered value.

     void mangle_me(T * p)
     {
         // use *p
     }
    

Reference semantics are important for may aspects of programming, and many programming languages supply them in some way or another. For example, C++ adds native reference support to the language, largely removing the needs for pointers. Go uses explicit pointers, but offers some notational "convenience" by sometimes automagically dereferencing a pointer. Java and Python "hide" pointer-ness inside their type system, e.g. the type of a variable is in some sense a pointer to the type of the object. In some languages, some types like ints are naked value types, and others (like lists and dictionaries) are "hidden-pointer-included" reference types. Your milage may vary.

Kerrek SB
  • 464,522
  • 92
  • 875
  • 1,084
  • Thanks for your intuitive answer. I'll be studying this in a little bit. – dthree Jul 12 '14 at 17:52
  • This is a pretty good explanation of pointers in general. It doesnt feel like it does much for someone coming from JS in particular, but rather someone having general experience with a high level language. JS does have a very particular way it passes values. Primitives are always by value, while objects are always by reference. Illustrating the concept of memory as one giant array could be useful, perhaps using the JS typed arrays and buffer views. Like it or not,JS is really widespread, and is the entry point for a lot of people. Someone explaining serious CS through it would do a noble thing – pailhead Jun 29 '17 at 18:00