I need clarification on Metatable.__index

Question

I asked earlier why my methods for a metatable weren't being located by Lua, and was told that by setting __index to my metatable, that it would resolve the issue, so I assumed that a method when called was searching by index in the metatable, but I've ran into an issue now that I need to use indexing brackets [ and ] on my metatable, so __indexis assigned to return an index from a table inside of it, how do I resolve the functionality needs of both using methods, and use of indexing brackets

I wrote a minimal example indicating the problem:

TestMetatable = {DataTable = {}}
TestMetatable.__index = TestMetatable

function TestMetatable.new()
    local Tmp = {}
    setmetatable(Tmp,TestMetatable)

    Tmp.DataTable = {1}

    return Tmp
end

function TestMetatable:TestMethod()
    print("Ran Successfully")
end

function TestMetatable.__index(self,index)
    return self.DataTable[index]
end

local Test = TestMetatable.new()

-- both functionalities are needed
print(Test[1])
Test:TestMethod()

Nicol Bolas · Accepted Answer · 2016-11-30T22:46:42.183

9

You need to understand the difference between __index and __newindex, and their relationship with the current contents of the main table.

__newindex is only called/accessed when all the following are true:

When you are setting a value into the main table, via tbl[index] = expr (or equivalent syntax, like tbl.name = expr).
When the key you are trying to set into the main table does not already exist in the main table.

The second one trips people up often. And that's your problem here, because __index is only accessed when:

When the key being read from the main table does not already exist in the main table.

So if you want to filter every read from and write to a table, then that table must always be empty. Therefore, those reads and writes need to go into some other table you create for each new object. So your new function needs to create two tables: one that remains empty and one that has all the data in it.

Honestly, I wish Lua had a way to create just an empty piece of userdata that you could bind a user-defined metatable to, just to avoid these issues.

edited Nov 30 '16 at 22:46

answered Aug 02 '13 at 21:48

Nicol Bolas

449,505
63
781
982

1

I understood __newindex, but I don't see how that applies to my problem, after reading your post ~4 times in a row, it seems that you suggest having the metatable empty, with all the functions stored in a table inside of the object, and the data in a seperate table, with __newindex being used as a track switcher, to decide which of the two tables are accessed when the metatable is indexed? Couldn't I already just write some code in __index and/or __newindex that will be a track switcher to the functions in the metatable, and the data in the table in the metatable? – Weeve Ferrelaine Aug 02 '13 at 22:02
my confusion is in how to identify when the metatable is being indexed for a function or a number, as a number is the only valid type for accessing the Data table (in specific, I am writing an image class) – Weeve Ferrelaine Aug 02 '13 at 22:03
1

Lua used to have a `proxy` function that created that piece of userdata you mention. It's easy to add it yourself. – lhf Aug 02 '13 at 22:15
@JohnDoe: "*I don't see how that applies to my problem*" Because `__index` has the *same restriction* as `__newindex`. It's the part where I said, "And that's your problem here...". If you want to filter accesses to a table, you *must* do what I said. If your main table has members directly in it, your `__index` metamethod will never be called for accesses to them. – Nicol Bolas Aug 02 '13 at 22:23
1

@lhf: "*It's easy to add it yourself.*" From the C API, certainly. But if you're writing a pure-Lua script, it's pretty much impossible. That's why it should be a simple standard library thing, so that all scripts can have access to it. – Nicol Bolas Aug 02 '13 at 22:25
Now I understood you, Look at the answer I posted below, it ran, surprisingly, and I didn't have to have the metatable object empty, thanks for the help <3 – Weeve Ferrelaine Aug 02 '13 at 22:56
I suspect it wasn't thrown into recursion because the metatable itself dosn't call __index, and only the objects created from it do, is this true? – Weeve Ferrelaine Aug 02 '13 at 22:58

score 0 · Answer 2 · answered Aug 02 '13 at 22:54

0

the way I resolved this problem, according to Nicol Bolas's solution, if it might give clarity to anyone else's confusion :-)

TestMetatable = {DataTable = {}, FunctionTable = {}}

function TestMetatable.new()
    local Tmp = {}
    setmetatable(Tmp,TestMetatable)

    Tmp.DataTable = {1}
    Tmp.FunctionTable = TestMetatable

    return Tmp
end

function TestMetatable:TestMethod()
    print("Ran Successfully")
end

function TestMetatable.__index(self,index)
    if type(index) == "string" then
        return self.FunctionTable[index]
    else
        return self.DataTable[index]
    end
end

local Test = TestMetatable.new()

-- both functionalities are needed
print(Test[1])
Test:TestMethod()

answered Aug 02 '13 at 22:54

Weeve Ferrelaine

683
1
7
12

Unless someone does `Test.TestMethod = anything`, and then your code breaks. – Nicol Bolas Aug 02 '13 at 23:01
true, but wouldn't Test.TestMethod return the value of the function in the metatable, and then the function in the metatable would be set to anything? regardless, I will never need to change the functionality of a class after its made (its against oop), but I could see how some might, so thank-you for pointing that out – Weeve Ferrelaine Aug 02 '13 at 23:10
No. `tbl.MemberName = X` will call `__newindex`, not `__index` to set `X` into the `tbl` field `MemberName`. And again, only if `MemberName` doesn't have a value in the table. That's why it's important to use both and to keep the table empty if you want control over where stuff gets stored. – Nicol Bolas Aug 02 '13 at 23:13
but wouldn't Test.TestMethod = X be accessing something that already exists in Test (because all the functions are declared in the scope of the metatable) even though they aren't the one's being accessed when you index it, I could see that Test.NewFunction = X will error, however, I will test Test.TestMethod out, and see :-) – Weeve Ferrelaine Aug 02 '13 at 23:18
1

@JohnDoe It might be helpful to think of it as `__index` handles read access and `__newindex` handles write access to a table. `Test.TestMethod = anything` is trying to assign some value to table `Test` so that's write access and so `__newindex` is used here. – greatwolf Aug 02 '13 at 23:18
but its write access to an already existing index? is __newindex still called for it then? I had __newindex explained as used only for creation of non-pre existing indexes – Weeve Ferrelaine Aug 02 '13 at 23:19
1

@JohnDoe in that case no. Remember, these are "fallback" handlers. Lua only consults them if the key *doesn't* already exist in the table. The proxy pattern explained by Nicola and in PiL works because you start off with an empty table *and* you keep it empty. You do this by intercepting all read and write operations on that table via `__index` and `__newindex`. – greatwolf Aug 02 '13 at 23:20
any changes to a function require violation of oop, so isn't this functionality being nonexistant also non-applicable, as the intent of metatables was to simulate oop – Weeve Ferrelaine Aug 02 '13 at 23:22
__index dosn't seem to operate as a fallback operator, in practice, it called my __index, even when TestMetatable.TestMethod was a valid key – Weeve Ferrelaine Aug 02 '13 at 23:24
1

That's because `__index` never assigns a value to that key. After the call, `TestMetable.TestMethod` still isn't a valid key in your table because it's still `nil`. The moment you make it non-nil by assigning something to it, then `__index` won't be called for `TestMetable.TestMethod` anymore. Again, you can of course *intercept* that assignment with `__newindex` and handle it differently. – greatwolf Aug 02 '13 at 23:27
run the code I posted, it functions properly, TestMetatable.TestMethod index isn't nil because __index was called for it, and not __newindex, have it print when its called, it shocked me too – Weeve Ferrelaine Aug 02 '13 at 23:31
@JohnDoe So how can you know if a table has a key or not? Simple, just iterator over the table and dump the key-values out. `for k, v in pairs(t) do print(k, v) end`. That's how you know for sure if table has a key. You'll find accessing any of those keys *never* trigger `__index`. – greatwolf Aug 02 '13 at 23:32
@JohnDoe: You keep missing the point. That point being that *you can change the `TestMethod` value*. Just stick `Test.TestMethod = 4` before calling it, and see whether the method gets called. – Nicol Bolas Aug 02 '13 at 23:34
Im on my windows machine, cant right now, Ill check in the morning, and possibly edit my post if it dosn't – Weeve Ferrelaine Aug 03 '13 at 00:05
Worked as expected, I got attempt to call a number value, as expected, and not attempt to call a nil value – Weeve Ferrelaine Aug 03 '13 at 00:17

I need clarification on Metatable.__index

2 Answers2

Linked