Can I avoid the for loop in normalizing data by using : operator in matlab?

Question

Here is the code I am using -

mu = mean(X);
sigma = std(X);

for iter = 1:size(X, 1)
  X_norm(iter,:)=((X(iter,:)-mu)./sigma)
end

I was wondering if there was a way to do this without using a loop and using only the basic operators, something by which I could add each row in X by mu and divide each by sigma.

X and X_norm are matrices. One way I found out is - (Although it uses the function ones )

one= ones(size(X, 1), 1);
X_norm = (X - one*(mean(X)))./(one*std(X));

Please note that I want to know more about using the basic operators so don't suggest any libraries or toolkits .

If you are going to post a function, post its implementation as well

Is your code correct? I would expect a `./` instead of `/`? I assume `X` is a matrix? — Daniel, Oct 24 '15 at 11:52
I think it should be `(X - one*(mean(X)))./(one*std(X))` rather. — Divakar, Oct 24 '15 at 13:37
So, we are done right? They lived happily everafter with the `ones()` and "basic" functions. — Divakar, Oct 24 '15 at 13:45
I want to know if there is any other way, this is just a trivial matrix multiplication and uses a function too, but thanks for your help too @Divakar, Because of you I got to know of bsxfun which is supposedly the best function for these kind of works, I am now trying to find its source — Naman Jain, Oct 24 '15 at 13:49

Divakar · Accepted Answer · 2015-10-24T13:07:53.077

3

Use bsxfun -

X_norm = bsxfun(@rdivide,bsxfun(@minus,X,mu),sigma)

You can also use ones() with your beloved operator : for replication as stated in Loren's blog and then perform the stated operations, like so -

M = size(X,1);
X_norm = (X - mu(ones(M,1),:))./sigma(ones(M,1),:)

For performance, I would go with bsxfun any day!

edited Oct 24 '15 at 13:07

answered Oct 24 '15 at 12:10

Divakar

218,885
19
262
358

It does the job but what I asked for is a way to use : operator to do the job if I have to use other functions, I might as well use the for loop – Naman Jain Oct 24 '15 at 12:19
1

@NamanJain `:` operator is to slice/ select elements from an array. You are asking to avoid loop, that is do everything in one go. Thus functionally, slicing and vectorizing would be two opposite things. What exactly do you have in mind? Could you clarify what's the motive behind using `:` operator in avoiding loop? – Divakar Oct 24 '15 at 12:25
I mean when you do X(:)=X(:) +5 it adds 5 to all the elements, In the same fashion I want to use it to add a row vector to all the rows of the matrix – Naman Jain Oct 24 '15 at 12:35
@NamanJain `bsxfun(@minus,X,mu)` subtracts row vector `mu` from the 2D array `X`, giving us a 2D array, such that all the rows of `X` are subtracted by the same row vector `mu`. How does this differ from what you just commented? – Divakar Oct 24 '15 at 12:42
For example consider X to be a vector of datatype Y . Case 1 : Y is int. X(:)=X(:)+Y adds Y to all the elements of vector Y . Case 2 : Y is a row vector. what I want is to figure out how to use : in this case. I dont know the implementation of bsxfun, it may use some loops in its code – Naman Jain Oct 24 '15 at 12:46
@NamanJain 1) `"consider X to be a vector of datatype Y . Case 1 : Y is int. X(:)=X(:)+Y "`, I am not sure how this whole thing went to datatypes and adding datatypes with variables `X(:)+Y`? 2) `" I dont know the implementation of bsxfun"`, well it doesn't use a loop at MATLAB level and it's [`fast`](http://stackoverflow.com/questions/29719674/comparing-bsxfun-and-repmat), so I don't see the apprehension with using it. – Divakar Oct 24 '15 at 12:58
Please read updated description, also I don't have any qualms with using repmat or bsxfun I just wanted to know if it is possible to do so using only the basic operators of which I have provided an example.I am not trying to get the work done here, I am trying to know more . – Naman Jain Oct 24 '15 at 13:03
@NamanJain I think you meant like the edited codes in this post? – Divakar Oct 24 '15 at 13:08
This - X_norm = X - (ones(size(X, 1), 1) * (mean(X)./std(X)); – Naman Jain Oct 24 '15 at 13:12
But you do get what I am trying to do here , Right ? – Naman Jain Oct 24 '15 at 13:18
@NamanJain Since that doesn't work, please remove that code from the question. Also, check out the edits in this post where it uses `ones()`, think you meant that way. – Divakar Oct 24 '15 at 13:22
Corrected the code, It should work now, My point is, I only want the basic functions, I don't even know how bsxfun works – Naman Jain Oct 24 '15 at 13:24
Did you just make my answer yours ? – Naman Jain Oct 24 '15 at 13:27
@NamanJain Your code in the question doesn't work. It gives you wrong answer. And No, I didn't copy it, see the codes more diligently and compare them carefully, if you are really trying to learn here. Also, if you are posting an answer, post an answer, not add into your question, unless, you have a code that is not working and you are asking here on how to rectify the issue. – Divakar Oct 24 '15 at 13:32
I added it in the description just as an example for what kind of answers I want – Naman Jain Oct 24 '15 at 13:36

score 3 · Answer 2 · answered Oct 24 '15 at 12:19

Note: @Divakar posted a very elegant answer using bsxfun, and I'd suggest to use that approach. Still, I am posting this answer (as I already wrote most of it when the other answer was posted), so you can see what was wrong with your existing code.

As your goal is to normalize every row of the matrix X to zero mean and variance 1, then your version doesn't work as expected. As @Dan remarked in a comment, by using / to divide, you do a matrix division between two row vectors, which creates a scalar as result. The output of the loop is therefore a n x 1 column vector (of which I don't know what it actually contains...).

First, realize that mean returns a row vector, where each entry contains the mean of the corresponding column. To normalize row-wise, you have to get the mean of each row, which can be done by mean(X,2). The same goes for std, i.e. use std(X,[],2).

mu = mean(X,2);
sigma = std(X,[],2);

A version using for loops would now be

X_norm = zeros(size(X));
for k= 1:size(X, 1)
  X_norm(k,:) = (X(k,:)-mu(k)) ./ sigma(k);
end

i.e. go through all rows and subtract the mean of the row, and divide by the standard deviation. By using mu(k), you use the correct mean/std dev for every row.

Also, don't forget to preallocate the matrix X_norm for performance reasons.

I know what the mean function returns, And I corrected the typo of ./ 10 minutes before your answer was posted. Also I did the preallocation part but didn't post it here because it was not relevant to the question. Your answer is useless for me as you just replaced the variable iter in the question and are now explaining the code to the guy who wrote it in the first place. — Naman Jain, Oct 24 '15 at 12:29
@NamanJain The code in the question (also with `./`) does not normalize the data correctly, which is why I still posted the answer. Sorry if it doesn't help you any further. — hbaderts, Oct 24 '15 at 13:24

IKavanagh · Answer 3 · 2015-10-24T12:41:35.693

2

Another solution for you using repmat is

(X - repmat(mu, length(X), 1)) ./ repmat(sigma, length(X), 1)

This is possibly a little easier to understand than bsxfun. I have posted this solution to demonstrate that to carry out the operation you want you need to replicate your vectors, mu and sigma. repmat can be used for that, : cant.

Additional Reading: Colon Operator in MATLAB

Although, I would recommend going with bsxfun.

edited Oct 24 '15 at 12:41

answered Oct 24 '15 at 12:26

IKavanagh

6,089
11
42
47

I don't wish to use another function, if it were so i would have used for loop, what I want is to use the : operator as mentioned in the question and commented in divakar's answer – Naman Jain Oct 24 '15 at 12:37
Yes and your understanding of how `:` works is wrong. `:` is used for indexing not replication which is what you are interested in. The `repmat` function I've used will replicate your vector which is the functionality you seem to be looking for `:` to carry out. This is why I posted this solution. – IKavanagh Oct 24 '15 at 12:39
I just want to know if there is a way to use : or other basic operators to accomplish this task. For example consider X to be a vector of datatype Y . Case 1 : Y is int. X(:)=X(:)+Y adds Y to all the elements of vector Y . Case 2 : Y is a row vector. what I want is to figure out how to use : in this case. If its not possible in any way that you know of, say so. – Naman Jain Oct 24 '15 at 12:56
No, what you want is not possible with `:`. – IKavanagh Oct 24 '15 at 13:15

score 2 · Answer 4 · answered Oct 24 '15 at 13:05

2

If you have the Statistics Toolbox, you can use the built-in function zscore()

X_norm = zscore(X);

answered Oct 24 '15 at 13:05

tashuhka

5,028
4
45
64

Can I avoid the for loop in normalizing data by using : operator in matlab?

4 Answers4