3

I have an array called skj. skj contain 2 million rows with numbers (2000000x1 uint32).

I want to compute the following

string_skj = num2str(skj);

When I run the above line it takes about 1 minute, is there a faster way of doing it?

IKavanagh
  • 6,089
  • 11
  • 42
  • 47
Søren J
  • 31
  • 6
  • 1
    `int2str` is faster as well than `num2str`. – Adriaan Nov 05 '15 at 21:34
  • @IKavanagh: That produces a different output – Daniel Nov 05 '15 at 21:34
  • @Daniel Yes, that was completely wrong. Thanks. – IKavanagh Nov 05 '15 at 21:44
  • 1
    @Adriaan Not on my machine :P Their execution is equal. – IKavanagh Nov 05 '15 at 21:45
  • By the way what are you trying to solve? I've shown how you can accelerate stuff, but the output would be a concatenated transposed string. Maybe you could work your way further from the concatenated string? – Hennadii Madan Nov 05 '15 at 22:23
  • 2
    Another very fast 'solution' the output is not exactly the same, but similar `a = sprintf('%d\n',skj)`. – Hennadii Madan Nov 05 '15 at 22:35
  • You should probably specify OS and Matlab version. I do not experience the same problems as you do. For an array 1:2e6 using `timeit`, `num2str` runs on about 1.1 seconds, while `sprintf` runs on about 0.8 seconds. Windows 7, Matlab 2014b. A column vector is even faster. – patrik Nov 06 '15 at 07:44

5 Answers5

5

Hennadii Madan's answer got me thinking if there was a way to do this for column vectors more efficiently than the standard Matlab num2str (or int2str) and I've come up with 2 solutions that do.

EDIT: And after all that work @Luis Mendo comes in and blows it all out of the water :'(

EDIT: Now @Daniel has improved on all of the previous options again!


Given our row vector, V, as

V = uint32(randi(100, 200000, 1));

we can achieve the same result as

A = num2str(V);

with *

B = char(strsplit(num2str(V.')).');

or without the error checking of num2str

C = char(strsplit(sprintf('%d\n', V)).');
C = C(1:end-1, :); % Remove extraneous '\n'

B and C are slightly different to A. num2str pre-pads with a space, ' ', whilst B and C post-pad with a space.

In the below D and E are pre-padded with 0's and so do not match A, B or C exactly.


Benchmarks

-----num2str() on row vector [Original]-----
Elapsed time is 3.501976 seconds.
  Name           Size              Bytes  Class    Attributes

  A         200000x3             1200000  char               

-----num2str() on column vector [IKavanagh modified from Hennadii Madan]-----
Elapsed time is 0.660878 seconds.
  Name           Size              Bytes  Class    Attributes

  B         200000x3             1200000  char               

-----sprintf() on row vector [IKavanagh]-----
Elapsed time is 0.582472 seconds.
  Name           Size              Bytes  Class    Attributes

  C         200000x3             1200000  char               

-----dec2base() on row vector [Luis Mendo]-----
Elapsed time is 0.042563 seconds.
  Name           Size              Bytes  Class    Attributes

  D         200000x3             1200000  char



-----myfastint2str() on row vector [Daniel]-----
Elapsed time is 0.011894 seconds.
  Name           Size              Bytes  Class    Attributes

  E         200000x3             1200000  char 

Code

clear all
close all
clc

V = uint32(randi(100, 200000, 1));

for k = 1:50000
    tic(); elapsed = toc(); % Warm up tic/toc
end

disp('-----num2str() on row vector [Original]-----');
tic;
A = num2str(V);
toc, whos A

disp('-----num2str() on column vector [IKavanagh modified from Hennadii Madan]-----');
tic;
B = char(strsplit(num2str(V.')).');
toc, whos B

disp('-----sprintf() on row vector [IKavanagh]-----');
tic;
C = char(strsplit(sprintf('%d\n', V)).');
C = C(1:end-1, :); % Remove extraneous '\n'
toc, whos C

disp('-----dec2base() on row vector [Luis Mendo]-----');
tic;
D = dec2base(V, 10);
toc, whos D

disp('-----myfastint2str() on row vector [Daniel]-----');
tic;
E = myfastint2str(V);
toc, whos E

Community
  • 1
  • 1
IKavanagh
  • 6,089
  • 11
  • 42
  • 47
  • This is the closest so far, I wonder if we could beat the space-is-not-there problem – Hennadii Madan Nov 05 '15 at 22:46
  • @HennadiiMadan The space is there to ensure the dimensions are the same for each row. We could move it to the end but to be honest I prefer it at the front. – IKavanagh Nov 05 '15 at 22:48
  • I updated my code to produce only a nx3 array. Should be faster now. – Daniel Nov 05 '15 at 22:55
  • @IKavanagh I just checked, on my machine sprintf version will work equally fast with or without transposing. [ LOL ] – Hennadii Madan Nov 05 '15 at 22:58
  • @Daniel It is considerably faster than all of the previous solutions now! – IKavanagh Nov 05 '15 at 23:02
  • @HennadiiMadan Yes I know, that's because of how the vectorised version of `sprintf` works. It basically for this example treats row and column vectors the same. – IKavanagh Nov 05 '15 at 23:03
4

The following is much faster on my machine:

y = dec2base(skj,10);

Here's a quick test:

>> skj = uint32(2^32*rand(1e6,1)); %// random data

>> tic, y = num2str(skj); toc
Elapsed time is 22.823348 seconds.

>> tic, z = dec2base(skj,10); toc
Elapsed time is 1.235942 seconds.

Note that using dec2base gives leading zeros instead of leading spaces.

>> y(1:5,:)
ans =
3864067979
1572155259
1067755677
2492696731
 561648530

>> z(1:5,:)
ans =
3864067979
1572155259
1067755677
2492696731
0561648530
Luis Mendo
  • 110,752
  • 13
  • 76
  • 147
4

Implementing int2str yourself, you can beat the performance of the original function by far.

function [ o ] = myfastint2str( x )
maxvalue=max(x(:));
%maxvalue=intmax(class(x));%Alternative implementation based on class
required_digits=ceil(log(double(maxvalue+1))/log(10));
o=repmat(x(1)*0,size(x,1),required_digits);%initialize array of required size
for c=size(o,2):-1:1
   o(:,c)=mod(x,10);
   x=(x-o(:,c))/10;
end
o=char(o+'0');
end

For the example input, my function required less than 0.15 seconds, while both int2str and num2str took about 15 seconds. The output is slightly different as it generates leading zeros instead of blanks.

Daniel
  • 36,610
  • 3
  • 36
  • 69
  • @IKavanagh: I removed the pre-pad after I saw your benchmark only using three digits :) The code always generating 10 digits is the third line (comment) – Daniel Nov 05 '15 at 22:56
  • That's a much better improvement now. Now it will change width according to the size of the largest number in the vector. – IKavanagh Nov 05 '15 at 23:03
1

If you really need to increase speed, have you considered writing a MEx function extension in C? It's a little bit complicated, but it's worth investing the time if you have some small routines that can easily be coded in C/C++. Once compiled, the MEx function can be called from the MATLAB command prompt, just like a .m function.

See http://www.mathworks.com/help/matlab/call-mex-files-1.html for more details.

gariepy
  • 3,576
  • 6
  • 21
  • 34
  • 1
    I don't think writing a mex function is the right choice in this case. For simple algorithms and large data, the overhead for calling a mex function is typically to large. Mex is perfect for complicated code (especially many loops). – Daniel Nov 05 '15 at 23:05
  • 1
    Well, if you've tried it, I understand. However, in my experience, the overhead has not been an issue, even for small functions and large data. MEx functions are loaded just like MATLAB built-ins The MEx interface would just create a C-structure wrapper around the array, and the MEx function would be accessing a pointer to the numeric array, so there is no copying involved. – gariepy Nov 05 '15 at 23:59
  • From my experience an implementation using efficient m code is the best choice, and having provided the fastest answer I think I am right. Nevertheless, I do not understand why this answer received a down vote, it is a reasonable idea. – Daniel Nov 06 '15 at 19:33
  • That is interesting...I have not benchmarked MEx files in a couple of years...it's quite possible that MATLAB has significantly improved the performance of .m files such that they are on par with MEx implementations. I'll have to go back and test some of my old routines again. – gariepy Nov 06 '15 at 21:40
  • @daniel You only need to make 1 mex call! Mex can absolutely crush it. For example: calling Matlab's str2double on a one million entry cell array of strings takes 25 seconds on my machine. Calling [a mex version](http://www.mathworks.com/matlabcentral/fileexchange/28893-fast-string-to-double-conversion) takes 0.24 seconds to convert to double!! 100x faster. There are certain situations where Matlab isn't efficient and coding a c++ function, though a pain, can be WAY faster. – Matthew Gunn Nov 09 '15 at 05:44
0

Warning: the output is wrong, but may be workable.

Edit: A super fast 'solution' output is not a column, but a string with line breaks as separators. If you try to print it it will look the same

>> tic;a = sprintf('%d\n',skj);toc
Elapsed time is 0.422143 seconds

Edit: Old 'solution'

Try transposing before and after. Like num2str(skj.').'

>> skj = ones(2000000,1,'uint32');
>> tic;num2str(skj);toc
Elapsed time is 23.305860 seconds.
>> tic;num2str(skj.');toc
Elapsed time is 1.044551 seconds.
IKavanagh
  • 6,089
  • 11
  • 42
  • 47
Hennadii Madan
  • 1,573
  • 1
  • 14
  • 30
  • This produces the wrong output because of the last `.'`. – IKavanagh Nov 05 '15 at 21:41
  • Is it faster? If you need transposed output just spare the last .' – Hennadii Madan Nov 05 '15 at 21:47
  • @HennadiiMadan why would `num2str(skj.')` be any faster than `num2str(skj)`? – David Nov 05 '15 at 21:49
  • I just tested and the transpose did seem to take a little longer. – David Nov 05 '15 at 21:51
  • @David I tested and it is much faster. Why it would be different - because of interplay of memory organization of the arrays and looping inside num2str. – Hennadii Madan Nov 05 '15 at 21:56
  • Never perform timing in the Command Prompt. There is some overhead experienced in between commands that you are introducing into the timing. Put this in a function script and run it (BTW, I didn't downvote you). – rayryeng Nov 05 '15 at 22:01
  • @rayryeng, using functions will also maximize available optimization, which would be unavailable interactively. – Andras Deak -- Слава Україні Nov 05 '15 at 22:04
  • Neither did I. But `num2str(skj.').'` produces an incorrect output because of how the transpose works on character arrays. You should change it to `num2str(skj.')`. – IKavanagh Nov 05 '15 at 22:04
  • Also, try not to use `tic/toc`. It requires warmup: Here's an example of why you need it: http://stackoverflow.com/questions/26904392/how-to-vectorize-finding-the-closest-point-out-of-a-vector - Use `timeit` instead. – rayryeng Nov 05 '15 at 22:05
  • 1
    It may be faster, but it just doesn't produce the correct result. Try with `skj = uint32(1:200000)'`. It either creates a single string with all the elements, or if transposed puts each character on a separate line, which I'm quite sure is not what anyone would want. – zelanix Nov 05 '15 at 22:10
  • 2
    Guys, this transpose problem should highlight a huge source of the timing difference. When num2str-ing a column vector, matlab has to determine the smallest common char length for constructing a square char matrix. This is not necessary when converting a single row! And doing so means determining the number of printable digits of 200k numbers... – Andras Deak -- Слава Україні Nov 05 '15 at 22:11
  • Yeah it really returns differnt output, sorry – Hennadii Madan Nov 05 '15 at 22:15
  • Moreover if I try specifying '%d\n' as a format (second) argument to num2str it hangs my comp :( – Hennadii Madan Nov 05 '15 at 22:17