2

The current scripts i have written so far: 

#!/bin/ksh

rm -fR /tmp/aclget
cd /

find ./ -type d \
| grep -v "^./tmp/" \
| xargs -I {} mkdir -p "/tmp/aclget/{}"

find ./ \
| grep -v "^./tmp/" \
| xargs -I {} aclget -o "/tmp/aclget/{}.acl" "{}"

cd /tmp
tar -cvf acl.tar aclget
gzip acl.tar

To replicate the permission on another machine

#!/bin/ksh
cd /tmp
gunzip acl.tar
tar -xvf acl.tar

cd /tmp/aclget
find ./ -exec aclput -i {} `echo "{}" | sed "s/^\.//g"`

The problems i am having is that the tmp always goes out of space when building the list. is it possible to make pipe so that it can work on the fly ?

update code after reading feedback and trying with some idea of uuencode, change after some testing. (probably kernel tuning is need for the 255 bytes limit in xargs)

Checking:

odmget -q "attribute=ncargs" PdAt          
lsattr -El sys0 | grep ncargs

Tunning:

chdev -l sys0 -a ncargs=1024

Scripts:

#!/bin/ksh

cd /

find ./ -name "^./tmp/" -o \
-exec sh "aclget '{}' | uuencode - '/tmp/acl/{}.acl'" \; \
| gzip acl.uu.gz

anyone got idea on the the reverse part ? i have come up so far, need to cut the file down since uudecode can only decode the first file

#!/bin/ksh

gunzip acl.uu.gz | uudecode
cd /tmp
find ./ -exec aclput -i {} `echo "{}" | sed "s/^\.//g"`
  • i knew that actually tar can have options to replicate the permissions as well, but the point is that i have to replicate the permissions only, but not the file contents. –  Nov 16 '13 at 12:10
  • in bounty reason you say answer does not contain enough details. What details are you missing? Or do you just want a complete and production ready solution? – akostadinov Nov 19 '13 at 09:57
  • i am still working on the production ready solution, still cannot get this things works.... just found that the sh didn't work.... also, the -exec got some problems too... not exactly know the answer. –  Nov 19 '13 at 14:29
  • i still have around 2 weeks to get this thing delivered to my boss... to solve the file permission difference issues on different environment.... Their content are very much the same, just the permission differ..... which we call a pit trap which everything tested in UAT may have potential issues when launched... –  Nov 19 '13 at 14:31
  • i did try ls -alR then perform diff on the files, which i found quite hard since it is impossible for me to replicate the acl... or actually i should generate a diff list before i perform aclget and aclput ??? –  Nov 19 '13 at 14:33
  • but the problems comes up again since the acl can differ despite in some situation the file looks exactly the same while perform ls -alR –  Nov 19 '13 at 14:34
  • If you like, try the while loop I posted to you. Comment on my answer what problems you are seeing. I updated it to be ksh compatibe (tested on aix7). Also filesize is irrelevant now. Just adjust $treshold according to `update 2`. – akostadinov Nov 20 '13 at 13:17

2 Answers2

0

I never used aclget but here are some generic approaches you can use.

First you could do the work on chunks directory by directory and hope no single chunk is too big. Also you can optimize tar like:

tar -vc aclget | gzip -9 > myfile1.tar.gz

btw bzip and xz have much better compression ratio if needed.

Another approach could be instead of:

xargs -I {} aclget -o "/tmp/aclget/{}.acl" "{}"

to have a while loop like:

(
  batch_idx=0
  iterations=1
  threshold=$(( 100 * 1024 * 1024 ))
  while read -r FILE; do
    aclget -o "/tmp/aclget/$FILE.acl" "$FILE"
    tar uv -C /tmp/aclget -f acl.tar $FILE.acl
    rm "/tmp/aclget/$FILE.acl"
    if (( iterations%threshold == 0 )); then
      mv acl.tar acl_$batch_idx.tar
      gzip acl_$batch_idx.tar
      (( batch_idx+=1 ))
      iterations=0
    fi
    (( iterations+=1 ))
  done
  mv acl.tar acl_$batch_idx.tar
  gzip acl_$batch_idx.tar
)

It is also very good to use find's -print0 option but then you would need to change the IFS variable within the while sub-shell. You need to test that though, hopefully it wont affect any of the other commands besides read.

You see, I'm not giving you a tested solution but I see you are more than capable to turn the above into working code. Regards!

update: forgot to tell you about the used pseudo-funciton filesize. It's very platform dependent what you have on your machines. If can use stat, wc, du and others.. see here for suggestions:

update 2: I updated the while loop to be ksh (from aix7) compatible as well I avoid using filesize as a criterion. You would need to adjust threshold variable so it is large enough to to avoid big number of output files and sill fit into your free space.

Community
  • 1
  • 1
akostadinov
  • 17,364
  • 6
  • 77
  • 85
  • It seems much more promising now.... i guess the point is that i am too greedy to do all the job at once and then copy and paste to a large dozen of servers. actually do it file by file and that would be more easy to handle. i mean that i can just append each acl to end of the tar file then to it 1 by 1... –  Nov 20 '13 at 14:49
  • @AntonyLee, you can't update `.tar` archive if already compressed. You need it decompressed. So it still can overflow your filesystem. That's why when tar grows big, you need to compress and start with another tar. Btw in my code it will be better to tar.gz each chunk at once instead of adding each acl one by one. It's gonna be slow. But I don't have time today to move tarring into the `if` block. Hey I just remembered, zip allows for adding to archive without devcompression. You can get away with a single `zip` archive that way. – akostadinov Nov 20 '13 at 14:56
0

It have been too long, so i open another tab to continue,

The issues have finally been conclude that i am too demanding in building simple and efficient scripts. there is no 100%, so it's my problems.

Also, it seems that gzip in my box is unable to perform on the fly compression, it accumulate the changes in memory, so compress is used instead.

The version now:

dump_acl2.sh

#!/bin/ksh
aclget $1 | uuencode -m $1.acl

dump_acl.sh

#!/bin/ksh

cd /
find ./ \
-type nfs \
-o -name "^./tmp/" \
-o -exec \tmp\dump_acl2.sh {} \; \
| compress -c \
> \tmp\dump_acl.`hostname -s`.uu.Z