1

I'm using the below script i found online to fix cloudfront routing on a nextjs statically generated website hosted on s3.

It Works but its terribly slow considering i have over 10000 pages ( it takes 5-10 seconds per copy ) is there a better/faster way to do this?

(cd out &&
  find . -type f -name '*.html' | while read HTMLFILE; do
    HTMLFILESHORT=${HTMLFILE:2}
    # HTMLFILE_WITHOUT_INDEX=${HTMLFILESHORT::${#HTMLFILESHORT}-11}

    HTMLFILE_WITHOUT_INDEX=${HTMLFILESHORT//index.html/}
    HTMLFILE_WITHOUT_HTML=${HTMLFILE_WITHOUT_INDEX//.html/}


    # cp /about/index.html to /about
    aws s3 cp s3://$S3_BUCKET/${HTMLFILESHORT} s3://$S3_BUCKET/$HTMLFILE_WITHOUT_HTML
    echo aws s3 cp s3://$S3_BUCKET/${HTMLFILESHORT} s3://$S3_BUCKET/$HTMLFILE_WITHOUT_HTML

    if [ $? -ne 0 ]; then
      echo "***** Failed renaming build to $S3_BUCKET (html)"
      exit 1
    fi
  done)
John Rotenstein
  • 241,921
  • 22
  • 380
  • 470
Trentfrompunchbowl1
  • 107
  • 9
  • If you have an aws ec2 instance. it might be quicker to run your script there. – Philippe Dec 20 '21 at 22:17
  • 1
    When you have some non-trivial number of files, it's generally faster to use `aws s3 sync` to copy files locally, then operate on the local copies. Or, stop calling the cli all together and do the work in a python script. – Anon Coward Dec 21 '21 at 00:02
  • @AnonCoward thanks the issue with the local files is i have a file called company and also a directory called the exact same so it doesn't work locally only in the bucket ( weird thing with nextjs ) ill give the python idea a shot! didn't think of that thanks mate – Trentfrompunchbowl1 Dec 21 '21 at 00:32
  • Have you tried mounting the S3 bucket? I have no idea if it would be faster (probably not), but is something possibly to look at. See https://stackoverflow.com/a/43720522/503621 – B. Shea Dec 21 '21 at 02:51

0 Answers0