GitHub Actions workflow to copy everything (all files and folders) into a folder in another branch

Question

I have two branches:

- main (Contains nothing but just a README.md)

- my-branch (Contains some files and folders with subfolders)

- gh-pages (This will be deployed using GitHub Pages settings)

I want to copy all content of my-branch into a folder called myFolder inside gh-pages whenever there's a push to my-branch. How can I implement that in YAML?

PS: I was initially trying to copy just the files that has changed, maybe using rsync, but found that it was too difficult, so I guess I'll copy the entire thing every time there's a push, but if you have any better way, please help.

Edit: Here's what I've gotten so far.

name: Copy files to another branch

on:
    push:
        branches:
            - better-page
    workflow_dispatch:

permissions:
    contents: write
    pages: write
    id-token: write

concurrency:
    group: "pages"
    cancel-in-progress: false

jobs:
    copy:
        runs-on: ubuntu-latest
        steps:
            - name: Checkout better-page
              uses: actions/checkout@v2
              with:
                  ref: better-page

            - name: Copy files
              run: |
                  git config --global user.name "GitHub Actions"
                  git config --global user.email "action@github.com"
                  rsync -r --exclude=.git --exclude=.github --exclude=about-me --exclude=node_modules --exclude=simplified . simplified/
                  git add .
                  git diff-index --quiet HEAD ||  git commit -am "create simplified directory"  # commit to the repository (ignore if no modification)
                  git push origin better-page # push to remote branch
                  git checkout gh-pages
                  git fetch origin gh-pages
                  git pull origin gh-pages
                  git push origin gh-pages
                  git checkout better-page
                  git fetch origin better-page
                  git pull origin better-page
                  git push origin better-page
                  git checkout gh-pages
                  git checkout remotes/origin/better-page -- simplified
                  git add -A
                  git diff-index --quiet HEAD ||  git commit -am "deploy files"
                  git push origin gh-pages
                  git checkout better-page
                  rm -rf simplified
                  git add -A
                  git diff-index --quiet HEAD ||  git commit -am "remove simplified directory"
                  git push origin better-page

But it gave me this error

Everything up-to-date
error: pathspec 'gh-pages' did not match any file(s) known to git
Error: Process completed with exit code 1.

I don't know what's going on or what to do.

When using the [`actions/checkout`](https://github.com/actions/checkout) in a github action workflow, you gain access to all the repository content (using the input `fetch-depth: 0` also givea you access to the tags and branches history). In that case, you could just create a folder to copy the branch content in the folder you want before performing ano other operation (using `mkdir`, `cp` or `mv` command lines in a `run` step). — GuiFalourd, Apr 17 '23 at 12:36
How would you do this in real life? In other words, suppose you were working on your local computer and I told you to copy a folder from one branch of a repo into another. How would you do it? — matt, Apr 17 '23 at 12:37
@GuiFalourd So, I create a new folder `myFolder` first in `my-branch`, copy all the file content to there, and then checkout that folder into the `gh-pages` branch? — DavidNyan10, Apr 17 '23 at 13:15
@matt I mean I'd create a tmp folder outside the repository, create a new folder `myFolder` in the tmp folder, copy all the content of `my-branch` into `myFolder`, git checkout `gh-pages`, then move `myFolder` into the repository folder. But how would I implement that in git or GitHub workflow? Maybe using tarpipe but I don't know how that works. — DavidNyan10, Apr 17 '23 at 13:18
You can run all the command lines you explained by using a `run` [step](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idsteps) in you workflow directly. — GuiFalourd, Apr 17 '23 at 13:28
This should work yes. If not, the final solution might not be too different from it. — GuiFalourd, Apr 17 '23 at 14:10
@GuiFalourd I have tried fixint it until https://pastebin.com/QbE2eScy but now I don't know how to fix the error ```Run git checkout better-page -- simplified fatal: invalid reference: better-page Error: Process completed with exit code 128.``` — DavidNyan10, Apr 17 '23 at 14:25
You don't need to use the actions/checkout so many times. Just once at the beginning of the workflow job. This might be interfering here. — GuiFalourd, Apr 17 '23 at 19:47
Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/253206/discussion-between-davidnyan10-and-guifalourd). — DavidNyan10, Apr 18 '23 at 06:55

score 0 · Answer 1 · answered Apr 18 '23 at 15:13

Let's first of all do git in the answer, then map that to actions. Localhost is the best host.

How-To "Git Tarpipe"

Given you want to copy over all files from blue-branch current HEAD, but only the sub-folder (recursively) into the current working tree, you can make use of git-archive(1) and the tar(1) utilities for that. This is called a tarpipe, which means, the output of git-archive(1) is directly read-in by tar(1) to write the files ("git pipes to tar", in the original tarpipe "tar pipes to tar").

Let's first see a command of that:

git archive --format tar blue-branch sub-folder | tar t -vf -

On the left side of the pipe symbol (|), there is git-archive(1) exporting sub-folder from blue-branch as a tar file (note the --format tar) writing to standard output.

On the right side of the pipe symbol (|), there is tar(1) reading a tar file in from standard input (note the -f -) while testing the tar file (note the t directly following the command-name tar) and being verbose (note the -v, verbose means more information is shown).

With a version of GNU tar it gives an output like this:

$ git archive --format tar blue-branch sub-folder | tar t -vf -
drwxrwxr-x root/root         0 2023-04-18 16:31 sub-folder/
-rw-rw-r-- root/root        45 2023-04-18 16:31 sub-folder/INFO

This is similar to a long directory listing.

Now we could see how we can test a git-archive(1) tarpipe, if we would know how the non-happy paths would look like. But beforehand, let's list the two important positional parameters of git-archive(1) again:

the <tree-ish>, here we're using the branch name blue-branch.
the <path>, which can be zero (just from the project root) or more paths. here we're using one path and that is the sub-folder.

Git tells you if you mix things up here:

Wrong <tree-ish>:

git archive --format tar blue-brnch sub-folder | tar t -vf - 
fatal: not a valid object name: blue-brnch
tar: This does not look like a tar archive
tar: Exiting with failure status due to previous errors

Wrong <path>:

git archive --format tar blue-branch sub-floder | tar t -vf -
fatal: pathspec 'sub-floder' did not match any files
tar: This does not look like a tar archive
tar: Exiting with failure status due to previous errors

Both <tree-ish> and <path> are wrong - you only get the first error, see first error case.

Back to the happy path where the command works:

$ git archive --format tar blue-branch sub-folder | tar t -vf -
drwxrwxr-x root/root         0 2023-04-18 16:31 sub-folder/
-rw-rw-r-- root/root        45 2023-04-18 16:31 sub-folder/INFO

No checkout of the blue-branch is necessary. As long as the local repository has that <tree-ish> with the <path>, git-archive will do.

If we're happy with the result of the test, the single t in tar t ..., we can replace it with the single letter x for extraction.

$ git archive --format tar blue-branch sub-folder | tar x -vf -
sub-folder/
sub-folder/INFO

Thanks to the verbose output (-v), we can see which files have been written. Afterwards the sub-folder is now in place:

$ file sub-folder 
sub-folder: directory
$ cat sub-folder/INFO 
contents of the INFO file in the sub-folder.

Hop, hop, Git, hub, hub aaaand Action

Some notes on how to use that within Microsoft Github Actions:

Obviously you need the git-archive(1) and tar(1) utilities. Never seen them missing on the run: |.
In a PR or push triggering situation, checking out the repository needs to fetch all the .git/refs/* you need/want. That may not only mean full history but fetching the correct name(s), e.g. git fetch origin branch-name:branch-name.
The git-archive(1) command does not fetch anything for you.
You need to create the commit your own, see git-add(1) and git-commit(1).

I tried using your method, but got an error saying my subfolder is a submodule: https://stackoverflow.com/questions/76042072/github-actions-workflow-error-thinks-theres-a-submodule-when-there-isnt — DavidNyan10, Apr 21 '23 at 05:21
if your subfolder is a submodule, then you may want to clone and tar-pipe it individually. but in any case, it looks to me you're starting to tackle with problems on the wrong end addressing them. start building where you are, not where you want to be. so build back first. only do the most important thing and do it right. than slowly add things back, again, doing it right. then you'll reach where you'd like to be. — hakre, Apr 21 '23 at 15:15

GitHub Actions workflow to copy everything (all files and folders) into a folder in another branch

1 Answers1

How-To "Git Tarpipe"

Hop, hop, Git, hub, hub aaaand Action

Linked