Fail to run fetch android while setting up chromium depot_tools

Question

I am following the following link [Install the Chromium depot tools.][1]

[1]: https://commondatastorage.googleapis.com/chrome-infra-docs/flat/depot_tools/docs/html/depot_tools_tutorial.html#_setting_up to fetch android.

As mentioned over the link I successfully clone the depot_tools repo.

git clone https://chromium.googlesource.com/chromium/tools/depot_tools.git
export PATH=/path/to/depot_tools:$PATH

Next I am trying to fetch android using below command

fetch android

Following is the output of above execution

Running: gclient root
WARNING: Your metrics.cfg file was invalid or nonexistent. A new one will be created.
Running: gclient config --spec 'solutions = [
  {
    "name": "src",
    "url": "https://chromium.googlesource.com/chromium/src.git",
    "managed": False,
    "custom_deps": {},
    "custom_vars": {},
  },
]
target_os = ["android"]
'
Running: gclient sync

________ running 'git -c core.deltaBaseCacheLimit=2g clone --no-checkout --progress https://chromium.googlesource.com/chromium/src.git /home/ubuntu/abc/_gclient_src_82brwjmn' in '/home/ubuntu/abc'
Cloning into '/home/ubuntu/abc/_gclient_src_82brwjmn'...

1>WARNING: subprocess '"git" "-c" "core.deltaBaseCacheLimit=2g" "clone" "--no-checkout" "--progress" "https://chromium.googlesource.com/chromium/src.git" "/home/ubuntu/abc/_gclient_src_82brwjmn"' in /home/ubuntu/abc failed; will retry after a short nap...

________ running 'git -c core.deltaBaseCacheLimit=2g clone --no-checkout --progress https://chromium.googlesource.com/chromium/src.git /home/ubuntu/abc/_gclient_src_82brwjmn' in '/home/ubuntu/abc' attempt 2 / 4
Cloning into '/home/ubuntu/abc/_gclient_src_82brwjmn'...


1>WARNING: subprocess '"git" "-c" "core.deltaBaseCacheLimit=2g" "clone" "--no-checkout" "--progress" "https://chromium.googlesource.com/chromium/src.git" "/home/ubuntu/abc/_gclient_src_82brwjmn"' in /home/ubuntu/abc failed; will retry after a short nap...

________ running 'git -c core.deltaBaseCacheLimit=2g clone --no-checkout --progress https://chromium.googlesource.com/chromium/src.git /home/ubuntu/abc/_gclient_src_82brwjmn' in '/home/ubuntu/abc' attempt 3 / 4
Cloning into '/home/ubuntu/abc/_gclient_src_82brwjmn'...
fatal: Malformed response in ref list: c3cf127a33b227abcf63

1>WARNING: subprocess '"git" "-c" "core.deltaBaseCacheLimit=2g" "clone" "--no-checkout" "--progress" "https://chromium.googlesource.com/chromium/src.git" "/home/ubuntu/abc/_gclient_src_82brwjmn"' in /home/ubuntu/abc failed; will retry after a short nap...

________ running 'git -c core.deltaBaseCacheLimit=2g clone --no-checkout --progress https://chromium.googlesource.com/chromium/src.git /home/ubuntu/abc/_gclient_src_82brwjmn' in '/home/ubuntu/abc' attempt 4 / 4
Cloning into '/home/ubuntu/abc/_gclient_src_82brwjmn'...
1>WARNING: subprocess '"git" "-c" "core.deltaBaseCacheLimit=2g" "clone" "--no-checkout" "--progress" "https://chromium.googlesource.com/chromium/src.git" "/home/ubuntu/abc/_gclient_src_82brwjmn"' in /home/ubuntu/abc failed; will retry after a short nap...
Syncing projects: 100% ( 1/ 1) src
src (ERROR)
----------------------------------------
[0:00:00] Started.
[0:00:00] 
Traceback (most recent call last):
  File "/home/ubuntu/depot_tools/gclient_scm.py", line 1043, in _Clone
    self._Run(clone_cmd, options, cwd=self._root_dir, retry=True,
  File "/home/ubuntu/depot_tools/gclient_scm.py", line 1411, in _Run
    gclient_utils.CheckCallAndFilter(cmd, env=env, **kwargs)
  File "/home/ubuntu/depot_tools/gclient_utils.py", line 673, in CheckCallAndFilter
    raise subprocess2.CalledProcessError(
subprocess2.CalledProcessError: Command 'git -c core.deltaBaseCacheLimit=2g clone --no-checkout --progress https://chromium.googlesource.com/chromium/src.git /home/ubuntu/abc/_gclient_src_82brwjmn' returned non-zero exit status 128 in /home/ubuntu/abc
[0:25:41] _____ Conflicting directory found in /home/ubuntu/abc/src. Moving to /home/ubuntu/abc/_bad_scm/srcml99933n.
----------------------------------------


Warnings:
Conflicting directory /home/ubuntu/abc/src moved to /home/ubuntu/abc/_bad_scm/srcml99933n.
Traceback (most recent call last):
  File "/home/ubuntu/depot_tools/gclient_scm.py", line 546, in update
    self._Clone(revision, url, options)
  File "/home/ubuntu/depot_tools/gclient_scm.py", line 1043, in _Clone
    self._Run(clone_cmd, options, cwd=self._root_dir, retry=True,
  File "/home/ubuntu/depot_tools/gclient_scm.py", line 1411, in _Run
    gclient_utils.CheckCallAndFilter(cmd, env=env, **kwargs)
  File "/home/ubuntu/depot_tools/gclient_utils.py", line 673, in CheckCallAndFilter
    raise subprocess2.CalledProcessError(
subprocess2.CalledProcessError: Command 'git -c core.deltaBaseCacheLimit=2g clone --no-checkout --progress https://chromium.googlesource.com/chromium/src.git /home/ubuntu/abc/_gclient_src_82brwjmn' returned non-zero exit status 128 in /home/ubuntu/abc

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ubuntu/.vpython-root/04a1a7/lib/python3.8/shutil.py", line 780, in move
    os.rename(src, real_dst)
FileNotFoundError: [Errno 2] No such file or directory: '/home/ubuntu/abc/src' -> '/home/ubuntu/abc/_bad_scm/srcml99933n/src'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ubuntu/depot_tools/metrics.py", line 267, in print_notice_and_exit
    yield
  File "/home/ubuntu/depot_tools/gclient.py", line 3195, in <module>
    sys.exit(main(sys.argv[1:]))
  File "/home/ubuntu/depot_tools/gclient.py", line 3181, in main
    return dispatcher.execute(OptionParser(), argv)
  File "/home/ubuntu/depot_tools/subcommand.py", line 252, in execute
    return command(parser, args[1:])
  File "/home/ubuntu/depot_tools/gclient.py", line 2737, in CMDsync
    ret = client.RunOnDeps('update', args)
  File "/home/ubuntu/depot_tools/gclient.py", line 1779, in RunOnDeps
    work_queue.flush(revision_overrides, command, args, options=self._options,
  File "/home/ubuntu/depot_tools/gclient_utils.py", line 968, in flush
    reraise(e[0], e[1], e[2])
  File "/home/ubuntu/depot_tools/gclient_utils.py", line 67, in reraise
    raise value
  File "/home/ubuntu/depot_tools/gclient_utils.py", line 1045, in run
    self.item.run(*self.args, **self.kwargs)
  File "/home/ubuntu/depot_tools/gclient.py", line 925, in run
    self._got_revision = self._used_scm.RunCommand(command, options, args,
  File "/home/ubuntu/depot_tools/gclient_scm.py", line 132, in RunCommand
    return getattr(self, command)(options, args, file_list)
  File "/home/ubuntu/depot_tools/gclient_scm.py", line 548, in update
    self._DeleteOrMove(options.force)
  File "/home/ubuntu/depot_tools/gclient_scm.py", line 207, in _DeleteOrMove
    shutil.move(self.checkout_path, dest_path)
  File "/home/ubuntu/.vpython-root/04a1a7/lib/python3.8/shutil.py", line 794, in move
    copy_function(src, real_dst)
  File "/home/ubuntu/.vpython-root/04a1a7/lib/python3.8/shutil.py", line 426, in copy2
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/home/ubuntu/.vpython-root/04a1a7/lib/python3.8/shutil.py", line 259, in copyfile
    with open(src, 'rb') as fsrc, open(dst, 'wb') as fdst:
FileNotFoundError: [Errno 2] No such file or directory: '/home/ubuntu/abc/src'

Subprocess failed with return code 1.

I have been stuck at this.

Appreaciate any leads here

Yes, that I am aware of. If you looks at the error logs at the start then you would find that,its not able to create /src folder. That's the starting point of the issue. raise subprocess2.CalledProcessError( subprocess2.CalledProcessError: Command 'git -c core.deltaBaseCacheLimit=2g clone --no-checkout --progress https://chromium.googlesource.com/chromium/src.git /home/ubuntu/abc/_gclient_src_82brwjmn' returned non-zero exit status 128 in Conflicting directory found in /home/ubuntu/abc/src. Moving to /home/ubuntu/abc/_bad_scm/srcml99933n. — user3214392, Sep 30 '20 at 15:24
You should run that fetch command in a directory where you have write access. That's where will create a src folder where all the code will reside — Asesh, Sep 30 '20 at 16:18
Yes I am doing that. I created a new directory "abc" and ran the fetch command inside it — user3214392, Oct 01 '20 at 04:37
Have run into the same issue, fetching without history helped to get at least past the error: `fetch --no-history chromium` Not sure what caused this. — monkeber, Mar 30 '21 at 09:41
After I fetched without history I had troubles with building because LASTCOMMIT files contained zeroes. Instead I found some tweaks to git config: http.postbuffer=1000M http.maxrequestbuffer=1000M pack.deltacachesize=2047m pack.packsizelimit=2047m pack.windowmemory=2047m After setting above values I had another issue with fetch but I could run `gclient sync` in the same directory without clearing all fetched files and finally everything was ok and the build was successful. — monkeber, Mar 31 '21 at 14:17

score 1 · Answer 1 · edited Dec 27 '21 at 18:24

1

You needed make sure Memory large enough. https://chromium.googlesource.com/chromium/src/+/refs/heads/main/docs/android_build_instructions.md#system-requirements

I had the same question and adding memory solved it.

edited Dec 27 '21 at 18:24

answered Dec 20 '21 at 05:03

huangqingping

11
1

Prakhar Shrivastav · Answer 2 · 2022-11-18T12:36:56.553

I too encountered this issue. This seems bad combination of Ubuntu 20 and Chromium.

This is because of very large commit histories of chromium it required too much size and ubuntu 20 doesn't support this much.

So to get android chromium in ubuntu20 you need to follow the below steps:

fetch --nohooks --no-history android

gclient sync -D --no-history

once this is done trigger your build command. Without patch history git pack size will not exceed 2GB limit and it will work fine same as older systems.

Fail to run fetch android while setting up chromium depot_tools

2 Answers2