2

At first we thought git shallow will resolve this. However we still see the problem. Its intermittent and occurs for repos without branch and tags too. Errors noticed on Jenkins build during git fetch. Don't see anything obvious in git trace or git debug.

Any ideas on how to resolve this?

22:41:16  stdout: 
22:41:16  stderr: remote: Enumerating objects: 781948, done.        
22:41:16  remote: Counting objects:   0% (1/8931)        
remote: Counting objects:   1% (90/8931)        
remote: Counting objects:   2% (179/8931)        
remote: Counting objects:   3% (268/8931)        
remote: Counting objects:   4% (358/8931)        
remote: Counting objects:   5% (447/8931)        
remote: Counting objects:  96% (8574/8931)        
remote: Counting objects:  97% (8664/8931)        
remote: Counting objects:  98% (8753/8931)        
remote: Counting objects:  99% (8842/8931)        
remote: Counting objects: 100% (8931/8931)        
remote: Counting objects: 100% (8931/8931), done.        
22:41:16  remote: Compressing objects:   0% (1/2514)        
remote: Compressing objects:   1% (26/2514)        
remote: Compressing objects:   2% (51/2514)        
remote: Compressing objects:   3% (76/2514)        
remote: Compressing objects:   4% (101/2514)        
remote: Compressing objects:   5% (126/2514)        
remote: Compressing objects:   6% (151/2514)        
remote: Compressing objects:   7% (176/2514)        
remote: Compressing objects:   8% (202/2514)        
remote: Compressing objects:   9% (227/2514)        
remote: Compressing objects:  10% (252/2514)        
remote: Compressing objects:  11% (277/2514)        
remote: Compressing objects:  12% (302/2514)        
remote: Compressing objects:  13% (327/2514)        
remote: Compressing objects:  14% (352/2514)        
remote: Compressing objects:  15% (378/2514)        
remote: Compressing objects:  71% (1785/2514)        
remote: Compressing objects:  72% (1811/2514)        
remote: Compressing objects:  73% (1836/2514)        
remote: Compressing objects:  74% (1861/2514)        
remote: Compressing objects:  75% (1886/2514)        
remote: Compressing objects:  76% (1911/2514)        
remote: Compressing objects:  77% (1936/2514)        
remote: Compressing objects:  78% (1961/2514)        
remote: Compressing objects:  79% (1987/2514)            
remote: Compressing objects:  97% (2439/2514)        
remote: Compressing objects:  98% (2464/2514)        
remote: Compressing objects:  99% (2489/2514)        
remote: Compressing objects: 100% (2514/2514)        
remote: Compressing objects: 100% (2514/2514), done.        
22:41:16  Receiving objects:   0% (1/781948)   
Receiving objects:   0% (1290/781948), 164.01 KiB | 157.00 KiB/s   
Receiving objects:   0% (2623/781948), 332.01 KiB | 161.00 KiB/s   
Receiving objects:   0% (3914/781948), 420.01 KiB | 161.00 KiB/s   
Receiving objects:  68% (533555/781948), 101.66 MiB | 146.00 KiB/s   
error: index-pack died of signal 15
22:41:16  fatal: index-pack failed
22:41:16    at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2734)
22:41:16    at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:2111)
22:41:16    at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$500(CliGitAPIImpl.java:87)
22:41:16    at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:623)
22:41:16    at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$2.execute(CliGitAPIImpl.java:852)
22:41:16    at
tomerpacific
  • 4,704
  • 13
  • 34
  • 52
dxs123
  • 21
  • 1

1 Answers1

2

Signal 15 is SIGTERM. Something is asking your process to exit gracefully (as opposed to SIGKILL (9), which is more drastic).

You need to figure out where that signal comes from. Maybe you already have a guess based on knowledge of your system, in which case you could confirm by attaching strace to the suspect. Otherwise, https://stackoverflow.com/a/62434615/4811803 may help you find the culprit.

Snild Dolkow
  • 6,669
  • 3
  • 20
  • 32
  • Thanks Snild for response. We do terminate the build in 10 min if not finished, that's limit set at the Jenkins stage. So SIGTERM should be result of that. 10 min however I feel is more then enough for something that usually finished under 1 min. – dxs123 May 31 '23 at 17:17
  • You seem to be downloading at only ~150 KiB/s, while your data is over 100 MiB (your log got to 101.66 MiB before it fails; that's not the _full_ size). Maybe this happens when your network is overloaded or something? A local mirror or git alternate (https://git-scm.com/docs/gitrepository-layout#Documentation/gitrepository-layout.txt-objectsinfoalternates) should help reduce the amount of data you fetch. Another option, if you don't need the full history, is to get a shallow clone with `--depth 1`. – Snild Dolkow Jun 03 '23 at 03:45