84

When I run my job on Gitlab CI/CD, after a while I obtain the following error message:

Job's log exceeded limit of 4194304 bytes.

How to change this limit?

Ortomala Lokni
  • 56,620
  • 24
  • 188
  • 240

3 Answers3

106

To change the build log size of your jobs in Gitlab CI/CD, you can edit your config.toml file and add a new limit in kilobytes:

[[runners]]
  output_limit = 10000

According to the documentation

output_limit : Maximum build log size in kilobytes. Default is 4096 (4MB).

For this to take effect, you need to restart the gitlab runner:

sudo gitlab-runner restart
Ortomala Lokni
  • 56,620
  • 24
  • 188
  • 240
  • 3
    I did but the changes did not reflect. I still got the same error message – CanCoder Oct 27 '20 at 02:45
  • 4
    For me also this change was not showing any effect. I found that the gitlab-runner needs to be restarted, Now its working. Command to restart gitlb-runner --> "sudo gitlab-runner restart" – Ashish Pratap Jan 08 '21 at 01:38
  • 1
    Even after restarting the runner, when I manually retried the relevant job in my pipeline I saw the same message for the default limit. And even after manually starting the entire pipeline, I still saw the default limit message. – Kenny Evitt Apr 27 '21 at 17:57
  • What seemed to finally work was to move the line with `output_limit` up above the sub-sections starting with the line containing `[runners.custom_build_dir]`. Maybe this is due to something with parsing the TOML file or specific to TOML syntax? – Kenny Evitt Apr 27 '21 at 18:05
  • 1
    The [latest docs](https://docs.gitlab.com/runner/configuration/advanced-configuration.html) also seem to claim that you shouldn't in fact need to restart the runner for this change. – Kenny Evitt Apr 27 '21 at 18:06
  • Just putting on some extra information. I had to switch to the root user using `sudo su` in order to access the gitlab runner setting inside `/etc/gitlab-runner` folder. – Reaz Murshed Jul 23 '22 at 22:00
19

So an answer for those that don't have access to the gitlab-runners configuration file that @Ortomala Lokni refers too.

You can redirect the logger output and archive it quite easily by doing the following (Note: this is done for maven builds).

quality-check:
    extends: .retry-on-job-failure
    stage: quality-check
    timeout: 2 hours
    artifacts:
        name: "$CI_BUILD"
        paths:
            - target/client-quality_check.log
        when: always
        expire_in: 3 days
    only:
        - main
        - merge_requests
    script:
        - echo "Sonar Qube Start"
        - mvn MAVEN_CLI_OPTS sonar:sonar --log-file target/client-quality_check.log \-Dsonar.projectKey=$PROJECT_ KEY \-Dsonar.host.url=$SONAR_HOST_URL \-Dsonar.login=$SONAR_TOKEN 
        - echo "Sonar Qube Complete"

Notice within the maven command I am using the --log-file to redirect the maven output to target/client-quality_check.log and then under artifacts I have set to archive this log file by providing the path to the file.

Once this Job finishes I can then look at the Jobs archives and can see my log file with all the logger output in it.

Popeye
  • 11,839
  • 9
  • 58
  • 91
5

Starting from gitlab 14.1 there is another configuration option that effects maximum log size: ci_jobs_trace_size_limit (100MB by default). So altering only runner limit, as described in other answers, is not sufficient anymore.

Since gitlab is all about speed and usability, modifying ci_jobs_trace_size_limit is possible only by executing command directly in rails console of the system (or docker container) where gitlab is running.

root@192:/# gitlab-rails console -e production
--------------------------------------------------------------------------------
 Ruby:         ruby 2.7.5p203 (2021-11-24 revision f69aeb8314) [x86_64-linux]
 GitLab:       14.8.2 (c7be43f6dd3) FOSS
 GitLab Shell: 13.23.2
 PostgreSQL:   12.7
-----------------------------------------------------------[ booted in 122.49s ]
Loading production environment (Rails 6.1.4.6)
irb(main):001:0> Plan.default.actual_limits.update!(ci_jobs_trace_size_limit: 100000000)
=> true
irb(main):002:0> quit

Note: if it seems like gitlab-rails console -e production doesn't do anything and console prompt doesn't pop up you'll need to wait.

user7860670
  • 35,849
  • 4
  • 58
  • 84