8

I am trying to implement CI/CD pipeline using Jenkins , docker and Ansible. I am using SVN code repository for my version control system. For deployment and SVN code repo, I am using AWS EC2. Deployment and code repo is in separate VM.

My Requirement

When I am committing my code into SVN repository , I need to trigger one Jenkins Job. That job will call a ansible playbook.Later it will build project, build Docker image and deploy into EC2. So for any change to my SVN code repository, I need to build Jenkins job.

My Current Attempt

I added the following script in post-commit.tmpl file under $repo/hooks folder.

REPOS="$1"
REV="$2"
UUID=`svnlook uuid $REPOS`
/usr/bin/wget \
  --header "Content-Type:text/plain;charset=UTF-8" \
  --post-data "`svnlook changed --revision $REV $REPOS`" \
  --output-document "-" \
  --timeout=2 \
  http://server/subversion/${UUID}/notifyCommit?rev=$REV

The following is the screenshot

enter image description here

And checked the "Poll SCM option in Jenkins Job":

enter image description here

NB: I am not looking minute/hours/week schedule to pull from repo. Instead of that, I am looking when there is a code change, then I need to build Jenkins project. So I did not add any schedule.

But still I am not getting the latest code in Jenkins. How can I find out the issue related with my configuration?

Updated post-commit.tmpl file

enter image description here

Mr.DevEng
  • 2,651
  • 14
  • 57
  • 115
  • Do you have "Prevent Cross Site Request Forgery exploits" security option enabled? And also have you checked Jenkins log if there is maybe an error logged? – Raoslaw Szamszur Nov 11 '18 at 11:56
  • You can check it under ManageJenkins - > ConfigureGlobalSecurity and look for Prevent Cross Site Request Forgery exploits checkbox. I think from Jenkins 2.x this option is enabled by default. – Raoslaw Szamszur Nov 11 '18 at 12:06
  • This should help https://stackoverflow.com/questions/42487563/github-webhook-url-config-issue-how-to-fix-it/49441937#49441937 – rohit thomas Nov 12 '18 at 02:53
  • @RaoslawSzamszur - Yes , its already checked that option in configure global security. – Mr.DevEng Nov 12 '18 at 06:04
  • @rohitthomas - thank you for response. I will check and read about this link that you given here. – Mr.DevEng Nov 12 '18 at 06:05
  • @rohitthomas - There only mentioning about the Github.I am here using subversion. No options for subversion. Is there anything need to do to hit the svn hook on jenkins side ? I only added the config in /hooks/post-commit.tmpl file and checked the Poll SCM in Jenkins.Is there any separate pluggin need to add? – Mr.DevEng Nov 12 '18 at 06:29
  • @Jacob Ignore the GitHUB keyword and try the scenarios ...2 question that is present in that link: is Jenkins run locally ?? and do you have a proxy ?? – rohit thomas Nov 12 '18 at 06:52
  • `# Every 2 minutes H/2 * * * *` Add this piece in the POLL SCM and do a check in and wait for 2 mins... if it doesn't happen most probably you are running it locally – rohit thomas Nov 12 '18 at 06:53
  • @rohitthomas - Thank you for your response. I checked with every 2 minutes. Its properly working. I cross checked with in every minute also. I am getting that also successfully.And I am not using any proxy. – Mr.DevEng Nov 12 '18 at 14:46
  • @Jacob you will have to look into the Logs(Manage Jenkins-->System logs) Do a commit and observe if a log is captured mostly it wont. If that's the case then you will have to change the Script `post-commit.tmp` – rohit thomas Nov 13 '18 at 02:56
  • @Jacob can you modify the `post-commit.tmp` to do the following `SET REPOS=%1 SET REV=%2 wget http://localhost:8080/jenkins/job/my_project/build` This should trigger the specific job(my_project) . – rohit thomas Nov 13 '18 at 02:59
  • Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/183528/discussion-between-jacob-and-rohit-thomas). – Mr.DevEng Nov 13 '18 at 05:45

4 Answers4

3

Like @bahrep said its hard to troubleshoot issues like this, but my guess is that your post-commit hook doesn't work because of "Prevent Cross Site Request Forgery exploits" Jenkins security option (You've confirmed it's enabled).

From Jenkins Wiki:

If your Jenkins uses the "Prevent Cross Site Request Forgery exploits" security option, the above request will be rejected with 403 errors ("No valid crumb was included"). The crumb needed in this request can be obtained from the URL http://server/crumbIssuer/api/xml (or /api/json). This can be included in the wget call above with something like this:

--header `wget -q --output-document - \
  'http://server/crumbIssuer/api/xml?xpath=concat(//crumbRequestField,":",//crumb)'`

The easiest way to confirm if this security option causes the problem to would be to disable it and try if post-commit hook will work. If yes enable again and try to configure hook with crumb. (In the end, you want to have things secure :) )

And also make sure that Jenkins has enabled anonymous read access:

For this to work, your Jenkins has to allow anonymous read access (specifically, "Job > Read" access) to the system. If access control to your Jenkins is more restrictive, you may need to specify the username and password, depending on how your authentication is configured.

enter image description here

Edit

I think the problems occur because you haven't provided Jenkins instance address. In your webhook example you have:

http://server/subversion/${UUID}/notifyCommit?rev=$REV

You should change server to your Jenkins instance address (Ip, domain or ip and port. It depends on your configuration.).

http://yourjenkins.com/subversion/${UUID}/notifyCommit?rev=$REV

http://<IP>:<Port>/subversion/${UUID}/notifyCommit?rev=$REV

http://<IP>/subversion/${UUID}/notifyCommit?rev=$REV

Or if you run everything locally (including svn repo):

http://localhost:8080/subversion/${UUID}/notifyCommit?rev=$REV

But remember to have:

  • "Prevent Cross Site Request Forgery exploits" security option disabled (You will create webhook to work with this option later, now we want to find the root cause)
  • "allow anonymous read access" security option enabled

I think the hook script works just fine but it's being sent to nowhere. This can be easily checked by logging your hook script. Just add at the end of hook:

echo "`$REPOS` change to revision `$REV` triggered @ `date`" >> ${REPOS}/post-commit-hook.log

and look if after commit log file was created. If yes it means the wget request is being sent incorrectly.

Raoslaw Szamszur
  • 1,723
  • 12
  • 21
  • Thank you for your response. I disabled the "Prevent Cross Site Request Forgery exploits" option and checked. Then also its not working. I think this is not the proper reason (Security related). – Mr.DevEng Nov 12 '18 at 14:48
  • @Jacob probably issue occurs before communicating with Jenkins, but still, this security option would stop your post hook. Now back to debugging, can your Jenkins job poll SCM at all (try entering any schedule value ex.: `H/5 * * * *`) This will tell if the problem is with communication or your post-hook. Also you've mentioned that code is in separate VM, can your EC2 instance talk with this VM at all? (Maybe you forgot to set ALC for this AWS instance) – Raoslaw Szamszur Nov 12 '18 at 17:18
  • Yes sir. I already tried and confirmed this. H/2 * * * * working. And also tried with every minute also. That also working. I am able to see my modified code in jenkins workspace. For every minute and every 2 minute , its working. Problem is related with post-hook. – Mr.DevEng Nov 13 '18 at 05:04
  • @Jacob do you have enabled "Allow anonymous read access" option (I've added a screenshot in edit)? This was also mentioned in Wiki that is necessary in order to post-commit hook to work. – Raoslaw Szamszur Nov 13 '18 at 13:27
  • I checked the option "Allow anonymous read access " that you given like in screenshot. But that also not working. – Mr.DevEng Nov 13 '18 at 13:52
  • @Jacob Recently I was thinking about other possible solutions, and I have found an issue with your post-commit hook. Please look at Edit. I hope this will resolve this :) – Raoslaw Szamszur Nov 15 '18 at 13:44
  • I modified the post-commit.tmpl file according to your edit for adding IP address of Jenkins. But still not building the job when modifying the svn repo. I added the screenshot for updated post-commit file above. Can you please check the screenshot above added with updation . I am also disabled "Prevent Cross Site Request Forgery exploits". And enabled "allow anonymous read access". – Mr.DevEng Nov 26 '18 at 09:16
  • Have you checked my updated screenshot for your edit? – Mr.DevEng Nov 30 '18 at 08:14
1

I made lot attempt to resolve this problem by using guidance from answers. Finally I got the actual issue that I was facing. I added the post-commit script in the file "post-commit.tmpl". This file defaulty I got when I created my SVN repository. Instead of adding the "post-commit.tmpl" need to create file just "post-commit". It resolved my problem.

Mr.DevEng
  • 2,651
  • 14
  • 57
  • 115
0

It's hard to troubleshoot this problem without seeing actual errors and the log. However, one of the possible reasons is that your SVN server requires authentication. You should specify a correct username and password and make sure that this user account has Read permissions to the SVN repository.

bahrep
  • 29,961
  • 12
  • 103
  • 150
  • Thank you for your response. Yes. I already checked the user permission. When I am manually running ansible playbook its checkouting (I added maven module using ansible playbook for checkouting). Also In my local editor also its checkouting. After I added the configuration in hooks, When I am commiting , my jenkins job not building. Within Jenkins I am calling the playbook for CI/CD pipeline inclusing the svn checkout stage. Manually its checkout is working. Only problem is related with Hooks. – Mr.DevEng Nov 11 '18 at 09:17
0

there is an easier way ...by simply defining the schedule for the trigger:

jenkins screenshot

or use trigger builds remotely, if wanting a push instead of a pull solution, which requires posting to https://username:api-token@JENKINS_URL/job/Example/build, with a predefined API token; authenticating scripted clients explains it. building only on changes suggested, because everything else would unnecessarily cost processing power (which equals money).

Martin Zeitler
  • 1
  • 19
  • 155
  • 216