Skip to main content

git, nagios and hooks, corrupted git repo

itemprop="text">


Background



We're
using nagios to monitor our infrastructure. We don't have the nagios configs under
version control at the moment, and there are two of us that manage nagios configuration.
As such, I'm working to get our nagios config into a central git repo, using some hooks
to do syntax checking and then if the configs look good, make them "active". I'm using
rel="noreferrer">this guy's post as a starting
point.



The general workflow I'm trying to
implement is:




  1. Edit local
    git repo of nagios config. Add edited files, commit
    locally.

  2. git push origin master
    to the remote repo.

  3. Push is intercepted by the
    pre-receive hook, which takes the files, moves them to a temporary directory on the
    server, and runs them through the nagios syntax
    checker.


  4. If the syntax checker passes, accept
    the push, then use the post-commit hook to git pull the new
    code into the live nagios configuration directory and then restart
    nagios.

  5. If the syntax checker fails, reject the push,
    showing the nagios syntax error to the
    user.



I'm running into an
odd behavior, though, when I reject a git push due to syntax errors in the nagios
config. What I expect to happen is that if I reject the hook, the attempted push should
leave the repository just how it was, untouched. That doesn't appear to be the case,
though. Below are the details of what I'm
seeing:



Problem



I
edit the nagios config locally, intentionally including a syntax error, add, then commit
locally:




host:nagios
erik$ vi nagios.cfg
host:nagios erik$ git add nagios.cfg
host:nagios
erik$ git commit -m "syntax error"
[master da71aed] syntax error
1
files changed, 1 insertions(+), 0
deletions(-)


Now I
push those changes to the master repo. This will be rejected due to the syntax
error:



host:nagios erik$ git push
origin master

Counting objects: 5, done.
Delta
compression using up to 8 threads.
Compressing objects: 100% (2/2),
done.
Writing objects: 100% (3/3), 12.74 KiB, done.
Total 3 (delta
1), reused 2 (delta 1)
remote: Previous HEAD position was 3ddc880... removed
syntax error
remote: HEAD is now at da71aed... syntax error
remote:
Nagios Config Check Exit Status: 254
remote: Your configs did not parse
correctly, there was an error. Output
follows.
remote:

remote: Nagios Core
3.2.3
remote: Copyright (c) 2009-2010 Nagios Core Development Team and
Community Contributors
remote: Copyright (c) 1999-2009 Ethan
Galstad
remote: Last Modified: 10-03-2010
remote: License:
GPL
remote:
remote: Website: http://www.nagios.org
remote:
Reading configuration data...
remote: Error in configuration file
'/tmp/nagiosworkdir/nagios.cfg' - Line 23 (NULL value)
remote: Error
processing main config
file!

remote:
remote:
remote:
remote:
***> One or more problems was encountered while processing the config
files...
remote:
remote: Check your configuration file(s) to ensure
that they contain valid
remote: directives and data defintions. If you are
upgrading from a previous
remote: version of Nagios, you should be aware that
some variables/definitions
remote: may have been removed or modified in this
version. Make sure to read
remote: the HTML documentation regarding the config
files, as well as the

remote: 'Whats New' section to find out what
has changed.
remote:
To
git@remote-server.example.com:nagios
! [remote rejected] master -> master
(pre-receive hook declined)
error: failed to push some refs to
'git@remote-server.example.com:nagios'


This
shouldn't have touched the remote repo, but it did. If I
change to another local temp directory and try to clone the repo, I
get:



host:temp erik$ git clone
git@remote-server.example.com:nagios

Cloning into
nagios...
remote: Counting objects: 30, done.
remote: Compressing
objects: 100% (29/29), done.
remote: Total 30 (delta 12), reused 0 (delta
0)
Receiving objects: 100% (30/30), 29.81 KiB, done.
Resolving
deltas: 100% (12/12), done.
error: Trying to write ref HEAD with nonexistant
object da71aedfde2e0469288acd9e45bb8b57a6e5a7b3
fatal: Cannot update the ref
'HEAD'.



Now
I go back to the original work directory, fix the syntax error, add, commit, and
push:



host:nagios erik$ vi
nagios.cfg
host:nagios erik$ git add nagios.cfg
host:nagios erik$
git commit -m "removing syntax error, push should succeed this time"
[master
f147ded] removing syntax error, push should succeed this time
1 files
changed, 0 insertions(+), 2 deletions(-)
host:nagios erik$ git push origin
master
Counting objects: 6, done.
Delta compression using up to 8
threads.

Compressing objects: 100% (4/4), done.
Writing
objects: 100% (4/4), 487 bytes, done.
Total 4 (delta 2), reused 0 (delta
0)
remote: Previous HEAD position was 4c80d45... syntax
error
remote: HEAD is now at f147ded... removing syntax error, push should
succeed this time
remote: Nagios Config Check Exit Status: 0
remote:
Your configs look good and parsed correctly.
To
git@remote-server.example.com:nagios
3ddc880..f147ded master ->
master



At
this point, the repository is fine, and I'm able to change to a temporary directory and
clone the repo again:



host:temp
erik$ git clone git@remote-server.example.com:nagios
Cloning into
nagios...
remote: Counting objects: 34, done.
remote: Compressing
objects: 100% (33/33), done.
remote: Total 34 (delta 14), reused 0 (delta
0)
Receiving objects: 100% (34/34), 30.22 KiB, done.
Resolving
deltas: 100% (14/14),
done.



Here
is the rel="noreferrer">pre-receive hook I'm
using.



I'm using git v1.7.5.4 on the client, and
v1.7.2.3 on the server.



So, to the
question: why is the repository being left in an
inconsistent state when I reject the push? Is something awry with my git hook or perhaps
my understanding of git is lacking?


class="post-text" itemprop="text">
class="normal">Answer



You're
doing:




export
GIT_WORK_TREE=/tmp/nagiosworkdir
/usr/bin/git checkout -f
$NEW_SHA1


in your
hook. Although it's not touching your usual working-copy it is
updating references in the git-dir (specifically the HEAD
reference), as shown in your
error:



...
remote: HEAD
is now at da71aed... syntax
error
...



Your
hook is doing exit 1 to reject the update, but it's not
(re)resetting the HEAD reference after
failure.



I think you need to update the failure
branch in your hook like
so:



...
if [
"$NAGIOS_CHECK_STATUS" -ne 0 ]
then
echo "Your configs did not
parse correctly, there was an error. Output follows."
cat
$GIT_WORK_TREE/check.out

/usr/bin/git reset --hard $OLD_SHA1 #
<-- Add This
exit 1
else

...

Comments

Popular posts from this blog

linux - iDRAC6 Virtual Media native library cannot be loaded

When attempting to mount Virtual Media on a iDRAC6 IP KVM session I get the following error: I'm using Ubuntu 9.04 and: $ javaws -version Java(TM) Web Start 1.6.0_16 $ uname -a Linux aud22419-linux 2.6.28-15-generic #51-Ubuntu SMP Mon Aug 31 13:39:06 UTC 2009 x86_64 GNU/Linux $ firefox -version Mozilla Firefox 3.0.14, Copyright (c) 1998 - 2009 mozilla.org On Windows + IE it (unsurprisingly) works. I've just gotten off the phone with the Dell tech support and I was told it is known to work on Linux + Firefox, albeit Ubuntu is not supported (by Dell, that is). Has anyone out there managed to mount virtual media in the same scenario?

hp proliant - Smart Array P822 with HBA Mode?

We get an HP DL360 G8 with an Smart Array P822 controller. On that controller will come a HP StorageWorks D2700 . Does anybody know, that it is possible to run the Smart Array P822 in HBA mode? I found only information about the P410i, who can run HBA. If this is not supported, what you think about the LSI 9207-8e controller? Will this fit good in that setup? The Hardware we get is used but all original from HP. The StorageWorks has 25 x 900 GB SAS 10K disks. Because the disks are not new I would like to use only 22 for raid6, and the rest for spare (I need to see if the disk count is optimal or not for zfs). It would be nice if I'm not stick to SAS in future. As OS I would like to install debian stretch with zfs 0.71 as file system and software raid. I have see that hp has an page for debian to. I would like to use hba mode because it is recommend, that zfs know at most as possible about the disk, and I'm independent from the raid controller. For us zfs have many benefits,

apache 2.2 - Server Potentially Compromised -- c99madshell

So, low and behold, a legacy site we've been hosting for a client had a version of FCKEditor that allowed someone to upload the dreaded c99madshell exploit onto our web host. I'm not a big security buff -- frankly I'm just a dev currently responsible for S/A duties due to a loss of personnel. Accordingly, I'd love any help you server-faulters could provide in assessing the damage from the exploit. To give you a bit of information: The file was uploaded into a directory within the webroot, "/_img/fck_uploads/File/". The Apache user and group are restricted such that they can't log in and don't have permissions outside of the directory from which we serve sites. All the files had 770 permissions (user rwx, group rwx, other none) -- something I wanted to fix but was told to hold off on as it wasn't "high priority" (hopefully this changes that). So it seems the hackers could've easily executed the script. Now I wasn't able