How to prevent audit logs appearing in /var/log/messages – RHEL7

Overview

For RHEL7.

This took more work than I anticipated.

My goal was this:

  1. Write audit logs to /var/log/audit
  2. Forward audit and syslog to central logging server
  3. Audit logs to NOT appear in /var/log/messages

The items 1 and 2 were easy, number 3, not so much.

audisp, rsyslogd and journald

After using rsyslog to send logs onto a central log collector, I noticed that all the audit logs are still going to ”/var/log/messages” as well as ”/var/log/audit”, not ideal, the only solution that worked, out of the many I tested, is detailed below, the audit logs still go to /var/log/audit (separate LV of course), /var/log/messages doesn’t fill up with audit logs, and all the logs get forwarded to a central log collector. The following configs below seem to work in general, but you may have to tweak them for your application, specifically the rate and burst limiting for journald.

Files of interest are:

  • /etc/audisp/plugins.d/syslog.conf – the plugin that forwards audit logs.
  • /etc/systemd/journald.conf – journald daemon config, change default values for rate-limiting otherwise message maybe be dropped.
  • /etc/rsyslog.conf – configure forwarding and exclude local0
  • /etc/auditd/rules.d/audit.rules – I needed exclude ”/var/lib/rsyslog” to stop logging loop, you may not have to do this.

Configure syslog plugin /etc/audisp/plugins.d/syslog.conf to be like the below

active = yes
direction = out
path = builtin_syslog
type = builtin 
#args = LOG_INFO
args = LOG_LOCAL0
format = string

Modify rsyslog.conf, the key change was adding the local0.none

.info;mail.none;authpriv.none;cron.none;local0.none      /var/log/messages

# Forward logs using TCP to central log collector

 *.* @@xxx.xxx.xxx.xxx:10514

And now journald.conf, the default is ”RateLimitInterval=30s” and ”RateLimitBurst=1000”, I had to change this as I was seeing dropped journalctl messages – ”journal: Suppressed 10361 messages from”. I had to exclude ”/var/lib/rsyslog” in audit.rules as it was creating thousands of messages, i.e. a logging loop. It is possible that rsyslog can drop messages as well, they will show up as ”imjournal: begin to drop messages due to rate-limiting”, to rate-limit rsyslog you make changes the rsyslog.conf file.

 RateLimitInterval=15s
 RateLimitBurst=3000

If you need to rate-limit rsyslog to can add the following, however, I’ve not tested this.

Old style syslog format

# File to store the position in the journal
$IMJournalStateFile imjournal.state
$imjournalRatelimitInterval 300
$imjournalRatelimitBurst 30000

Or if you want the new style, all one line.

module(load="imjournal" StateFile="/var/lib/rsyslog/imjournal.state" ratelimit.interval="300" ratelimit.burst="30000")

Testing rate-limiting

Tail your log file

tail -f /var/log/messages

From a second shell

journalctl -f

From a third shell generate some traffic

seq 1 3000 | logger
Advertisements
Posted in Linux & Solaris | Leave a comment

Adding a custom unit file RHEL7, for swatch

swatch is a Linux tool for watching (it uses ”tail -f”) log files and then demonizes it.

I had a need to know when a host needed rebooting after yum-cron had updated the kernel, rather than logging into each host I decided I could make use of ”swatch”, in my case, it watches the ”yum.conf” file for the string ”Installed: kernel”, if the string appears in the log it sends an email out.

I have not found away for a single daemon instance to watch more than a single log file, there are some examples online showing this configuration, but when I tested it only ever worked on a single file being watched. After further reading and research this is the expected behavior.

Installing swatch, configuring start-up to watch a single log source

  • First install it

yum install swatch

  • Next create a swatch directory

mkdir -p /etc/swatch

  • Create a ”swatch” conf file for what you want to monitor

vi swatch-yum.conf
watchfor /Installed: kernel/
echo bold
mail=root@localhost, subject=”New kernel has been installed – reboot required”

  • Now create a ”.swatchrc file in ”/root”

touch /root/.swatchrc

  • You can now test the configuration, you might want to change it to something that is easy to test and validate, ssh logins or something.

swatch –config-file=/etc/swatch/swatch-yum.conf –tail-file=/var/log/yum.log –daemon

  • Check it started

ps -ef | grep swatch

root 1187 1 0 12:59 ? 00:00:00 /usr/bin/swatch –config-file=/etc/swatch/swatch-yum.conf –tail-file=/var/log/yum.log –pid-file=/var/run/sw

  • Ok, so we know it works, so go ahead and kill it to stop it as we need to create a custom unit file (service) so it can start at boot.

kill -15 1187

  • Because this is systemd you do not need to touch init scripts or anything, but you do need to create a custom unit file, cd to ”/etc/systemd/system”
  • Next, create your unit file using an editor

vi swatch.service
[Unit]
Description=Swatch Log Monitoring Daemon
After=syslog.target network.target auditd.service sshd.service

[Service]
ExecStart=/usr/bin/swatch –config-file=/etc/swatch/swatch-yum.conf –tail-file=/var/log/yum.log –pid-file=/var/run/swatch.pid –daemon
ExecStop=/usr/bin/kill -s KILL $(cat /var/run/swatch.pid)
Type=forking
PIDFile=/var/run/swatch.pid

[Install]
WantedBy=multi-user.target

  • Now reload it, start it and enabled and check its status.

systemctl daemon-reload
systemctl start swatch.service
systemctl enable swatch.service
systemctl status swatch.service

  • If all has gone well it should look like this
  • systemctl status swatch.service
    ● swatch.service – Swatch Log Monitoring Daemon
    Loaded: loaded (/etc/systemd/system/swatch.service; enabled; vendor preset: disabled)
    Active: active (running) since Wed 2017-10-11 12:59:05 CDT; 39min ago
    Main PID: 1187 (/usr/bin/swatch)
    CGroup: /system.slice/swatch.service
    ├─1187 /usr/bin/swatch –config-file=/etc/swatch/swatch-yum.conf –tail-file=/var/log/yum.log –pid-file=/var/run/sw
    └─1188 /usr/bin/tail -n 0 -F /var/log/yum.log
    Oct 11 12:59:03 rhel7-.local systemd[1]: Starting Swatch Log Monitoring Daemon…
    Oct 11 12:59:05 rhel7-.local systemd[1]: Started Swatch Log Monitoring Daemon.

Configuring swatch to monitor more than a single log file source

This is pretty much the same as the above, but I split things up a little, it makes it easier to manage. I’ve assumed you’ve done the initial install, also note that the file paths have changed in the service start-up files to reflect having to run more than a single instance of swatch.

  • Create your swatch.conf files, in this case I’m going watch /var/log/yum.log for kernel updates and /var/log/secure for invalid user and failed password attempts.

vi swatch-yum.conf
watchfor /Installed: kernel/
echo bold
mail=root@localhost, subject=”New kernel has been installed – reboot required”

vi swatch-secure.conf
watchfor /Invalid user/
echo bold
mail=root@localhost, subject=”Invalid user”

watchfor /Failed password/
echo bold
mail=root@localhost, subject=”Failed password”

  • Next you need to create 2 custom unit file (service), one for each file we want to monitor. Note that we have to created two separate PID files now.

vi swatch-yum.service
[Unit]
Description=Swatch Log Monitoring Daemon
After=syslog.target network.target auditd.service sshd.service

[Service]
ExecStart=/usr/bin/swatch –config-file=/etc/swatch/swatch-yum.conf –tail-file=/var/log/yum.log –pid-file=/var/run/swatch-yum.pid –daemon
ExecStop=/usr/bin/kill -s KILL $(cat /var/run/swatch-yum.pid)
Type=forking
PIDFile=/var/run/swatch-yum.pid

[Install]
WantedBy=multi-user.target

And..

vi swatch-secure.service
[Unit]
Description=Swatch Log Monitoring Daemon
After=syslog.target network.target auditd.service sshd.service

[Service]
ExecStart=/usr/bin/swatch –config-file=/etc/swatch/swatch-secure.conf –tail-file=/var/log/yum.log –pid-file=/var/run/swatch-secure.pid –daemon
ExecStop=/usr/bin/kill -s KILL $(cat /var/run/swatch-secure.pid)
Type=forking
PIDFile=/var/run/swatch-secure.pid

[Install]
WantedBy=multi-user.target

  • Now reload it, start it and enabled and check its status.

systemctl daemon-reload
systemctl start swatch-yum
systemctl enable swatch-yum
systemctl start swatch-secure
systemctl enable swatch-secure

You can test it by echoing the watch text to the log

echo Failed password >> /var/log/secure

Posted in Linux & Solaris | Leave a comment

2008 Kia Sedona – A/C stops working.

2008 Kia Sedona LX, A/C was working great, we stop, go into a store, come out and now the A/C does not work. Blowers still blow, and there have been no unusual noises from the engine compartment.

The 7.5A fuse was blown, so replaced, and on turning the A/C on it blows again.

Here is how I diagnosed and fixed it.

A/C was working fine
No mechanical noises
A/C suddenly ceases to work
A/C 7.5 fuse was blown
A/C fuse replaced, immediately blows the fuse
A/C relay checks out good
Removed A/C relay and measured from pin/socket 87 to ground, reads open-circuit
Removed old field coil, it measures between 1.4 and 1.7 ohms
New coil measured between 3.4 and 3.5 ohms
Disconnect battery
Removed under tray/splash shielding from underneath minivan
Removed pulley wheel from A/C compressor
Install new field coil
Re-assemble pulley etc
A/C now works

The correct part is Kia K97641-4D900 A/C Field Coil, cost $53.50 + tax from a Kia main dealer.

Posted in Vehicle maintenance | Leave a comment

Adding vmware-tools to Centos5 or RHEL

Adding vmware-tools to RHEL6 or greater is easy, assuming you have the EPEL repo enabled all you do is:

yum install open-vm-tools.x86_64

That’s it your done.

Things are a tad more complicated for lower version of RHEL, for example RHEL5, some of this is down to the way VMWare have changed the way they support vmware-tools on Linux, they scrapped having the vm-tools.rpm the linux.iso which you use to push out from the vSpehre client,  they now advise you to use your disto’s repos.

Here’s how you do for RHEL5.

Check your version/architecture etc

[root@foo ~]# cat /etc/redhat-release 
 CentOS release 5.6 (Final)
 [root@foo ~]# uname -a
 Linux foo.local 2.6.18-308.20.1.el5 #1 SMP Tue Nov 13 10:15:12 EST 2012 x86_64 x86_64 x86_64 GNU/Linux

Create repo file

touch /etc/yum.repos.d/vmware-tools.repo
Add this and adjust for your release/architecture, tip, DO NOT USE the ''latest'' repo, it has caused issues, always go for a point/named release

 [vmware-tools]
 name=VMware Tools
 #baseurl=http://packages.vmware.com/tools/esx/5.1latest/rhel6/x86_64 # DO NOT USE
 #baseurl=http://packages.vmware.com/tools/esx/5.1latest/rhel6/i386
 #baseurl=http://packages.vmware.com/tools/esx/5.1latest/rhel5/i386
 #baseurl=http://packages.vmware.com/tools/esx/5.1latest/rhel5/x86_64
 baseurl=http://packages.vmware.com/tools/esx/5.5u2/rhel5/x86_64   <====== THIS WORKS, NOTE THE PATH
 #baseurl=http://packages.vmware.com/tools/esx/4.0latest/rhel6/x86_64
 #baseurl=http://packages.vmware.com/tools/esx/4.0latest/rhel5/i686
 #baseurl=http://packages.vmware.com/tools/esx/4.0latest/rhel6/i686
 enabled=1
 gpgcheck=1

Download the keys, you may have to export proxy settings

 wget http://packages.vmware.com/tools/keys/VMWARE-PACKAGING-GPG-DSA-KEY.pub
 wget http://packages.vmware.com/tools/keys/VMWARE-PACKAGING-GPG-RSA-KEY.pub

Import the keys

 rpm --import ./VMWARE-PACKAGING-GPG-RSA-KEY.pub
 rpm --import ./VMWARE-PACKAGING-GPG-DSA-KEY.pub

Now you are ready to install – IMPORTANT, install ”vmware-tools-esx-kmods.x86_64” first
yum install vmware-tools-esx-kmods.x86_64 – and it can take some time.

 yum install vmware-tools-esx-nox.x86_64

”vmware-tools-esx-nox.x86_64” is for the non GUI version or headless, which is what we want for us as we do not run GNOME etc on our servers.

Amongst other things, having vmware-tools installed allows the full potential of the vmxnet3 NIC to be exploited, and allows you to shutdown the guest from with the vSphere client.

Posted in Linux & Solaris | Tagged | 4 Comments

2001 GE Spacemaster XL1800 over-stove microwave oven repair

Microwave went faulty a few days ago, with the following symptoms, play close attention to the symptoms, as this was leading me to think it was a cap going out, and nothing that serious as the oven was still heating. There are quite a few articles out there on the web where people have replaced transformers, megatron’s, controller boards etc (each costing anywhere from $25 – $100 each), when all they needed to replace was 10 cent cap.

  • Garbled buzzing/beeping sound when the door opened, or any cooking function was selected
  • The buzzing/beeping sound changes when any of the buttons are pressed, it seemed unwanted electrical signals were getting to the piezo speaker/buzzer
  • The oven light flickered in time with the buzzing sound
  • Turntable not turning – I recently replaced the turntable motor so I knew it was ok
  • It would still heat food even though it was doing all of the above

So, disconnect power, then take the oven off the wall, remove the outer case, then remove the circuit board that hosts the display and control panel.

A guy on appliancepartspros.com forum said cap C1 50v 220uF goes out, so I take a look at that one and the others nearby, they all look ok.  I decided to remove C1 and the other 3 that make up a block of 4, C2 50v 47uF, C3 16v 470uF and C4 2200uF 16v. I tested C1 with my multimeter and sure enough it’s dud, just for good measure I also replaced the others in that block of 4, even though they tested ok.

I then decided to test the other caps next to this block, this column of caps are all the same physical size, unlike the first block of 4 which have various sizes – they are C5 100uF 10v, C6 22uF 16v, C14 22uF 16v, C15 4.7uF 50v. Glad I did, as found that C5 was also dud.

Resistor R51 39k ohm also looked as if it had got hot, which is possible as it works along with the relay which has been chattering away, so I replaced that as well, it did test ok though.

So, to summarized, replace 2 faulty caps and a resistor

C1 50v 220uF

C5 100uF 10v replaced with 100uF 16v, remember it’s usually ok to replace with a higher voltage cap, never lower though.

R51 39k 1% resistor

I leave up to you to decide if you want to replace the other caps that test ok.

oven-pcb-small

Posted in Electronics & Gadgets | Tagged , , | Leave a comment

Foscam FI8910W compared to FI8918W

I already own several Foscam FI8918Ws, so I have some experience with Foscam, I had it up and running within a few minutes, assigned a static IP using a wired ethernet connection, then set-up the wireless.

I use all of my cameras with BlueIris, running on a XP VM running on VMware ESXi, this has worked well for around 3 years, with hardly an issue. This may have an impact, seeing as I use BlueIris, I let this control all of the motion detection, sending alerts and the such like, consequently I turn just about everything off in the camera firmwware, so this must present less load on the CPU.

Jan 2014 – latest firmware and webGUI firmware (I updated it)

FI8910W – the good

Much better image quality when compared to the FI8918W (comes at a price though, more on that later), you can actually discern colors with this!

IR cut filter really works, you can see much more at night with this camera, the filter does make an audible click though, if you are in a quiet area, this may alert somebody that you have a camera.

The power supply has a very long cord, which is nice.

Seems to be rock solid over wired ethernet.

The bad.

Through out the house, I have several wireless routers, using Tomato USB and DD-WRT, every single wireless device we own have no problems working with these, we are talking multiple smartphones, tablets, XBOX360, PCs here, no isssue…and you guessed it, except for the FI8910W.

I set it up where the FI8918W was located, I initially got it to connect, and then the video would drop out, the signal strength was around -62dbm. I ran a constant ping, it would ping for about 12 seconds or so, then time out for some amount of time, then connect again (watch dog probably kicking in), then drop out and stay dropped out, I tried all sorts of settings and nothing worked, dropped it down 320×240, tried a 12db antenna, all to no avail.

So, knowing that it might struggle to work on a slightly lower than normal signal I decided to relocate it, this time it was around 30ft away, line of sight, only thing between it and the WiFi router was a single glass door, I had high hopes this time, -55dbm signal (some devices in my house work well on >-70dbm), nope still didn’t work, BlueIris reports 2.5fps coming in at 250k/Bs, webGUI was slightly more responsive though, but that was about it, still no video.

Other users have gone into detail regarding the expected performance of this camera over WiFi, it seems it needs a steady 750k/Bs to work well, and from what I see, this seems to be true.

It is of my opinion that the FI8910W is border line usable over WiFi, and its radio performance is weak – if you have a wired connection then you should be good, but to work reliably over WiFi you will need a very strong signal.

I’m now using the camera over wired ethernet and it has been rock solid.

The camera did work over WiFi when it was around 15ft away from the router and in the same room, not really much use for me, and I suspect many others.

I have just received a D-Link DCS-930L, doesn’t have pan/tilt, but is good value, colors are okay, image is just ok, not very detailed but usable, and in the same location as I tried the Foscam, this is giving ~14fps and 450k/Bs.

Also, I’m suspicious of all the “Foscam support is so good” reviews I’m seeing all the time, every time a negative review pops up, you can bet you bottom dollar that a “Foscam support is so good..” review will soon follow.

Posted in Home surveillance | Tagged | Leave a comment

GPS, Garmin 850 touchscreen goes crazy

I have a Garmin 850 GPS (with European maps installed) which has served me well for around 4 years, the only issue being that the device that mounts it to the windshield, the rubber suction cup, well, loses suction and it falls off from time to time, and for whatever reason, it does this more so in my new 2103 Sonata (angle of the windshield maybe?).

It has survived these falls without any noticeable damaged, until last week that is, this time the lower left hand corner of the touchscreen was failing to work, tried turning on and off etc, all to no avail, I was residing to the fact that I might have to get a new one, then it dawned on me that I could probably still get by, by using the voice recognition, but this was not ideal, I then tried a ‘pre-boot reset’ and recalibration and it is now working fine again!

This is what I did

  1. Turn the GPS off
  2. Remove battery
  3. Press and hold the touchscreen then put the battery back in
  4. The unit will power on and go into pre-boot mode. Continue holding the screen until pre-boot goes away and the progress bar appears then release the screen, this can take around 1 minute.
  5. Now the calibration will start, two intersecting lines should display on the screen along with the message: Press Dot
  6. Follow the rest of the on screen prompts to complete the screen recalibration

That’s it.

Posted in Electronics & Gadgets | Tagged | Leave a comment