551-600 of 8120 results (7ms)
2010-07-14 §
12:19 <mark> Fixed ganglia and puppet on stafford [production]
11:54 <mark> Migrated DNS monitoring to puppet [production]
10:31 <mark> Migrated ZFS RAID nagios check to puppet [production]
10:14 <mark> Migrated monitoring of lucene to puppet [production]
09:37 <mark> Migrated monitoring of image scalers to puppet [production]
08:49 <Tim> using stafford for some pbuilder experimentation [production]
2010-07-13 §
22:02 <mark> Migrated monitoring of application servers to Puppet [production]
20:29 <mark> Fixed puppet on ms4 [production]
20:16 <mark> Hacked up nagios conf.php to not create host entries for most servers (now in puppet), except special cases [production]
19:58 <mark> Hacked up nagios conf.php to not create host entries [production]
16:51 <mark> Migrated Squid Nagios monitoring to puppet, commented some functionality in nagios conf.php [production]
15:51 <mark> Split puppet nagios config over multiple files [production]
2010-07-12 §
16:54 <Fred> changed LONGQUERIES check threshold [production]
16:08 <Fred> restarting morebots since it had died. [production]
16:08 <Fred> restarting Nagios since it was down. [production]
14:29 <mark> Added "cfg_file=/etc/nagios/puppet_hosts.cfg" to nagios.cfg [production]
13:25 <JeLuF> added disk space monitoring for apaches [production]
12:51 <jeluf> synchronized php-1.5/wmf-config/InitialiseSettings.php '24306 - Create namespaces for Lithuanian Wiktionary' [production]
12:48 <jeluf> synchronized php-1.5/wmf-config/InitialiseSettings.php '24321 - ml.wikiquote.org lost its project namespace' [production]
12:46 <jeluf> synchronized php-1.5/wmf-config/InitialiseSettings.php '24321 - ml.wikiquote.org lost its project namespace' [production]
12:41 <jeluf> synchronized php-1.5/wmf-config/InitialiseSettings.php '24344 - Namespace changes - si.wiktionary' [production]
11:45 <JeLuF> fixed broken ganglia-metrics installation on srv146 (chown gmetric /var/log/gmetricd/gmetricd.log) [production]
11:41 <JeLuF> added DPKG status monitoring for all app servers to nagios. Reports all packages that are not in state 'rc' or 'ii'. [production]
10:43 <JeLuF> lots of false alerts from nagios due to missing SSL setup for NRPE. Working on it. [production]
09:53 <JeLuF> changed puppet config to install nrpe on all app servers [production]
09:28 <JeLuF> replacing opsview-nrpe agents by nagios-nrpe agents (image_scalers, some other apaches). Most apaches already use nagios-nrpe [production]
07:40 <Tim> set up NRPE disk space monitoring on ms4, discovered that /mnt2 is full [production]
04:54 <Tim> updated NFS host/service groups to monitor the actual NFS servers, not a random collection of miscellaneous ex-NFS servers [production]
04:46 <Tim> installed NRPE on nfs1 and nfs2 [production]
04:08 <Tim> adding rendering, m, bits.esams, recursor0, recursor1, recursor0.esams to nagios [production]
04:02 <Tim> added forward DNS entry for recursor0.esams, modified reverse DNS entry resolver0.esams -> recursor0.esams [production]
03:55 <Tim> fixed reverse DNS entries for recursor0 and recursor1, were set incorrectly to non-existent hostnames "resolver0" and "recursor1" [production]
03:36 <Tim> renamed db6.mgmt to locke.mgmt [production]
2010-07-10 §
14:14 <rainman-sr> search7 disk was full, deleting some old unneccessary indexes [production]
12:50 <Fred> applied security updates on all machine running Karmic or Lucid (per USN-959-1) [production]
2010-07-09 §
18:07 <domas> forgot to log, rebooted locke, put startup stuff to rc.local, maybe Tim changed it afterwards, hehe. beer is good too. [production]
15:31 <Rob> wikimania2011wiki is now using vector [production]
15:31 <robh> synchronized php-1.5/wmf-config/InitialiseSettings.php [production]
12:48 <robh> ran sync-common-all [production]
01:06 <tstarling> synchronized php-1.5/includes/filerepo/RepoGroup.php [production]
01:04 <tstarling> synchronized php-1.5/includes/filerepo/RepoGroup.php [production]
01:04 <root> synchronized php-1.5/includes/filerepo/RepoGroup.php [production]
01:03 <tstarling> synchronized php-1.5/includes/filerepo/RepoGroup.php [production]
00:59 <tstarling> synchronized php-1.5/includes/filerepo/RepoGroup.php [production]
2010-07-08 §
22:27 <apergos> powercycled db9 fromm drac after shutdown failed [production]
22:20 <Fred> re-imaging srv225 back to normal until wikimedia-task*can be ported to lucid. [production]
22:15 <apergos> rebooting db9, mysqld was defunct but the port was in use so couldn't restart it the nice way [production]
17:06 <mark> Set temporary 91.198.174.0/24 null0 route on br1-knams, to investigate prefix announcement problems [production]
16:10 <Rob> updated puppet to add zak to the mortals admin group and allowed access to shell on fenari as non-root [production]
04:10 <Tim> starting upload of BnF images, using importImages.php in screen on fenari [production]