|
2009-11-26
§
|
| 17:23 |
<mark> |
Removing wikimedia-nis-client on all search servers |
[production] |
| 17:01 |
<mark> |
Running apt-get upgrade on all search servers |
[production] |
| 16:43 |
<mark> |
Removed wikimedia-nis-client on srv225 |
[production] |
| 16:41 |
<mark> |
Removed wikimedia-nis-client on spence |
[production] |
| 16:38 |
<mark> |
Removed wikimedia-nis-client on hume |
[production] |
| 16:33 |
<mark> |
Removed wikimedia-nis-client on fenari |
[production] |
| 16:32 |
<mark> |
Removed wikimedia-nis-client on amane |
[production] |
| 16:27 |
<mark> |
Removed wikimedia-nis-client on srv124 |
[production] |
| 16:27 |
<mark> |
Replacing all NIS clients by puppet managed users |
[production] |
| 15:08 |
<mark> |
Set domain of fenari back to 'wikimedia.org' from 'pmtpa.wmnet'. Why was it changed? Quite a few things got confused... |
[production] |
| 15:04 |
<mark> |
Made NFS /home mounting puppet managed on all relevant servers |
[production] |
| 13:18 |
<mark> |
Running apt-get dist-upgrade on pdf1, in part to upgrade libpoppler (PDF rendering library) |
[production] |
| 13:18 |
<mark> |
Removed rogue apt repositories on pdfd1 |
[production] |
| 02:24 |
<Tim> |
started apache on spence |
[production] |
|
2009-11-24
§
|
| 23:50 |
<fvassard> |
synchronized php-1.5/wmf-config/CommonSettings.php 'Adding 2009_Notice30_bold' |
[production] |
| 23:00 |
<midom> |
synchronized php-1.5/wmf-config/db.php 'deebee seventeen and twenteeseven' |
[production] |
| 22:13 |
<midom> |
synchronized php-1.5/wmf-config/db.php 'adding db21 and db29 back' |
[production] |
| 21:20 |
<midom> |
synchronized php-1.5/wmf-config/db.php 'taking out db17 and db29 too' |
[production] |
| 21:16 |
<midom> |
synchronized php-1.5/wmf-config/db.php 'taking out db21 and db27, AND NO PUPPETS' |
[production] |
| 21:14 |
<mark> |
Setup logrotate for /home/w/logs/ on both nfs1/nfs2 (of course, with puppet ;-) |
[production] |
| 19:07 |
<mark> |
Resized JFS filesystem of /home on nfs2 to fit the new LV size on nfs1/nfs2 |
[production] |
| 19:03 |
<mark> |
Moved cronned rsync of /home from db20 to nfs1 and nfs2, managed by puppet |
[production] |
| 18:46 |
<mark> |
MediaWiki logging, broken by yesterday's NFS service IP migration, restored (using puppet) |
[production] |
| 18:15 |
<mark> |
Puppetised rsyncd setup on nfs1/nfs2 |
[production] |
| 18:00 |
<andrew> |
synchronized php-1.5/wmf-config/InitialiseSettings.php 'Flood flag for strategywiki bug 20347' |
[production] |
| 01:43 |
<Tim> |
enabled write-back caching on db14 despite broken controller battery, due to excessive lag and MASTER_POS_WAIT() |
[production] |
| 00:24 |
<fvassard> |
synchronized php-1.5/wmf-config/CommonSettings.php 'It turns out that this time FiveFacts is actually referenced as Five_Facts_About_Wikipedia... who knew...' |
[production] |
| 00:10 |
<fvassard> |
synchronized php-1.5/wmf-config/CommonSettings.php 'It turns out that this time FiveFacts is actually referenced as Five_Facts_About_Wikipedia... who knew...' |
[production] |
| 00:01 |
<fvassard> |
synchronized php-1.5/wmf-config/CommonSettings.php 'Adding Notice_[36,37,28] and FiveFacts landing page' |
[production] |
|
2009-11-23
§
|
| 21:42 |
<fvassard> |
synchronized php-1.5/wmf-config/CommonSettings.php 'Adding RU and IL for GEOIP location.' |
[production] |
| 21:27 |
<fvassard> |
synchronized php-1.5/wmf-config/CommonSettings.php 'Adding RU and IL for GEOIP location.' |
[production] |
| 20:53 |
<river> |
moving ptolemy to TS vlan to reinstall it as the TS OSM db |
[production] |
| 20:11 |
<mark> |
Finished migrating NFS off db20 to a DRBD cluster of nfs2 (primary) and nfs1 (secondary) |
[production] |
| 19:14 |
<RoanKattouw> |
prototype got stuck, rebooted it. Linode sucks and we should not use it ever again |
[production] |
| 16:53 |
<mark> |
Re-signed puppet cert for fenari... why was it gone? |
[production] |
| 12:39 |
<domas> |
spence was taken down by runaway rrdtool process, http://p.defau.lt/?8FAf9FtfQTbkSQKCcP_qmw |
[production] |
| 11:00 |
<Andrew> |
Scapping to prod message updates |
[production] |
| 10:49 |
<andrew> |
synchronized php-1.5/extensions/LiquidThreads_alpha/pages/NewUserMessagesView.php 'Merge r59353' |
[production] |
| 10:35 |
<Andrew> |
Scapping |
[production] |
| 10:34 |
<Andrew> |
Updating LiquidThreads to trunk state on alpha only |
[production] |
| 08:28 |
<domas> |
sq14 has failed sdc |
[production] |
| 02:05 |
<Tim> |
fixed clear-profile so that it works on fenari and other ubuntu hosts |
[production] |
| 01:54 |
<tstarling> |
synchronized php-1.5/index.php 'test' |
[production] |
| 01:54 |
<Tim> |
set up fenari's /root/.ssh/config like the one on zwinger, with its user known hosts file pointing to the global one |
[production] |
| 01:41 |
<Tim> |
Moved logmsgbot to fenari. Opened up its UDP port to zwinger and added firewall rules to /etc/rc.local for honour-system access control. |
[production] |
| 01:21 |
<Tim> |
commented out the nagios-specific nickserv stuff in ircecho, so that logmsgbot doesn't ghost nagios-wm |
[production] |