2009-09-15
§
|
15:08 |
<Rob> |
shutting down db12 for raid battery swap |
[production] |
15:04 |
<mark> |
Swapped drive c3t6d0 in ms2, readded it to /dev/md14 and moved back the spare /dev/sdao into the spare pool (/dev/md1) |
[production] |
14:42 |
<mark> |
Shutting down MySQL on ms2 |
[production] |
14:33 |
<Rob> |
removed a number of decommissioned servers from nagios |
[production] |
14:30 |
<Rob> |
wipe running on srv44, srv45, srv47 |
[production] |
14:26 |
<Rob> |
srv31, srv32, srv33 running wipe in screen sessions, do not try to use them ;] |
[production] |
14:22 |
<Rob> |
srv30=srv80 to be decommissioned, wiping the drives with them in rack now. mark already depooled from apache and memcached |
[production] |
14:19 |
<Rob> |
srv145 is back up and ok |
[production] |
14:07 |
<Rob> |
srv145 coming back up, my bad |
[production] |
13:53 |
<Rob> |
srv52 hdd died. |
[production] |
13:51 |
<domas> |
cron jobs work way better, if one figures how to set permissions right (like, executable? :) |
[production] |
00:03 |
<atglenn> |
grrr.. that would be ms4. |
[production] |
00:03 |
<atglenn> |
testing mailiferr (workaround for no MAILTO on solaris) on ms5 for hourly snaps |
[production] |
2009-09-14
§
|
23:26 |
<atglenn> |
rerunning the rsync list of changed files again on ms6 (last run was borked). it's in screen as root |
[production] |
23:23 |
<aZaFred_> |
added nagios/ganglia monitoring for snapshot[1..3] |
[production] |
22:32 |
<atglenn> |
created a directory /export/upload/wikipedia/common/thumb with no write perms on ms1 so that static html dump ext doesn't try to write in it |
[production] |
21:39 |
<aZaFred_> |
upgrading Ubuntu on Sage to 8.04 LTS |
[production] |
21:12 |
<tomaszf> |
running timings for en static html snapshot from hume |
[production] |
19:50 |
<robh> |
synchronized php-1.5/wmf-config/InitialiseSettings.php |
[production] |
19:47 |
<Rob> |
left out the virtual host, resyncing and restarting apaches |
[production] |
19:44 |
<robh> |
ran sync-common-all |
[production] |
18:21 |
<Rob> |
added volunteer.wikimedia.org to dns and pushed authdns-update |
[production] |
18:03 |
<Rob> |
added flagrevs stats update to hume crontab |
[production] |
16:24 |
<brion> |
restarted apache on wikitech |
[production] |