2012-11-20
§
|
05:19 |
<Ryan_Lane> |
depooling ssl3001 |
[production] |
05:19 |
<Ryan_Lane> |
repooling ssl3002 (upgrade complete) |
[production] |
05:14 |
<paravoid> |
deploying squid config for score |
[production] |
05:11 |
<Ryan_Lane> |
rebooting ssl3002 |
[production] |
04:48 |
<Ryan_Lane> |
depooling ssl3002 for upgrade to precise |
[production] |
04:48 |
<Ryan_Lane> |
repooling ssl3003 (upgrade finished) |
[production] |
04:42 |
<Ryan_Lane> |
rebooting ssl3003 (upgraded via ssh, not console) |
[production] |
04:25 |
<notpeter> |
repooling srv258-srv280 |
[production] |
04:04 |
<Ryan_Lane> |
repooling ssl1 (upgrade complete) |
[production] |
03:52 |
<Tim> |
on brewster: reduced rotate count for squid logs to zero and ran logrotate -f |
[production] |
03:44 |
<Tim> |
on brewster: root partition is full, removing some useless squid logs |
[production] |
02:52 |
<logmsgbot_> |
LocalisationUpdate completed (1.21wmf3) at Tue Nov 20 02:52:09 UTC 2012 |
[production] |
02:28 |
<logmsgbot_> |
LocalisationUpdate completed (1.21wmf4) at Tue Nov 20 02:28:20 UTC 2012 |
[production] |
02:10 |
<notpeter> |
depooling srv296-srv301 for upgrades to precise |
[production] |
02:08 |
<notpeter> |
repooling srv290-srv295 |
[production] |
01:21 |
<Ryan_Lane> |
depooling ssl1 |
[production] |
01:21 |
<Ryan_Lane> |
repooling ssl2 (upgrade complete) |
[production] |
01:10 |
<Ryan_Lane> |
depooling ssl3003 to upgrade to precise |
[production] |
01:02 |
<MaxSem> |
Scap hung and had to be aborted. Since what was being deployed wasn't enabled, no clusters were harmed. |
[production] |
00:46 |
<Ryan_Lane> |
all eqiad https hosts upgraded to precise |
[production] |
00:46 |
<Ryan_Lane> |
repooling ssl1001 (upgrade complete) |
[production] |
00:23 |
<Ryan_Lane> |
depooling ssl2 for upgrade to precise |
[production] |
00:23 |
<Ryan_Lane> |
repooling ssl4 (upgrade complete) |
[production] |
00:21 |
<logmsgbot_> |
maxsem Started syncing Wikimedia installation... : Black-deployment of ext:Solarium, not enabled anywhere yet |
[production] |
00:01 |
<Ryan_Lane> |
depooling ssl1001 |
[production] |
00:01 |
<Ryan_Lane> |
repooling ssl1002 (upgrade complete) |
[production] |
2012-11-19
§
|
23:59 |
<logmsgbot_> |
reedy synchronized wmf-config/CommonSettings.php |
[production] |
23:37 |
<Ryan_Lane> |
repooling ssl3 depooling ssl4 |
[production] |
23:08 |
<logmsgbot_> |
maxsem synchronized wmf-config 'https://gerrit.wikimedia.org/r/#/c/34220/ and https://gerrit.wikimedia.org/r/#/c/33786/' |
[production] |
23:06 |
<Ryan_Lane> |
depooling ssl1002 for upgrade to precise |
[production] |
23:06 |
<Ryan_Lane> |
repooling ssl1003 (upgrade to precise complete) |
[production] |
22:59 |
<Ryan_Lane> |
make that rebooting ssl1003 |
[production] |
22:59 |
<Ryan_Lane> |
rebooting ssl1004 |
[production] |
22:53 |
<Ryan_Lane> |
rebooting ssl3 |
[production] |
22:51 |
<Ryan_Lane> |
re-pooling ssl4 and depooling ssl3 |
[production] |
22:35 |
<Ryan_Lane> |
depooling ssl4 |
[production] |
22:35 |
<Ryan_Lane> |
upgrading ssl1003 to precise |
[production] |
22:13 |
<Ryan_Lane> |
depooling ssl1003 |
[production] |
20:14 |
<mark> |
Restoring bits.esams traffic |
[production] |
19:56 |
<mark> |
Moving bits.esams traffic to pmtpa |
[production] |
19:20 |
<notpeter> |
depooling srv290-srv301 for upgrade to precise |
[production] |
19:20 |
<notpeter> |
depooling srv258-srv280 for upgrade to precise |
[production] |
19:11 |
<logmsgbot_> |
py synchronized wmf-config/db.php 'repooling es1' |
[production] |
18:45 |
<logmsgbot_> |
maxsem synchronized wmf-config/mobile.php 'https://gerrit.wikimedia.org/r/#/c/34105/' |
[production] |
18:14 |
<logmsgbot_> |
reedy rebuilt wikiversions.cdb and synchronized wikiversions files: enwiki to 1.21wmf4 |
[production] |
17:11 |
<logmsgbot_> |
reedy synchronized php-1.21wmf3/cache/interwiki.cdb |
[production] |
17:09 |
<logmsgbot_> |
reedy synchronized php-1.21wmf4/cache/interwiki.cdb |
[production] |
15:58 |
<paravoid> |
powercycling ms-be3, unresponsive |
[production] |
15:48 |
<logmsgbot_> |
reedy synchronized php-1.21wmf4/extensions/OAI |
[production] |
14:48 |
<logmsgbot_> |
reedy synchronized php-1.21wmf3/cache/interwiki.cdb 'Updating interwiki cache' |
[production] |