|
2012-11-20
§
|
| 05:19 |
<Ryan_Lane> |
depooling ssl3001 |
[production] |
| 05:19 |
<Ryan_Lane> |
repooling ssl3002 (upgrade complete) |
[production] |
| 05:14 |
<paravoid> |
deploying squid config for score |
[production] |
| 05:11 |
<Ryan_Lane> |
rebooting ssl3002 |
[production] |
| 04:48 |
<Ryan_Lane> |
depooling ssl3002 for upgrade to precise |
[production] |
| 04:48 |
<Ryan_Lane> |
repooling ssl3003 (upgrade finished) |
[production] |
| 04:42 |
<Ryan_Lane> |
rebooting ssl3003 (upgraded via ssh, not console) |
[production] |
| 04:25 |
<notpeter> |
repooling srv258-srv280 |
[production] |
| 04:04 |
<Ryan_Lane> |
repooling ssl1 (upgrade complete) |
[production] |
| 03:52 |
<Tim> |
on brewster: reduced rotate count for squid logs to zero and ran logrotate -f |
[production] |
| 03:44 |
<Tim> |
on brewster: root partition is full, removing some useless squid logs |
[production] |
| 02:52 |
<logmsgbot_> |
LocalisationUpdate completed (1.21wmf3) at Tue Nov 20 02:52:09 UTC 2012 |
[production] |
| 02:28 |
<logmsgbot_> |
LocalisationUpdate completed (1.21wmf4) at Tue Nov 20 02:28:20 UTC 2012 |
[production] |
| 02:10 |
<notpeter> |
depooling srv296-srv301 for upgrades to precise |
[production] |
| 02:08 |
<notpeter> |
repooling srv290-srv295 |
[production] |
| 01:21 |
<Ryan_Lane> |
depooling ssl1 |
[production] |
| 01:21 |
<Ryan_Lane> |
repooling ssl2 (upgrade complete) |
[production] |
| 01:10 |
<Ryan_Lane> |
depooling ssl3003 to upgrade to precise |
[production] |
| 01:02 |
<MaxSem> |
Scap hung and had to be aborted. Since what was being deployed wasn't enabled, no clusters were harmed. |
[production] |
| 00:46 |
<Ryan_Lane> |
all eqiad https hosts upgraded to precise |
[production] |
| 00:46 |
<Ryan_Lane> |
repooling ssl1001 (upgrade complete) |
[production] |
| 00:23 |
<Ryan_Lane> |
depooling ssl2 for upgrade to precise |
[production] |
| 00:23 |
<Ryan_Lane> |
repooling ssl4 (upgrade complete) |
[production] |
| 00:21 |
<logmsgbot_> |
maxsem Started syncing Wikimedia installation... : Black-deployment of ext:Solarium, not enabled anywhere yet |
[production] |
| 00:01 |
<Ryan_Lane> |
depooling ssl1001 |
[production] |
| 00:01 |
<Ryan_Lane> |
repooling ssl1002 (upgrade complete) |
[production] |
|
2012-11-19
§
|
| 23:59 |
<logmsgbot_> |
reedy synchronized wmf-config/CommonSettings.php |
[production] |
| 23:37 |
<Ryan_Lane> |
repooling ssl3 depooling ssl4 |
[production] |
| 23:08 |
<logmsgbot_> |
maxsem synchronized wmf-config 'https://gerrit.wikimedia.org/r/#/c/34220/ and https://gerrit.wikimedia.org/r/#/c/33786/' |
[production] |
| 23:06 |
<Ryan_Lane> |
depooling ssl1002 for upgrade to precise |
[production] |
| 23:06 |
<Ryan_Lane> |
repooling ssl1003 (upgrade to precise complete) |
[production] |
| 22:59 |
<Ryan_Lane> |
make that rebooting ssl1003 |
[production] |
| 22:59 |
<Ryan_Lane> |
rebooting ssl1004 |
[production] |
| 22:53 |
<Ryan_Lane> |
rebooting ssl3 |
[production] |
| 22:51 |
<Ryan_Lane> |
re-pooling ssl4 and depooling ssl3 |
[production] |
| 22:35 |
<Ryan_Lane> |
depooling ssl4 |
[production] |
| 22:35 |
<Ryan_Lane> |
upgrading ssl1003 to precise |
[production] |
| 22:13 |
<Ryan_Lane> |
depooling ssl1003 |
[production] |
| 20:14 |
<mark> |
Restoring bits.esams traffic |
[production] |
| 19:56 |
<mark> |
Moving bits.esams traffic to pmtpa |
[production] |
| 19:20 |
<notpeter> |
depooling srv290-srv301 for upgrade to precise |
[production] |
| 19:20 |
<notpeter> |
depooling srv258-srv280 for upgrade to precise |
[production] |
| 19:11 |
<logmsgbot_> |
py synchronized wmf-config/db.php 'repooling es1' |
[production] |
| 18:45 |
<logmsgbot_> |
maxsem synchronized wmf-config/mobile.php 'https://gerrit.wikimedia.org/r/#/c/34105/' |
[production] |
| 18:14 |
<logmsgbot_> |
reedy rebuilt wikiversions.cdb and synchronized wikiversions files: enwiki to 1.21wmf4 |
[production] |
| 17:11 |
<logmsgbot_> |
reedy synchronized php-1.21wmf3/cache/interwiki.cdb |
[production] |
| 17:09 |
<logmsgbot_> |
reedy synchronized php-1.21wmf4/cache/interwiki.cdb |
[production] |
| 15:58 |
<paravoid> |
powercycling ms-be3, unresponsive |
[production] |
| 15:48 |
<logmsgbot_> |
reedy synchronized php-1.21wmf4/extensions/OAI |
[production] |
| 14:48 |
<logmsgbot_> |
reedy synchronized php-1.21wmf3/cache/interwiki.cdb 'Updating interwiki cache' |
[production] |