|
2012-05-02
§
|
| 15:31 |
<notpeter> |
no nagios bot, kicking nagios on spence |
[production] |
| 15:04 |
<RobH> |
shutting down mw64 for hw test per rt 1890 |
[production] |
| 15:03 |
<RobH> |
bellin crashed, unresponsive to ssh or serial console |
[production] |
| 14:43 |
<mark> |
Built varnish for precise as 3.0.2-2wm5 and imported it into APT repository precise-wikimedia |
[production] |
| 11:52 |
<mark> |
Started distribution upgrade of server stafford from Lucid to Precise |
[production] |
| 10:41 |
<mutante> |
refreshLinks.php - started it once again in a screen on hume, just for s1. last cron failed with "mwscript command not found"?? well now it is there again and running |
[production] |
| 10:09 |
<mark___> |
Started distribution upgrade of server sockpuppet from Lucid to Precise |
[production] |
| 09:20 |
<mutante> |
upgrading bugzilla to 4.0.6 |
[production] |
| 08:43 |
<mutante> |
kaulen: installing various upgrades (apache,mysql,cron,php-wikidiff2,...) |
[production] |
| 08:40 |
<hashar> |
synchronized php-1.20wmf2/includes/GitInfo.php 'Fix Special:Version for 1.20wmf2 (commit ae12df0)' |
[production] |
| 08:26 |
<hashar:> |
using my own logging script |
[production] |
| 08:20 |
<hashar> |
cherry-picked ae12df0 commit to 1.20wmf2 since there are mobilefrontend commits pending. |
[production] |
| 02:35 |
<LocalisationUpdate> |
completed (1.20wmf2) at Wed May 2 02:35:51 UTC 2012 |
[production] |
| 02:32 |
<K4-713> |
updated production civicrm to [[rev:1723|r1723]] |
[production] |
| 02:13 |
<LocalisationUpdate> |
completed (1.20wmf1) at Wed May 2 02:13:30 UTC 2012 |
[production] |
| 01:01 |
<notpeter> |
starting innobackupex from db57 to db53 for new s2 slave for the one zillionth time |
[production] |
|
2012-05-01
§
|
| 22:28 |
<logmsgbot_> |
asher synchronized wmf-config/db.php 'returning db45' |
[production] |
| 22:23 |
<logmsgbot_> |
asher synchronized wmf-config/db.php 'pulling db45, last coredb on prior fb mysql build' |
[production] |
| 22:17 |
<logmsgbot_> |
reedy synchronized wmf-config/InitialiseSettings.php 'Enable doublepage on test2wiki' |
[production] |
| 22:11 |
<binasher> |
upgraded percona-toolkit on coredbs to 2.1.1 - now with the potential to run online schema changes on tables without single column unique keys!! |
[production] |
| 21:39 |
<binasher> |
created an ops db on all core mysql shards |
[production] |
| 21:00 |
<notpeter> |
reinstalling db53. this time with correct raid! |
[production] |
| 20:40 |
<logmsgbot_> |
awjrichards synchronized wmf-config/CommonSettings.php 'Fixing mailto links on mobilefrontend feedback form to properly populate subject lines' |
[production] |
| 19:32 |
<LeslieCarr> |
reverting vrrp mastership of row a to cr2-eqiad |
[production] |
| 19:29 |
<LeslieCarr> |
switching vrrp mastership of row a to cr1-eqiad |
[production] |
| 18:32 |
<logmsgbot_> |
awjrichards synchronized wmf-config/InitialiseSettings.php 'Make testwiki use mobile domain for URLs' |
[production] |
| 18:28 |
<LeslieCarr> |
making routing change, higher risk |
[production] |
| 17:51 |
<Ryan_Lane> |
make that virt0 |
[production] |
| 17:51 |
<Ryan_Lane> |
switching the session cache back to filesystem on virt1, since it isn't working properly with memcache |
[production] |
| 17:29 |
<maplebed> |
kicking nagios to check a change to fix the mobile LVS alert |
[production] |
| 17:25 |
<logmsgbot_> |
nikerabbit synchronized php-1.20wmf2/extensions/TranslationNotifications/ 'Deploying TranslationNotifications code' |
[production] |
| 17:08 |
<logmsgbot_> |
reedy rebuilt wikiversions.cdb and synchronized wikiversions files: testwiki to 1.20wmf2 |
[production] |
| 16:27 |
<notpeter> |
starting innobackupex from db1034 to db53 for new s2 slave |
[production] |
| 16:27 |
<notpeter> |
starting innobackupex from db57 to db52 for new s2 slave |
[production] |
| 16:03 |
<notpeter> |
rebuilding db52 and db53 as s2 slaves |
[production] |
| 15:47 |
<logmsgbot_> |
asher synchronized wmf-config/db.php 's1: raising db59,60 weights, pulling db53/54 for reuse' |
[production] |
| 09:23 |
<logmsgbot_> |
reedy synchronized wmf-config/CommonSettings.php 'hewiki account creation high throttle limits' |
[production] |
| 04:04 |
<Tim> |
on all apaches, running "chmod -R a+rX /usr/local/apache/common-local/" to clean up after killed rsyncs which left files unreadable |
[production] |
| 02:23 |
<logmsgbot_> |
LocalisationUpdate completed (1.20wmf2) at Tue May 1 02:23:29 UTC 2012 |
[production] |
| 02:21 |
<logmsgbot_> |
preilly synchronized php-1.20wmf1/extensions/MobileFrontend/specials/SpecialMobileFeedback.php |
[production] |
| 02:14 |
<logmsgbot_> |
LocalisationUpdate completed (1.20wmf1) at Tue May 1 02:14:06 UTC 2012 |
[production] |
| 02:06 |
<logmsgbot_> |
preilly synchronized php-1.20wmf1/extensions/MobileFrontend/specials/SpecialMobileOptions.php |
[production] |
| 01:51 |
<Ryan_Lane> |
bringing up all labs instances with a 60 second lag |
[production] |
| 01:40 |
<Ryan_Lane> |
rebooting virt0 |
[production] |
| 01:35 |
<Ryan_Lane> |
rebooting virt3 |
[production] |
| 01:33 |
<logmsgbot_> |
preilly synchronized php-1.20wmf1/extensions/MobileFrontend/HtmlFormatter.php |
[production] |
| 01:26 |
<Ryan_Lane> |
rebooting virt5 |
[production] |
| 01:18 |
<Ryan_Lane> |
rebooting virt4 |
[production] |
| 01:03 |
<Ryan_Lane> |
rebooting virt2 |
[production] |
| 00:51 |
<LeslieCarr> |
restarted swift-container-auditor on ms-be5 |
[production] |