|
2012-10-13
§
|
| 21:39 |
<logmsgbot_> |
preilly Finished syncing Wikimedia installation... : update zero rated mobile access |
[production] |
| 21:18 |
<logmsgbot_> |
preilly Started syncing Wikimedia installation... : update zero rated mobile access |
[production] |
| 21:06 |
<logmsgbot_> |
preilly synchronized php-1.21wmf1/extensions/MobileFrontend 'update for landing page' |
[production] |
| 18:51 |
<logmsgbot_> |
reedy synchronized php-1.21wmf1/extensions/Collection 'Revert to master' |
[production] |
| 18:41 |
<logmsgbot_> |
reedy synchronized php-1.21wmf1/extensions/Collection 'Revert to original again..' |
[production] |
| 18:38 |
<logmsgbot_> |
reedy synchronized php-1.21wmf1/extensions/Collection 'Revert to f980f2380b830e712edfdbce065784d1bcba82fd' |
[production] |
| 18:36 |
<logmsgbot_> |
reedy synchronized php-1.21wmf1/extensions/Collection 'Revert to 4f898444ce5d9c2c9a5491f2303db15a3143a5c5' |
[production] |
| 18:34 |
<logmsgbot_> |
reedy synchronized php-1.21wmf1/extensions/Collection 'Revert to 72b7d4d645489dd092605a2183a944fac33cb40b' |
[production] |
| 18:26 |
<logmsgbot_> |
reedy synchronized php-1.21wmf1/extensions/Collection 'Revert to original' |
[production] |
| 18:25 |
<logmsgbot_> |
reedy synchronized php-1.21wmf1/extensions/Collection 'Revert to 8415a4564bf429b76e113368a3e5e1c19fb84a47' |
[production] |
| 18:23 |
<logmsgbot_> |
reedy synchronized php-1.21wmf1/extensions/Collection 'Revert to 72b7d4d645489dd092605a2183a944fac33cb40b' |
[production] |
| 18:21 |
<logmsgbot_> |
reedy synchronized php-1.21wmf1/extensions/Collection 'Revert to 9f37005e78a068821c45d9d313394c6728f55b4a' |
[production] |
| 16:34 |
<logmsgbot_> |
reedy synchronized php-1.21wmf1/extensions/Collection |
[production] |
| 16:27 |
<logmsgbot_> |
reedy synchronized php-1.21wmf1/extensions/Collection |
[production] |
| 13:17 |
<logmsgbot_> |
reedy synchronized wmf-config/throttle.php |
[production] |
| 02:00 |
<logmsgbot_> |
LocalisationUpdate failed: git pull of extensions failed |
[production] |
|
2012-10-12
§
|
| 23:51 |
<logmsgbot_> |
asher synchronized wmf-config/CommonSettings.php 'enabling redis debug log' |
[production] |
| 23:40 |
<LeslieCarr> |
moving traffic back to esams via authdns-scenario |
[production] |
| 23:40 |
<LeslieCarr> |
reenabled nagios alerts |
[production] |
| 23:20 |
<logmsgbot_> |
asher synchronized wmf-config/CommonSettings.php 'enabling session multiwrites to redis' |
[production] |
| 23:16 |
<LeslieCarr> |
rebooting esams core switch stack, expect reachability problems to esams |
[production] |
| 23:01 |
<logmsgbot_> |
asher synchronized wmf-config/CommonSettings.php 'disabling session multiwrites to redis' |
[production] |
| 22:36 |
<LeslieCarr> |
switching europe traffic to pmtpa via authdns-scenario |
[production] |
| 22:36 |
<LeslieCarr> |
muted nagios |
[production] |
| 20:20 |
<AaronSchulz> |
Confirmed all swift -> netapp sync scripts finished (s4 one kept dying to random http errors) |
[production] |
| 19:58 |
<logmsgbot_> |
reedy synchronized wmf-config/ |
[production] |
| 19:01 |
<cmjohnson1> |
powering down search32 to replace cpu1 |
[production] |
| 18:33 |
<logmsgbot_> |
reedy synchronized wmf-config/InitialiseSettings.php 'AFT (non v5) doesn't use clicktracking now' |
[production] |
| 18:28 |
<logmsgbot_> |
reedy synchronized php-1.21wmf1/extensions/ArticleFeedback/ |
[production] |
| 18:19 |
<cmjohnson1> |
troubleshooting db42 w/IBM tech |
[production] |
| 17:46 |
<cmjohnson1> |
pc3 shutting down to replace hdd's with ssd's rt3703 |
[production] |
| 17:29 |
<cmjohnson1> |
pc2 shutting down to replaced hdd w/ ssd rt3703 |
[production] |
| 16:27 |
<logmsgbot_> |
reedy synchronized wmf-config/CommonSettings.php 'Fix extdist mount related warnings' |
[production] |
| 16:14 |
<cmjohnson1> |
running installer on storage3 |
[production] |
| 15:54 |
<paravoid> |
moving bits from esams to pmtpa (authdns normal scenario) |
[production] |
| 15:52 |
<cmjohnson1> |
storage 3 replacing disk 12 and rebuilding raid |
[production] |
| 15:32 |
<mark> |
Disabled csw1-esams:e8/2 |
[production] |
| 14:37 |
<mark> |
Rebooting cr1-esams |
[production] |
| 14:17 |
<mark> |
Upgrading JunOS on cr1-esams to 11.4R5.5, preparing for reboot |
[production] |
| 13:59 |
<mark> |
Activated aggregate generation on cr1-esams |
[production] |
| 13:49 |
<mark> |
Moved AS38930 transit from csw1-esams:e8/4 to cr1-esams:xe-1/3/0 |
[production] |
| 13:24 |
<cmjohnson1> |
storage3 powering down |
[production] |
| 12:27 |
<mark> |
Upgraded new stack member asw-oe11-esams to junos 11.4R5.5 and rebooted it |
[production] |
| 12:16 |
<mark> |
Added new EX4200-48T asw-oe11-esams to csw2-esams stack |
[production] |
| 11:08 |
<mark> |
Disabled csw2-oe12-esams:vcp-1 |
[production] |
| 02:01 |
<logmsgbot_> |
LocalisationUpdate failed: git pull of extensions failed |
[production] |
| 01:21 |
<mutante> |
moving ancient copy of bugzilla out of /h/w/htdocs (to /root/backup on fenari temp. just in case) |
[production] |
| 01:15 |
<mutante> |
deleting old document root dirs on fenari. /home/wikipedia/htdocs/noc and noc.old (use /h/w/common/docroot/noc and mediawiki-config repo) |
[production] |
| 01:08 |
<mutante> |
restarting Apache on fenari for noc.wm change |
[production] |
| 00:42 |
<Jeff_Green> |
switched payments back to pmtpa and increased TTL to 5 min |
[production] |