1051-1100 of 10000 results (8ms)
2012-05-02 §
16:16 <notpeter> starting innobackupex from db1040 to db1022 for new s6 snapshot slave [production]
15:31 <notpeter> no nagios bot, kicking nagios on spence [production]
15:04 <RobH> shutting down mw64 for hw test per rt 1890 [production]
15:03 <RobH> bellin crashed, unresponsive to ssh or serial console [production]
14:43 <mark> Built varnish for precise as 3.0.2-2wm5 and imported it into APT repository precise-wikimedia [production]
11:52 <mark> Started distribution upgrade of server stafford from Lucid to Precise [production]
10:41 <mutante> refreshLinks.php - started it once again in a screen on hume, just for s1. last cron failed with "mwscript command not found"?? well now it is there again and running [production]
10:09 <mark___> Started distribution upgrade of server sockpuppet from Lucid to Precise [production]
09:20 <mutante> upgrading bugzilla to 4.0.6 [production]
08:43 <mutante> kaulen: installing various upgrades (apache,mysql,cron,php-wikidiff2,...) [production]
08:40 <hashar> synchronized php-1.20wmf2/includes/GitInfo.php 'Fix Special:Version for 1.20wmf2 (commit ae12df0)' [production]
08:26 <hashar:> using my own logging script [production]
08:20 <hashar> cherry-picked ae12df0 commit to 1.20wmf2 since there are mobilefrontend commits pending. [production]
02:35 <LocalisationUpdate> completed (1.20wmf2) at Wed May 2 02:35:51 UTC 2012 [production]
02:32 <K4-713> updated production civicrm to [[rev:1723|r1723]] [production]
02:13 <LocalisationUpdate> completed (1.20wmf1) at Wed May 2 02:13:30 UTC 2012 [production]
01:01 <notpeter> starting innobackupex from db57 to db53 for new s2 slave for the one zillionth time [production]
2012-05-01 §
22:28 <logmsgbot_> asher synchronized wmf-config/db.php 'returning db45' [production]
22:23 <logmsgbot_> asher synchronized wmf-config/db.php 'pulling db45, last coredb on prior fb mysql build' [production]
22:17 <logmsgbot_> reedy synchronized wmf-config/InitialiseSettings.php 'Enable doublepage on test2wiki' [production]
22:11 <binasher> upgraded percona-toolkit on coredbs to 2.1.1 - now with the potential to run online schema changes on tables without single column unique keys!! [production]
21:39 <binasher> created an ops db on all core mysql shards [production]
21:00 <notpeter> reinstalling db53. this time with correct raid! [production]
20:40 <logmsgbot_> awjrichards synchronized wmf-config/CommonSettings.php 'Fixing mailto links on mobilefrontend feedback form to properly populate subject lines' [production]
19:32 <LeslieCarr> reverting vrrp mastership of row a to cr2-eqiad [production]
19:29 <LeslieCarr> switching vrrp mastership of row a to cr1-eqiad [production]
18:32 <logmsgbot_> awjrichards synchronized wmf-config/InitialiseSettings.php 'Make testwiki use mobile domain for URLs' [production]
18:28 <LeslieCarr> making routing change, higher risk [production]
17:51 <Ryan_Lane> make that virt0 [production]
17:51 <Ryan_Lane> switching the session cache back to filesystem on virt1, since it isn't working properly with memcache [production]
17:29 <maplebed> kicking nagios to check a change to fix the mobile LVS alert [production]
17:25 <logmsgbot_> nikerabbit synchronized php-1.20wmf2/extensions/TranslationNotifications/ 'Deploying TranslationNotifications code' [production]
17:08 <logmsgbot_> reedy rebuilt wikiversions.cdb and synchronized wikiversions files: testwiki to 1.20wmf2 [production]
16:27 <notpeter> starting innobackupex from db1034 to db53 for new s2 slave [production]
16:27 <notpeter> starting innobackupex from db57 to db52 for new s2 slave [production]
16:03 <notpeter> rebuilding db52 and db53 as s2 slaves [production]
15:47 <logmsgbot_> asher synchronized wmf-config/db.php 's1: raising db59,60 weights, pulling db53/54 for reuse' [production]
09:23 <logmsgbot_> reedy synchronized wmf-config/CommonSettings.php 'hewiki account creation high throttle limits' [production]
04:04 <Tim> on all apaches, running "chmod -R a+rX /usr/local/apache/common-local/" to clean up after killed rsyncs which left files unreadable [production]
02:23 <logmsgbot_> LocalisationUpdate completed (1.20wmf2) at Tue May 1 02:23:29 UTC 2012 [production]
02:21 <logmsgbot_> preilly synchronized php-1.20wmf1/extensions/MobileFrontend/specials/SpecialMobileFeedback.php [production]
02:14 <logmsgbot_> LocalisationUpdate completed (1.20wmf1) at Tue May 1 02:14:06 UTC 2012 [production]
02:06 <logmsgbot_> preilly synchronized php-1.20wmf1/extensions/MobileFrontend/specials/SpecialMobileOptions.php [production]
01:51 <Ryan_Lane> bringing up all labs instances with a 60 second lag [production]
01:40 <Ryan_Lane> rebooting virt0 [production]
01:35 <Ryan_Lane> rebooting virt3 [production]
01:33 <logmsgbot_> preilly synchronized php-1.20wmf1/extensions/MobileFrontend/HtmlFormatter.php [production]
01:26 <Ryan_Lane> rebooting virt5 [production]
01:18 <Ryan_Lane> rebooting virt4 [production]
01:03 <Ryan_Lane> rebooting virt2 [production]