4651-4700 of 10000 results (47ms)
2016-04-04 §
14:46 <elukey> de-pooled aqs1001.eqiad from the confd pool for nodejs upgrade [production]
13:40 <ema> nginx rolling restart for openssl upgrade on cache hosts [production]
10:42 <elukey> re-pooled aqs1001.eqiad (no node upgrade, need more info about restbase) [production]
10:23 <godog> reduce reserved blocks for /srv on restbase2004 [production]
09:59 <moritzm> installing pcre3 updates [production]
09:57 <godog> start expanding raid0 on restbase2003 [production]
09:54 <ema> nginx rolling restart for openssl upgrade: cp1046, cp1052, cp1068, cp1071, cp1099 [production]
09:53 <elukey> de-pooled aqs1001.eqiad.wmnet as pre-step for nodejs upgrade [production]
09:41 <godog> depool restbase2003 before raid expansion [production]
09:37 <godog> repool restbase2004 [production]
09:33 <godog> deploy restbase ba39d2bcd2f5 to restbase2004 before repooling [production]
09:26 <volans> Re-enabling Puppet on cluster mysql and parsercache to deploy change 279596, T111654 [production]
09:10 <volans> Disabling Puppet on cluster mysql and parsercache to merge and test change 279596 on db2040, T111654 [production]
08:50 <moritzm> installing gnupg updates [production]
06:54 <moritzm> installing apt bugfix updates [production]
04:18 <TimStarling> iridium came back up, but mcelog reports high CPU temperature prior to the shutdown [production]
04:13 <TimStarling> attempting to turn iridium back on via drac. "getraclog" says it powered itself off after resetting four times [production]
03:15 <greg-g> Phabricator is down [production]
02:33 <l10nupdate@tin> ResourceLoader cache refresh completed at Mon Apr 4 02:33:39 UTC 2016 (duration 8m 31s) [production]
02:25 <mwdeploy@tin> sync-l10n completed (1.27.0-wmf.19) (duration: 11m 45s) [production]
00:49 <krinkle@tin> Synchronized wmf-config/InitialiseSettings.php: (no message) (duration: 00m 29s) [production]
00:30 <krinkle@tin> Synchronized w/static.php: (no message) (duration: 00m 35s) [production]
00:21 <Krinkle> mwscript deleteEqualMessages.php --wiki ruwiki (T45917) [production]
00:06 <Krinkle> mwscript deleteEqualMessages.php --wiki metawiki --delete [production]
2016-04-03 §
22:37 <Krinkle> mwscript deleteEqualMessages.php --wiki eswiki --lang-code ca (T45917) [production]
22:37 <Krinkle> mwscript deleteEqualMessages.php --wiki eswiki (T45917) [production]
22:34 <Krinkle> mwscript deleteEqualMessages.php --wiki eswiki [production]
19:16 <Krenair> mw1146 began to respond with 503s to all requests, tried restarting apache2/hhvm and shortly afterwards it started working again [production]
13:26 <reedy@tin> Synchronized wmf-config/InitialiseSettings-labs.php: labs copy upload thing. noooooop for prod (duration: 00m 33s) [production]
13:02 <reedy@tin> Synchronized wmf-config/InitialiseSettings.php: labs copy upload thing. noooooop for prod (duration: 00m 28s) [production]
11:37 <ori@tin> Synchronized wmf-config/extension-list-labs: I0d081186: Also add Newsletter to extension-list-labs (duration: 00m 27s) [production]
11:34 <ori@tin> Synchronized wmf-config/InitialiseSettings-labs.php: I3ffe65b8: Load the Newsletter extension on the beta cluster (2/2) (duration: 00m 33s) [production]
11:34 <ori@tin> Synchronized wmf-config/CommonSettings-labs.php: I3ffe65b8: Load the Newsletter extension on the beta cluster (1/2) (duration: 00m 34s) [production]
03:14 <mwdeploy@tin> sync-l10n completed (1.27.0-wmf.19) (duration: 56m 43s) [production]
2016-04-02 §
20:28 <hashar> Restarted Jenkins on gallium [production]
19:16 <dereckson@tin> Synchronized wmf-config/InitialiseSettings.php: 350K celebration logo for cs.wikipedia (T131605) (duration: 00m 29s) [production]
19:14 <dereckson@tin> Synchronized static/images/project-logos/cswiki.png: 350K celebration logo for cs.wikipedia (duration: 00m 33s) [production]
10:45 <reedy@tin> rebuilt wikiversions.php and synchronized wikiversions files: wikitech back to .19 [production]
10:01 <reedy@tin> Synchronized php-1.27.0-wmf.19/extensions/OATHAuth: Fix for 2FA testing (duration: 00m 30s) [production]
07:22 <krinkle@tin> Synchronized php-1.27.0-wmf.19/extensions/NavigationTiming/modules/ext.navigationTiming.js: T131565 (duration: 00m 33s) [production]
02:32 <l10nupdate@tin> ResourceLoader cache refresh completed at Sat Apr 2 02:32:42 UTC 2016 (duration 8m 36s) [production]
02:24 <mwdeploy@tin> sync-l10n completed (1.27.0-wmf.19) (duration: 10m 48s) [production]
02:03 <l10nupdate@tin> LocalisationUpdate failed (1.27.0-wmf.18) at 2016-04-02 02:03:32+00:00 [production]
01:04 <krinkle@tin> Synchronized php-1.27.0-wmf.19/includes/specials/SpecialRedirect.php: T131328 (duration: 00m 39s) [production]
2016-04-01 §
23:43 <krinkle@tin> Synchronized php-1.27.0-wmf.19/extensions/MobileFrontend/includes/MobileFrontend.hooks.php: T131337 (duration: 00m 38s) [production]
22:39 <Pchelolo> rolling restart restbase to pick up https://gerrit.wikimedia.org/r/#/c/280951/ config change [production]
22:13 <mutante> hooft - shutdown [production]
21:36 <mutante> hooft: stopping ganglia aggregators, remove from icinga/storedconfigclean [production]
21:25 <mutante> hooft.esams - stop puppet, stop salt [production]
20:57 <mutante> bast3001 - sign puppet certs, salt keys [production]