6051-6100 of 10000 results (31ms)
2014-09-01 §
07:27 <godog> deploy latest ring to swift eqiad-prod [production]
07:11 <godog> powercycle ms-be1010 "cpu soft lockup" on console [production]
05:28 <springle> xtrabackup clone db1027 to db1044 [production]
05:26 <springle> Synchronized wmf-config/db-eqiad.php: depool db1027 while cloning (duration: 00m 07s) [production]
03:15 <LocalisationUpdate> ResourceLoader cache refresh completed at Mon Sep 1 03:14:22 UTC 2014 (duration 14m 21s) [production]
02:28 <LocalisationUpdate> completed (1.24wmf19) at 2014-09-01 02:27:39+00:00 [production]
02:17 <LocalisationUpdate> completed (1.24wmf18) at 2014-09-01 02:16:18+00:00 [production]
2014-08-31 §
19:56 <hashar> Jenkins updated HHVM to (3.3-dev+20140728+wmf5) over (3.3-dev+20140728+wmf4) [production]
14:38 <bblack> restarted apache on strontium [production]
14:34 <bblack> restarted apache on tungsten, machine is overloaded [production]
03:15 <LocalisationUpdate> ResourceLoader cache refresh completed at Sun Aug 31 03:14:41 UTC 2014 (duration 14m 40s) [production]
02:29 <LocalisationUpdate> completed (1.24wmf19) at 2014-08-31 02:28:42+00:00 [production]
02:18 <LocalisationUpdate> completed (1.24wmf18) at 2014-08-31 02:17:24+00:00 [production]
02:00 <ori> Stopped HHVM jobrunner and disabled Puppet on mw1053 due to bug 70177. [production]
2014-08-30 §
08:18 <godog> restart mailman on sodium, pending https://gerrit.wikimedia.org/r/#/c/156766/ [production]
08:01 <springle> Synchronized wmf-config/db-eqiad.php: repool db1071, warm up (duration: 00m 07s) [production]
06:33 <jgage> analytics1021 back in service after election [production]
06:14 <jgage> upgraded & rebooted analytics1021 [production]
06:01 <springle> Synchronized wmf-config/db-eqiad.php: return db1037 to normal load (duration: 00m 06s) [production]
05:03 <springle> Synchronized wmf-config/db-eqiad.php: repool es1001 (duration: 00m 06s) [production]
04:06 <springle> upgrade es1001 to mariadb 10 [production]
03:56 <springle> xtrabackup clone db1037 to db1071 [production]
03:55 <springle> Synchronized wmf-config/db-eqiad.php: reduce db1037 load while cloning (duration: 00m 06s) [production]
03:31 <springle> Synchronized wmf-config/db-eqiad.php: pool db1073, depool db1071 (duration: 00m 07s) [production]
03:19 <LocalisationUpdate> ResourceLoader cache refresh completed at Sat Aug 30 03:17:57 UTC 2014 (duration 17m 56s) [production]
02:33 <LocalisationUpdate> completed (1.24wmf19) at 2014-08-30 02:32:06+00:00 [production]
02:21 <LocalisationUpdate> completed (1.24wmf18) at 2014-08-30 02:20:39+00:00 [production]
2014-08-29 §
21:15 <ori> Synchronized wmf-config: I812c0bb6c: Scrap unused Twemproxy config files (duration: 00m 04s) [production]
21:12 <ori> updated /a/common to {{Gerrit|I812c0bb6c}}: Scrap unused Twemproxy config files [production]
21:03 <mutante> restarted uwsgi on tungsten [production]
20:49 <mutante> tungsten extremely busy, graphite down, logging in since 5 minutes :p [production]
20:48 <mutante> powercycling ms-be1006 - BUG: soft lockup - CPU#0 stuck ... [production]
19:19 <mutante> installing package upgrades on iron, bast1001 [production]
18:41 <aaron> Synchronized php-1.24wmf18/maintenance/findMissingFiles.php: 994d4a556a070156fd04fb4951492f10696cc63c (duration: 00m 03s) [production]
18:30 <ori> Synchronized php-1.24wmf19/resources/src/mediawiki.action/mediawiki.action.view.redirect.js: I19221a25a: mediawiki.action.view.redirect: Work around a IE 10+ HTML5 history API bug (duration: 00m 06s) [production]
18:30 <ori> Synchronized php-1.24wmf18/resources/src/mediawiki.action/mediawiki.action.view.redirect.js: I19221a25a: mediawiki.action.view.redirect: Work around a IE 10+ HTML5 history API bug (duration: 00m 07s) [production]
15:36 <hashar_> Jenkins: pooled a new slave 10.68.16.162 as wikidata-jenkins3 on behalf of addshore / wmde [production]
15:04 <_joe_> shutting down mw1163, filled RT 8243 for repair. [production]
14:54 <_joe_> re-enabled mw1130 [production]
14:41 <aude> Synchronized wmf-config/Wikibase.php: enable Wikibase badges css, follow up from last night deploy (duration: 00m 06s) [production]
14:22 <_joe_> syncing mw1130 [production]
14:06 <_joe_> disable mw1130 from the api pool whil it gets resynced [production]
12:30 <Krinkle> Running extensions/GlobalCssJs/removeOldManualUserPages.php per [[m:GlobalCssJs]] [production]
11:06 <springle> Synchronized wmf-config/db-eqiad.php: repool db1070 (duration: 00m 09s) [production]
08:04 <hashar> Jenkins: in the jenkins-job-builder-config branch 'cloudbees' has been merged in 'master'. Unifying CI and browser tests jobs!  \\O/ [production]
07:05 <_joe_> re-enabling puppet on the jobrunner, to check if the luasandbox fix works [production]
06:33 <springle> Synchronized wmf-config/db-eqiad.php: return db1056 to normal load (duration: 00m 06s) [production]
04:14 <LocalisationUpdate> ResourceLoader cache refresh completed at Fri Aug 29 04:13:03 UTC 2014 (duration 13m 2s) [production]
03:32 <springle> xtrabackup clone db1056 to db1070 [production]
03:31 <springle> Synchronized wmf-config/db-eqiad.php: reduce db1056 load while cloning (duration: 00m 06s) [production]