4751-4800 of 10000 results (35ms)
2014-09-02 §
03:16 <LocalisationUpdate> ResourceLoader cache refresh completed at Tue Sep 2 03:15:22 UTC 2014 (duration 15m 21s) [production]
02:53 <springle> restarted dbstore1002 mysqld for upgrade [production]
02:26 <LocalisationUpdate> completed (1.24wmf19) at 2014-09-02 02:25:36+00:00 [production]
02:15 <LocalisationUpdate> completed (1.24wmf18) at 2014-09-02 02:14:30+00:00 [production]
2014-09-01 §
23:00 <Krinkle> Running extensions/GlobalCssJs/removeOldManualUserPages.php per m:GlobalCssJs [production]
21:50 <ori> disabled gerrit account Caothu9669; spam [production]
19:12 <Reedy> Deleted php-1.24wmf[6-8] from apaches via dsh [production]
19:01 <reedy> Purged l10n cache for 1.24wmf13 [production]
19:01 <reedy> Purged l10n cache for 1.24wmf14 [production]
19:00 <reedy> Purged l10n cache for 1.24wmf15 [production]
18:59 <reedy> Purged l10n cache for 1.24wmf16 [production]
18:58 <reedy> Purged l10n cache for 1.24wmf17 [production]
16:44 <ottomata> removed some large slow query logs from elastic* nodes, need to look into this... [production]
12:10 <springle> Synchronized wmf-config/db-eqiad.php: repool db1044, take 2 (duration: 00m 06s) [production]
12:04 <springle> Synchronized wmf-config/db-eqiad.php: depool db1044 (duration: 00m 06s) [production]
11:31 <springle> Synchronized wmf-config/db-eqiad.php: pool db1044, warm up (duration: 00m 06s) [production]
09:31 <springle> Synchronized wmf-config/db-eqiad.php: repool db1027 (duration: 00m 07s) [production]
07:27 <godog> deploy latest ring to swift eqiad-prod [production]
07:11 <godog> powercycle ms-be1010 "cpu soft lockup" on console [production]
05:28 <springle> xtrabackup clone db1027 to db1044 [production]
05:26 <springle> Synchronized wmf-config/db-eqiad.php: depool db1027 while cloning (duration: 00m 07s) [production]
03:15 <LocalisationUpdate> ResourceLoader cache refresh completed at Mon Sep 1 03:14:22 UTC 2014 (duration 14m 21s) [production]
02:28 <LocalisationUpdate> completed (1.24wmf19) at 2014-09-01 02:27:39+00:00 [production]
02:17 <LocalisationUpdate> completed (1.24wmf18) at 2014-09-01 02:16:18+00:00 [production]
2014-08-31 §
19:56 <hashar> Jenkins updated HHVM to (3.3-dev+20140728+wmf5) over (3.3-dev+20140728+wmf4) [production]
14:38 <bblack> restarted apache on strontium [production]
14:34 <bblack> restarted apache on tungsten, machine is overloaded [production]
03:15 <LocalisationUpdate> ResourceLoader cache refresh completed at Sun Aug 31 03:14:41 UTC 2014 (duration 14m 40s) [production]
02:29 <LocalisationUpdate> completed (1.24wmf19) at 2014-08-31 02:28:42+00:00 [production]
02:18 <LocalisationUpdate> completed (1.24wmf18) at 2014-08-31 02:17:24+00:00 [production]
02:00 <ori> Stopped HHVM jobrunner and disabled Puppet on mw1053 due to bug 70177. [production]
2014-08-30 §
08:18 <godog> restart mailman on sodium, pending https://gerrit.wikimedia.org/r/#/c/156766/ [production]
08:01 <springle> Synchronized wmf-config/db-eqiad.php: repool db1071, warm up (duration: 00m 07s) [production]
06:33 <jgage> analytics1021 back in service after election [production]
06:14 <jgage> upgraded & rebooted analytics1021 [production]
06:01 <springle> Synchronized wmf-config/db-eqiad.php: return db1037 to normal load (duration: 00m 06s) [production]
05:03 <springle> Synchronized wmf-config/db-eqiad.php: repool es1001 (duration: 00m 06s) [production]
04:06 <springle> upgrade es1001 to mariadb 10 [production]
03:56 <springle> xtrabackup clone db1037 to db1071 [production]
03:55 <springle> Synchronized wmf-config/db-eqiad.php: reduce db1037 load while cloning (duration: 00m 06s) [production]
03:31 <springle> Synchronized wmf-config/db-eqiad.php: pool db1073, depool db1071 (duration: 00m 07s) [production]
03:19 <LocalisationUpdate> ResourceLoader cache refresh completed at Sat Aug 30 03:17:57 UTC 2014 (duration 17m 56s) [production]
02:33 <LocalisationUpdate> completed (1.24wmf19) at 2014-08-30 02:32:06+00:00 [production]
02:21 <LocalisationUpdate> completed (1.24wmf18) at 2014-08-30 02:20:39+00:00 [production]
2014-08-29 §
21:15 <ori> Synchronized wmf-config: I812c0bb6c: Scrap unused Twemproxy config files (duration: 00m 04s) [production]
21:12 <ori> updated /a/common to {{Gerrit|I812c0bb6c}}: Scrap unused Twemproxy config files [production]
21:03 <mutante> restarted uwsgi on tungsten [production]
20:49 <mutante> tungsten extremely busy, graphite down, logging in since 5 minutes :p [production]
20:48 <mutante> powercycling ms-be1006 - BUG: soft lockup - CPU#0 stuck ... [production]
19:19 <mutante> installing package upgrades on iron, bast1001 [production]