3051-3100 of 10000 results (34ms)
2015-09-14 §
10:04 <jynus> db1029 (x1-master) temporarily saturated by connections- flow was unresponsive for 10 minutes; migration partially aborted [production]
09:08 <jynus> applying schema change to flowdb [production]
08:52 <godog> rename cassandra test cluster and restart [production]
08:44 <godog> silence mendelevium for today, status unclear T111532 [production]
08:30 <jynus> endinf profiling and executing pt-query-digest on db1043 [ETA:4h] [production]
07:52 <godog> reboot ms-be1010 to pick up disk ordering change [production]
04:47 <l10nupdate@tin> ResourceLoader cache refresh completed at Mon Sep 14 04:47:58 UTC 2015 (duration 47m 57s) [production]
02:29 <l10nupdate@tin> LocalisationUpdate completed (1.26wmf22) at 2015-09-14 02:29:48+00:00 [production]
02:26 <l10nupdate@tin> Synchronized php-1.26wmf22/cache/l10n: l10nupdate for 1.26wmf22 (duration: 06m 59s) [production]
01:31 <Krinkle> mwscript deleteEqualMessages.php --wiki sqwiki [production]
2015-09-13 §
06:02 <l10nupdate@tin> ResourceLoader cache refresh completed at Sun Sep 13 06:02:52 UTC 2015 (duration 2m 51s) [production]
02:40 <l10nupdate@tin> LocalisationUpdate completed (1.26wmf22) at 2015-09-13 02:40:43+00:00 [production]
02:34 <l10nupdate@tin> Synchronized php-1.26wmf22/cache/l10n: l10nupdate for 1.26wmf22 (duration: 10m 13s) [production]
2015-09-12 §
20:15 <ori> Rolling back Echo to 1.26wmf21 branch on mw1017 (testwiki) to measure increase in render-blocking CSS size [production]
19:21 <urandom> performing Cassandra repair on restbase1002 (nodetool repair -pr) [production]
14:50 <jynus> phab.wmfusercontent.org has been temporarily switched to phab.wikivoyage.org due to cert issues [production]
04:52 <l10nupdate@tin> ResourceLoader cache refresh completed at Sat Sep 12 04:52:01 UTC 2015 (duration 52m 0s) [production]
02:35 <l10nupdate@tin> LocalisationUpdate completed (1.26wmf22) at 2015-09-12 02:35:36+00:00 [production]
02:32 <l10nupdate@tin> Synchronized php-1.26wmf22/cache/l10n: l10nupdate for 1.26wmf22 (duration: 06m 54s) [production]
2015-09-11 §
21:21 <hashar> shutdown nodepool on labnodepool1001.eqiad.wmnet until monday [production]
18:01 <legoktm@tin> Synchronized php-1.26wmf22/extensions/Echo/: Echo regression fixes #2 (duration: 00m 12s) [production]
16:43 <krinkle@tin> Synchronized php-1.26wmf22/resources/src/mediawiki/mediawiki.js: T112232 (duration: 00m 12s) [production]
16:37 <legoktm@tin> Synchronized php-1.26wmf22/extensions/Echo/: Echo regression backports (duration: 00m 12s) [production]
16:35 <legoktm@tin> Synchronized php-1.26wmf22/resources/src/mediawiki/mediawiki.js: resourceloader: Document internal mw.loader#jobs property (again) (duration: 00m 13s) [production]
16:33 <legoktm> ssh: connect to host mw1156.eqiad.wmnet port 22: Connection timed out [production]
16:32 <paravoid> powercycling mw1156, multiple kernel backtraces in console output [production]
16:32 <legoktm@tin> Synchronized php-1.26wmf22/resources/src/mediawiki/mediawiki.js: resourceloader: Document internal mw.loader#jobs property (duration: 01m 07s) [production]
16:15 <cmjohnson1> mw1031 rebooting for f/w update [production]
16:07 <bblack> enabled LRO+GRO on lvs200[123], starting pybal there again ([456] testing looks good so far) [production]
15:44 <bblack> enabled LRO+GRO on lvs200[456] (backups). Stopping pybal on lvs200[123] to test... [production]
15:11 <cmjohnson1> swapping pem2 cr2-eqiad [production]
10:03 <jynus> starting nodepool in labnodepool1001 [production]
09:21 <jynus> starting profiling of phabricator db (db1043). Very low overhead. [production]
06:03 <l10nupdate@tin> ResourceLoader cache refresh completed at Fri Sep 11 06:03:00 UTC 2015 (duration 2m 59s) [production]
02:41 <l10nupdate@tin> LocalisationUpdate completed (1.26wmf22) at 2015-09-11 02:41:24+00:00 [production]
02:34 <l10nupdate@tin> Synchronized php-1.26wmf22/cache/l10n: l10nupdate for 1.26wmf22 (duration: 11m 18s) [production]
01:16 <ori@tin> Synchronized php-1.26wmf22/extensions/TitleBlacklist: 9bf13dbe0b, 3203b045f7 (duration: 00m 12s) [production]
2015-09-10 §
23:52 <krenair@tin> Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/237064/ (duration: 00m 11s) [production]
23:47 <krenair@tin> Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/237056/ (duration: 00m 11s) [production]
23:13 <krenair@tin> Synchronized wmf-config/wikitech.php: https://gerrit.wikimedia.org/r/221825 (duration: 00m 13s) [production]
23:04 <krenair@tin> Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/224771 (duration: 00m 12s) [production]
21:12 <legoktm@tin> Synchronized php-1.26wmf22/extensions/Echo/modules: Align popup footer buttons to take 50% width each (duration: 00m 15s) [production]
20:50 <jynus@tin> Synchronized wmf-config/db-eqiad.php: depool es1001; increase weight of es1015 and es1019 (duration: 00m 19s) [production]
20:47 <ottomata> restarting eventlogging with 12 client side processors on eventlog1001 [production]
20:31 <ottomata> turning off varnishncsa eventlogging eventlistener instances on frontend caches, it is now superseded by varnishkafka [production]
20:28 <mutante> killed/restarted ganglia aggregator process for mobile-cache, upload cache, misc esams ... [production]
20:22 <jynus> last SCAP failed on 266/466 hosts [production]
20:21 <mutante> killed/restarted ganglia aggregator process for text-caches esams on hooft [production]
20:17 <yurik> deployed kartotherian [production]
20:08 <jynus@tin> Synchronized wmf-config/db-eqiad.php: Depool es1001; increase weight of es1015 and es1019 (duration: 00m 11s) [production]