4201-4250 of 10000 results (64ms)
2018-10-27 §
03:41 <SMalyshev> depool wdqs1003 again to let it catch up some more [production]
02:35 <smalyshev@deploy1001> Finished deploy [wdqs/wdqs@7eeede7]: Re-deploy Updater to deal with performance issues (duration: 00m 38s) [production]
02:34 <smalyshev@deploy1001> Started deploy [wdqs/wdqs@7eeede7]: Re-deploy Updater to deal with performance issues [production]
02:34 <smalyshev@deploy1001> Finished deploy [wdqs/wdqs@e9392f4]: Re-deploy Updater to deal with performance issues (duration: 00m 05s) [production]
02:33 <smalyshev@deploy1001> Started deploy [wdqs/wdqs@e9392f4]: Re-deploy Updater to deal with performance issues [production]
00:00 <mutante> icinga1001 - using wmf-auto-reimage to reinstall gets stuck at initial puppet run after reboot - Still waiting for Puppet after 105.0 minutes - aborting on cumin, loggin in directly and manually running puppet (T202782 T208100) [production]
2018-10-26 §
22:54 <mutante> sodium - attempted to replace broken disk for RAID - did not go well [production]
21:38 <ejegg> updated fundraising CiviCRM from 97506677e8 to 65130ef3dd [production]
21:36 <aaron@deploy1001> Synchronized php-1.33.0-wmf.1/includes: 86c0b56b0d1bf66073fafb9bc00bafb87d2e3b9c (duration: 01m 14s) [production]
21:34 <aaron@deploy1001> Synchronized php-1.33.0-wmf.1/autoload.php: 86c0b56b0d1bf66073fafb9bc00bafb87d2e3b9c (duration: 00m 52s) [production]
21:33 <aaron@deploy1001> Synchronized php-1.33.0-wmf.1/tests: 86c0b56b0d1bf66073fafb9bc00bafb87d2e3b9c (duration: 01m 08s) [production]
20:03 <mutante> icinga1001 - disabled puppet, changed: check_result_reaper_frequency=2 ; max_check_result_reaper_time=10 to test if it lowers latency (T208066) [production]
19:40 <chasemp> remove 2fa for charlottepotero and cwd users in phab (so they can readd) [production]
19:09 <SMalyshev> repooled wdqs1003 - looks like it caught up now [production]
17:18 <SMalyshev> depool wdqs1003 again to let it catch up some more [production]
16:10 <ejegg> updated payments-wiki to 34506ce636 [production]
15:32 <elukey> rolling restart of all prometheus-mcrouter-exporters on app/api servers - metrics not reported after the last mcrouter restart [production]
15:20 <gehel> repooling wdqs1003, other nodes are starting to lag as well [production]
14:56 <reedy@deploy1001> Synchronized wmf-config/CommonSettings.php: Prevent sysops from disabling 2FA for other users as part of upcoming feature (duration: 00m 53s) [production]
13:02 <gehel> depool wdqs1003 to catch up on updates [production]
07:51 <bawolff> adjust patch T207916 [production]
07:16 <bawolff@deploy1001> Synchronized wmf-config/InitialiseSettings.php: T207916 13b993ab9f - auth log on in arwiki (duration: 00m 54s) [production]
06:51 <bawolff@deploy1001> Synchronized wmf-config/InitialiseSettings.php: T207916 13b993ab9f - auth log on in group1 (duration: 00m 54s) [production]
06:33 <moritzm> uploaded openjdk-8 backport for recent Java 8 security updates to apt.wikimedia.org/jessie-wikimedia [production]
06:24 <bawolff@deploy1001> Synchronized wmf-config/InitialiseSettings.php: ecf579e9f9 - T207916 - enable auth log group0 (duration: 00m 55s) [production]
06:01 <bawolff> adjust patch for T207916 [production]
05:07 <bawolff@deploy1001> Synchronized wmf-config/InitialiseSettings.php: d3b2c346 T207916 (duration: 00m 55s) [production]
04:58 <SMalyshev> depooled wdqs1003 again, let's see if it helps it catch up now [production]
04:12 <bawolff> deploy patch T207916 [production]
03:19 <bawolff@deploy1001> Synchronized wmf-config/InitialiseSettings.php: bc9b863e - T207900 - enable CSP report only for users w/session everywhere (duration: 00m 55s) [production]
03:10 <bawolff@deploy1001> Synchronized wmf-config/InitialiseSettings.php: 745d0b61 - T207900 - enable CSP report only for users w/session enwiki (duration: 00m 55s) [production]
03:01 <bawolff@deploy1001> Synchronized wmf-config/CommonSettings.php: T207900 - Add wikimedia.org (no subdomain) to allow list for math (duration: 00m 53s) [production]
02:54 <bawolff@deploy1001> Synchronized wmf-config/InitialiseSettings.php: 745d0b61 - T207900 - enable CSP report only for users w/session enwiki (duration: 00m 53s) [production]
02:43 <bawolff@deploy1001> Synchronized wmf-config/InitialiseSettings.php: a8aa9d6aae - T207900 - enable CSP report only for users w/session fawiki, frwiki, svwiki, eswiki, ruwiki, zhwiki, dewiki (duration: 00m 56s) [production]
02:22 <bawolff@deploy1001> Synchronized wmf-config/InitialiseSettings.php: d743db261 - T207900 - enable CSP report only for users w/session arwiki (duration: 00m 54s) [production]
02:05 <bawolff@deploy1001> Synchronized wmf-config/InitialiseSettings.php: bd55034d122 - T207900 - enable CSP report only for users w/session on medium wikis (duration: 00m 55s) [production]
01:41 <bawolff@deploy1001> Synchronized wmf-config/InitialiseSettings.php: T207900 (b74911f6201) enable csp users with session all group1 wikis (duration: 00m 55s) [production]
01:28 <bawolff@deploy1001> Synchronized wmf-config/CommonSettings.php: Ia518c031 (duration: 00m 55s) [production]
01:26 <smalyshev@deploy1001> Finished deploy [wdqs/wdqs@e9392f4]: Re-deploy Updater to deal with performance issues (duration: 31m 28s) [production]
01:21 <bawolff@deploy1001> Synchronized wmf-config/CommonSettings.php: T207900 - deploy CSP to people with session on enwikiquote (duration: 00m 54s) [production]
01:19 <bawolff@deploy1001> Synchronized wmf-config/InitialiseSettings.php: T207900 - deploy CSP to people with session on enwikiquote (duration: 00m 55s) [production]
00:56 <twentyafterfour> twentyafterfour@deploy1001 rebuilt and synchronized wikiversions files: group2 wikis to 1.33.0-wmf.1 refs T206655 [production]
00:55 <smalyshev@deploy1001> Started deploy [wdqs/wdqs@e9392f4]: Re-deploy Updater to deal with performance issues [production]
00:54 <twentyafterfour@deploy1001> rebuilt and synchronized wikiversions files: group2 wikis to 1.33.0-wmf.1 refs T206655 [production]
00:39 <jforrester@deploy1001> Synchronized wmf-config/Wikibase.php: Post-SWAT: De-register all entities on WBMI installations calling themselves Commons I09e066f2 (duration: 00m 56s) [production]
00:32 <ejegg> updated payments-wiki from f5999d963d to 57e8438e9c [production]
00:18 <addshore@deploy1001> Synchronized wmf-config/Wikibase.php: Define and specify lexeme NS for wikidatawiki (duration: 00m 55s) [production]
2018-10-25 §
23:59 <twentyafterfour@deploy1001> Synchronized php-1.33.0-wmf.1/includes/parser/Parser.php: deploy https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/469799/ refs T208000 (duration: 00m 56s) [production]
23:57 <twentyafterfour> deploying https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/469799/ [production]
23:53 <jforrester@deploy1001> Synchronized php-1.32.0-wmf.26/extensions/CentralNotice/special/SpecialCentralNotice.php: SWAT Sync versions of SpecialCentralNotice to avoid dirty repo checkout T208004 (duration: 00m 56s) [production]