5351-5400 of 10000 results (104ms)
2020-02-14 §
08:46 <moritzm> installing 4.19.98 kernel update on Buster systems [production]
08:06 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db1107 with weight 100 for 10.4 testing - T242702', diff saved to https://phabricator.wikimedia.org/P10408 and previous config saved to /var/cache/conftool/dbconfig/20200214-080600-marostegui.json [production]
06:51 <vgutierrez> updating puppet compiler facts [production]
01:27 <dpifke@deploy1001> Finished deploy [performance/navtiming@2eec00a]: (no justification provided) (duration: 00m 05s) [production]
01:27 <dpifke@deploy1001> Started deploy [performance/navtiming@2eec00a]: (no justification provided) [production]
00:53 <jforrester@deploy1001> Synchronized wmf-config/InitialiseSettings.php: T245202 cirrus: Move all move_like traffic to codfw (duration: 01m 02s) [production]
00:51 <jforrester@deploy1001> Synchronized wmf-config/PoolCounterSettings.php: T245202 cirrus: Increase the pool counter limits a bit (duration: 01m 05s) [production]
2020-02-13 §
22:13 <jeh> running filesystem tests on cloudvirt1024 T241884 [production]
21:42 <otto@deploy1001> helmfile [EQIAD] Ran 'apply' command on namespace 'eventgate-logging-external' for release 'canary' . [production]
21:41 <otto@deploy1001> helmfile [EQIAD] Ran 'apply' command on namespace 'eventgate-logging-external' for release 'production' . [production]
21:40 <jbond42> refresh facts on compilers [production]
21:38 <otto@deploy1001> helmfile [CODFW] Ran 'apply' command on namespace 'eventgate-logging-external' for release 'canary' . [production]
21:37 <otto@deploy1001> helmfile [CODFW] Ran 'apply' command on namespace 'eventgate-logging-external' for release 'production' . [production]
21:35 <ottomata> deploying production and canary releases for eventgate-logging-external (and destroying the 'logging-external' release) (safe because eventgate-logging-external is not in use) - T245203 [production]
21:29 <otto@deploy1001> helmfile [STAGING] Ran 'apply' command on namespace 'eventgate-logging-external' for release 'canary' . [production]
21:28 <otto@deploy1001> helmfile [STAGING] Ran 'apply' command on namespace 'eventgate-logging-external' for release 'production' . [production]
20:33 <marxarelli> rollback to group1 due to 500 spike (2k/min) (T233867) [production]
20:32 <dduvall@deploy1001> rebuilt and synchronized wikiversions files: (no justification provided) [production]
20:30 <marxarelli> varnish 500 spike. rolling back [production]
20:20 <gehel> restarting blazegraph + updater on wdqs2006 [production]
20:19 <dduvall@deploy1001> rebuilt and synchronized wikiversions files: all wikis to 1.35.0-wmf.19 [production]
19:44 <jforrester@deploy1001> Synchronized php-1.35.0-wmf.19/includes/api/ApiRollback.php: T245159 ApiRollback: Properly deal with UserIdentity (duration: 01m 04s) [production]
19:20 <jforrester@deploy1001> Synchronized php-1.35.0-wmf.19/includes/resourceloader/ResourceLoaderSkinModule.php: T245182 ResourceLoaderSkinModule: Don't hard-deprecate wgLogoHD just now (duration: 01m 03s) [production]
19:17 <jforrester@deploy1001> Synchronized wmf-config/InitialiseSettings.php: T219534 Add new MLR models for Cirrus on zh/ja/kowiki (duration: 01m 03s) [production]
19:10 <moritzm> installing e2fsprogs security updates [production]
18:48 <bblack> ns1.wikimedia.org - re-routing back to authdns2001 instead of dns2002 on cr[12]-codfw - T242017 [production]
18:38 <bblack> authdns2001 - reboot - T242017 [production]
18:36 <bblack> ns1.wikimedia.org - re-routing from authdns2001 to dns2002 on cr[12]-codfw - T242017 [production]
18:09 <krinkle@deploy1001> Synchronized wmf-config/CommonSettings.php: I9d0c8af3c577 (duration: 01m 06s) [production]
18:00 <krinkle@deploy1001> Synchronized wmf-config/etcd.php: Iae1f45896 (duration: 01m 06s) [production]
17:59 <volans> downtimed mgmt in eqiad for 1h [production]
17:58 <krinkle@deploy1001> Synchronized wmf-config/CommonSettings.php: Iae1f45896 (duration: 01m 08s) [production]
17:49 <krinkle@deploy1001> Synchronized wmf-config/etcd.php: Ibfca686f681 (duration: 01m 06s) [production]
17:41 <krinkle@deploy1001> Synchronized wmf-config/etcd.php: Iefff596955e (duration: 01m 08s) [production]
17:40 <krinkle@deploy1001> Synchronized wmf-config/CommonSettings.php: Iefff596955e (duration: 01m 06s) [production]
17:35 <krinkle@deploy1001> Synchronized wmf-config/etcd.php: I2e4fb0c086de0f8ac (duration: 01m 06s) [production]
17:32 <krinkle@deploy1001> Synchronized wmf-config/CommonSettings.php: I2e4fb0c086de0f8ac (duration: 01m 06s) [production]
17:10 <jforrester@deploy1001> Synchronized wmf-config/CommonSettings.php: No-op (code style only) deploy sync (duration: 01m 07s) [production]
17:09 <jforrester@deploy1001> sync aborted: wmf-config/CommonSettings.php No-op (code style only) deploy sync (duration: 00m 04s) [production]
17:09 <jforrester@deploy1001> Started scap: wmf-config/CommonSettings.php No-op (code style only) deploy sync [production]
16:32 <robh> ps1-a8-codfw.mgmt.codfw.wmnet firmware upgraded via T245164 [production]
16:28 <papaul> rebooting elastic2043 for firmware upgrade [production]
16:22 <gehel> canceled the restart of elastic2043 - T243715 [production]
16:21 <gehel> restarting elastic2043 - T243715 [production]
16:10 <_joe_> depooling/repooling mw1240 [production]
16:02 <_joe_> pooled mw1238 again [production]
15:59 <_joe_> depooling mw1238 for analysis [production]
15:42 <vgutierrez> rolling restart of ats-be on esams - T170567 [production]
15:38 <vgutierrez> disable allow_half_open on ats-tls @ cp4031 - T236458 [production]
15:27 <vgutierrez> turning on TLSv1.3 between ats-be and applayer in cp30[51-52] - T170567 [production]