4551-4600 of 10000 results (30ms)
2020-05-06 §
08:02 <_joe_> restarted php-fpm with tweaked parameters on mw1407, now briefly pooling for traffic (T99740) [production]
07:38 <kormat@cumin1001> dbctl commit (dc=all): 'Set es1023 (es5 master) to 0 weight after reimaging es1024 T250666', diff saved to https://phabricator.wikimedia.org/P11158 and previous config saved to /var/cache/conftool/dbconfig/20200506-073856-kormat.json [production]
07:32 <vgutierrez> downgrade to ATS 8.0.7-1wm3 on cp4026, cp4031, cp5006 and cp5011 [production]
06:00 <elukey> powercycle analytics1060 - host stuck - T251973 [production]
05:03 <marostegui@cumin1001> dbctl commit (dc=all): 'Pool db1103:3314 in vslow on s4 while db1121 is out T250055', diff saved to https://phabricator.wikimedia.org/P11157 and previous config saved to /var/cache/conftool/dbconfig/20200506-050340-marostegui.json [production]
05:02 <marostegui> Deploy schema change on db1121 [production]
2020-05-05 §
23:44 <catrope@deploy1001> Synchronized wmf-config/flaggedrevs.php: Restore the reviewer group on fawiki (T249643) (duration: 01m 06s) [production]
23:22 <crusnov@deploy1001> Finished deploy [netbox/deploy@03cc2dd]: Netbox upgrade to 2.8.1 (part3) (duration: 00m 11s) [production]
23:22 <crusnov@deploy1001> Started deploy [netbox/deploy@03cc2dd]: Netbox upgrade to 2.8.1 (part3) [production]
23:22 <crusnov@deploy1001> Finished deploy [netbox/deploy@03cc2dd]: Netbox upgrade to 2.8.1 (part1) (duration: 01m 14s) [production]
23:21 <crusnov@deploy1001> Started deploy [netbox/deploy@03cc2dd]: Netbox upgrade to 2.8.1 (part1) [production]
23:21 <crusnov@deploy1001> Finished deploy [netbox/deploy@03cc2dd]: Netbox upgrade to 2.8.1 (part1) (duration: 01m 20s) [production]
23:20 <crusnov@deploy1001> Started deploy [netbox/deploy@03cc2dd]: Netbox upgrade to 2.8.1 (part1) [production]
22:00 <reedy@deploy1001> Synchronized php-1.35.0-wmf.31/includes/parser/CoreParserFunctions.php: T251952 take 2 (duration: 01m 06s) [production]
21:57 <reedy@deploy1001> Synchronized php-1.35.0-wmf.31/includes/parser/CoreParserFunctions.php: T251952 (duration: 01m 05s) [production]
21:55 <reedy@deploy1001> Synchronized php-1.35.0-wmf.31/includes/specials/SpecialNewpages.php: T251950 (duration: 01m 06s) [production]
20:02 <herron> added ryankemper to wmf and ops ldap groups T251572 [production]
19:38 <mforns@deploy1001> Finished deploy [analytics/refinery@6868fc0] (thin): Regular analytics weekly train THIN [analytics/refinery@ebd624a5e4c88ac6983387d4603971f8a326ee7c] (duration: 00m 08s) [production]
19:38 <mforns@deploy1001> Started deploy [analytics/refinery@6868fc0] (thin): Regular analytics weekly train THIN [analytics/refinery@ebd624a5e4c88ac6983387d4603971f8a326ee7c] [production]
19:38 <mforns@deploy1001> Finished deploy [analytics/refinery@6868fc0]: Regular analytics weekly train (2nd try) [analytics/refinery@ebd624a5e4c88ac6983387d4603971f8a326ee7c] (duration: 25m 18s) [production]
19:19 <brennen@deploy1001> rebuilt and synchronized wikiversions files: group0 wikis to 1.35.0-wmf.31 [production]
19:13 <mforns@deploy1001> Started deploy [analytics/refinery@6868fc0]: Regular analytics weekly train (2nd try) [analytics/refinery@ebd624a5e4c88ac6983387d4603971f8a326ee7c] [production]
19:12 <brennen@deploy1001> Finished scap: testwikis wikis to 1.35.0-wmf.31 (duration: 97m 23s) [production]
19:02 <brennen> train status: 1.35.0-wmf.31: presently pressing enter through scap-cdb-rebuild; at 8% (T249963, T223287) [production]
18:39 <cdanis> depool mw2221 for some manual testing [production]
18:35 <mforns@deploy1001> Finished deploy [analytics/refinery@ebd624a] (thin): Regular analytics weekly train THIN [analytics/refinery@ebd624a5e4c88ac6983387d4603971f8a326ee7c] (duration: 00m 09s) [production]
18:35 <mforns@deploy1001> Started deploy [analytics/refinery@ebd624a] (thin): Regular analytics weekly train THIN [analytics/refinery@ebd624a5e4c88ac6983387d4603971f8a326ee7c] [production]
18:34 <mforns@deploy1001> Finished deploy [analytics/refinery@ebd624a]: Regular analytics weekly train [analytics/refinery@ebd624a5e4c88ac6983387d4603971f8a326ee7c] (duration: 18m 54s) [production]
18:15 <mforns@deploy1001> Started deploy [analytics/refinery@ebd624a]: Regular analytics weekly train [analytics/refinery@ebd624a5e4c88ac6983387d4603971f8a326ee7c] [production]
17:35 <brennen@deploy1001> Started scap: testwikis wikis to 1.35.0-wmf.31 [production]
16:48 <brennen> 1.35.0-wmf.31 was branched at 4d3fed31a435e7bd24925a154f89a9407670986d for T249963 [production]
16:34 <brennen> triggering branch cut for 1.35.0-wmf.31 (T249963) via https://releases-jenkins.wikimedia.org/job/MediaWiki%20Train%20Branch%20Cut/build?delay=0sec [production]
16:18 <brennen> notice: planning branch cut for 1.35.0-wmf.31 (T249963) at 16:30 UTC [production]
15:47 <cstone> SmashPig revision changed from 8c30ed7fe5 to cd1a49da5f [production]
15:38 <kormat@cumin1001> dbctl commit (dc=all): 'Repool es1024 to 100% after reimaging T250666', diff saved to https://phabricator.wikimedia.org/P11153 and previous config saved to /var/cache/conftool/dbconfig/20200505-153843-kormat.json [production]
15:26 <elukey@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
15:24 <elukey@cumin1001> START - Cookbook sre.hosts.downtime [production]
15:03 <elukey@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
15:00 <elukey@cumin1001> START - Cookbook sre.hosts.downtime [production]
14:58 <hnowlan@deploy1001> Finished deploy [changeprop/deploy@6c65779]: Enabling on_transclusion_update on k8s, disabling on scb (duration: 01m 31s) [production]
14:56 <hnowlan@deploy1001> Started deploy [changeprop/deploy@6c65779]: Enabling on_transclusion_update on k8s, disabling on scb [production]
14:44 <hnowlan@deploy1001> helmfile [EQIAD] Ran 'sync' command on namespace 'changeprop' for release 'production' . [production]
14:43 <hnowlan@deploy1001> helmfile [CODFW] Ran 'sync' command on namespace 'changeprop' for release 'production' . [production]
14:31 <kormat@cumin1001> dbctl commit (dc=all): 'Repool es1024 to 75% after reimaging T250666', diff saved to https://phabricator.wikimedia.org/P11149 and previous config saved to /var/cache/conftool/dbconfig/20200505-143158-kormat.json [production]
13:46 <akosiaris> deploy cxserver chart 0.0.15 to staging, codfw, eqiad. T219921 [production]
13:45 <akosiaris@deploy1001> helmfile [EQIAD] Ran 'sync' command on namespace 'cxserver' for release 'production' . [production]
13:41 <akosiaris@deploy1001> helmfile [CODFW] Ran 'sync' command on namespace 'cxserver' for release 'production' . [production]
13:41 <hashar> Updated Jenkins job https://integration.wikimedia.org/ci/job/operations-puppet-catalog-compiler to have it defined in JJB # T97513 [production]
13:36 <akosiaris@deploy1001> helmfile [STAGING] Ran 'sync' command on namespace 'cxserver' for release 'staging' . [production]
13:18 <vgutierrez> upgrade ATS to version 8.1 () on cp4026, cp4032, cp5006 and cp5011 [production]