3501-3550 of 10000 results (27ms)
2020-07-28 ยง
14:02 <jayme@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
14:02 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1148', diff saved to https://phabricator.wikimedia.org/P12085 and previous config saved to /var/cache/conftool/dbconfig/20200728-140220-marostegui.json [production]
14:02 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1148', diff saved to https://phabricator.wikimedia.org/P12084 and previous config saved to /var/cache/conftool/dbconfig/20200728-140207-marostegui.json [production]
14:00 <jayme@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
13:59 <jayme@cumin1001> START - Cookbook sre.hosts.downtime [production]
13:58 <jayme@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
13:58 <moritzm> installing perl security updates [production]
13:56 <jayme@cumin1001> START - Cookbook sre.hosts.downtime [production]
13:56 <jayme@cumin1001> START - Cookbook sre.hosts.downtime [production]
13:55 <jayme@cumin1001> conftool action : set/pooled=inactive; selector: name=wtp1041.eqiad.wmnet [production]
13:55 <jayme@cumin1001> conftool action : set/pooled=yes; selector: name=wtp1037.eqiad.wmnet [production]
13:50 <godog> remove stale ipvs thanos-query service on port 80 [production]
13:39 <jayme@cumin1001> conftool action : set/pooled=inactive; selector: name=wtp1040.eqiad.wmnet [production]
13:38 <jayme@cumin1001> conftool action : set/pooled=yes; selector: name=wtp1036.eqiad.wmnet [production]
13:38 <jayme@cumin1001> conftool action : set/pooled=inactive; selector: name=wtp1039.eqiad.wmnet [production]
13:37 <jayme@cumin1001> conftool action : set/pooled=yes; selector: name=wtp1035.eqiad.wmnet [production]
13:37 <jayme@cumin1001> conftool action : set/pooled=inactive; selector: name=wtp1038.eqiad.wmnet [production]
13:36 <jayme@cumin1001> conftool action : set/pooled=yes; selector: name=wtp1034.eqiad.wmnet [production]
13:29 <godog> roll-restart pybal on eqiad lvs low-traffic to change port for thanos-query [production]
13:25 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db1075', diff saved to https://phabricator.wikimedia.org/P12083 and previous config saved to /var/cache/conftool/dbconfig/20200728-132520-marostegui.json [production]
13:20 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db1075 with less weight', diff saved to https://phabricator.wikimedia.org/P12082 and previous config saved to /var/cache/conftool/dbconfig/20200728-132023-marostegui.json [production]
13:09 <godog> roll-restart pybal on lvs low-traffic to apply thanos-query changes [production]
13:04 <XioNoX> standardize cr3-esams interfaces [production]
13:03 <liw@deploy1001> rebuilt and synchronized wikiversions files: group0 wikis to 1.36.0-wmf.2 [production]
12:41 <XioNoX> standardize cr2-esams interfaces [production]
12:38 <jayme@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
12:36 <jayme@cumin1001> START - Cookbook sre.hosts.downtime [production]
12:35 <jayme@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
12:33 <jayme@cumin1001> START - Cookbook sre.hosts.downtime [production]
12:32 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1075', diff saved to https://phabricator.wikimedia.org/P12081 and previous config saved to /var/cache/conftool/dbconfig/20200728-123201-marostegui.json [production]
12:28 <jayme@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
12:26 <jayme@cumin1001> START - Cookbook sre.hosts.downtime [production]
12:26 <jayme@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
12:24 <jayme@cumin1001> START - Cookbook sre.hosts.downtime [production]
12:17 <jayme@cumin1001> conftool action : set/pooled=inactive; selector: name=wtp1037.eqiad.wmnet [production]
12:14 <jayme@cumin1001> conftool action : set/pooled=inactive; selector: name=wtp1036.eqiad.wmnet [production]
12:08 <jayme@cumin1001> conftool action : set/pooled=inactive; selector: name=wtp1035.eqiad.wmnet [production]
12:07 <jayme@cumin1001> conftool action : set/pooled=yes; selector: name=wtp1032.eqiad.wmnet [production]
12:07 <jayme@cumin1001> conftool action : set/pooled=yes; selector: name=wtp1033.eqiad.wmnet [production]
12:05 <jayme@cumin1001> conftool action : set/pooled=yes; selector: name=wtp1031.eqiad.wmnet [production]
12:04 <jayme@cumin1001> conftool action : set/pooled=inactive; selector: name=wtp1034.eqiad.wmnet [production]
12:04 <tstarling@deploy1001> Synchronized wmf-config/CommonSettings.php: disabling lilypond rendering in Score again due to error running gs (duration: 01m 05s) [production]
11:56 <tstarling@deploy1001> Synchronized wmf-config/CommonSettings.php: re-enabling Score in safe mode (duration: 01m 04s) [production]
11:50 <Urbanecm> EU B&C window done [production]
11:49 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: 1a5672628b82709350ca74bb784197e7ff5fdc19: Add Turkish powered by MW and Wikimedia project icons (T257732) (duration: 00m 59s) [production]
11:46 <urbanecm@deploy1001> Synchronized static/images/footer/: 1a5672628b82709350ca74bb784197e7ff5fdc19: Add Turkish powered by MW and Wikimedia project icons (T257732) (duration: 01m 01s) [production]
11:43 <urbanecm@deploy1001> Synchronized static/images: df9b9acf0876dad9b11d5641fe6fa174c7066f8b: Move footer logos to /static/images/footer (T257732) (duration: 01m 02s) [production]
11:38 <marostegui> Deploy schema change on s3 codfw, this will generate lag on codfw T256682 [production]
11:38 <ema> A:cp-text varnish ban pt.wikiversity.org T256750 [production]
11:37 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: df9b9acf0876dad9b11d5641fe6fa174c7066f8b: Move footer logos to /static/images/footer (T257732) (duration: 00m 58s) [production]