3401-3450 of 10000 results (36ms)
2020-05-20 ยง
12:33 <ladsgroup@deploy1001> Synchronized dblists: Creating Wiktionary Konkani (gomwiktionary) - T249506 (duration: 01m 06s) [production]
12:28 <godog> roll-restart pybal on codfw low-traffic - T233956 [production]
12:26 <dzahn@cumin1001> START - Cookbook sre.ganeti.makevm [production]
12:22 <dzahn@cumin1001> END (FAIL) - Cookbook sre.hosts.decommission (exit_code=1) [production]
12:22 <ladsgroup@deploy1001> Synchronized wmf-config/interwiki.php: Update interwiki cache (duration: 03m 01s) [production]
12:21 <dzahn@cumin1001> START - Cookbook sre.hosts.decommission [production]
12:18 <ladsgroup@deploy1001> Synchronized langlist: Create Awadhi Wikipedia (awawiki) - T251371 (duration: 01m 06s) [production]
12:16 <ladsgroup@deploy1001> Synchronized static/images/project-logos: Create Awadhi Wikipedia (awawiki) - T251371 (duration: 01m 06s) [production]
12:14 <ladsgroup@deploy1001> Synchronized multiversion/MWMultiVersion.php: Create Awadhi Wikipedia (awawiki) - T251371 (duration: 01m 06s) [production]
12:12 <ladsgroup@deploy1001> rebuilt and synchronized wikiversions files: Create Awadhi Wikipedia (awawiki) - T251371 [production]
12:07 <ladsgroup@deploy1001> Synchronized dblists: (no justification provided) (duration: 01m 08s) [production]
11:37 <mutante> rebooting ganeti1009 and ganeti1011 to hopefully clear icinga alerts about microcode mitigations [production]
11:10 <marostegui@cumin1001> dbctl commit (dc=all): 'Fully repool new host db1142 and db1084', diff saved to https://phabricator.wikimedia.org/P11253 and previous config saved to /var/cache/conftool/dbconfig/20200520-111013-marostegui.json [production]
11:07 <jynus@cumin1001> dbctl commit (dc=all): 'Repool es1018, es1015 fully', diff saved to https://phabricator.wikimedia.org/P11252 and previous config saved to /var/cache/conftool/dbconfig/20200520-110732-jynus.json [production]
11:04 <jbond42> roll out update or exim4 [production]
10:46 <moritzm> installing 4.19.118 Linux packages on Buster hosts [production]
10:27 <vgutierrez> rolling restart of ats-tls in text@esams - T249335 [production]
10:19 <marostegui@cumin1001> dbctl commit (dc=all): 'Increase weight for db1142 and db1084 on s4', diff saved to https://phabricator.wikimedia.org/P11250 and previous config saved to /var/cache/conftool/dbconfig/20200520-101928-marostegui.json [production]
10:07 <jynus@cumin1001> dbctl commit (dc=all): 'Repool es1018, es1015 at 50% weight', diff saved to https://phabricator.wikimedia.org/P11249 and previous config saved to /var/cache/conftool/dbconfig/20200520-100726-jynus.json [production]
09:43 <vgutierrez> disable KA for POST/PUT requests on esams - T249335 [production]
09:36 <XioNoX> create ROAs for 198.35.26.0/24 and 198.35.27.0/24 - T253196 [production]
09:33 <dzahn@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
09:31 <marostegui@cumin1001> dbctl commit (dc=all): 'Increase weight for db1142 and db1084 on s4', diff saved to https://phabricator.wikimedia.org/P11247 and previous config saved to /var/cache/conftool/dbconfig/20200520-093141-marostegui.json [production]
09:30 <dzahn@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
09:29 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
09:28 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime [production]
09:28 <XioNoX> create ARIN inetnum 198.35.27.0/24 and route 198.35.26.0/24 + 198.35.27.0/24 - T253196 [production]
09:26 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
09:26 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime [production]
09:26 <marostegui> Upgrade db1083 (s1 master) to 10.1.43-2 without restarting T251982 [production]
09:25 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime [production]
09:24 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime [production]
09:15 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
09:12 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime [production]
09:11 <marostegui@cumin1001> dbctl commit (dc=all): 'Increase weight for new host db1142 and start to repool db1084', diff saved to https://phabricator.wikimedia.org/P11246 and previous config saved to /var/cache/conftool/dbconfig/20200520-091153-marostegui.json [production]
09:08 <dzahn@cumin1001> END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) [production]
09:08 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
09:05 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime [production]
09:02 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
09:01 <jayme@deploy1001> helmfile [STAGING] Ran 'sync' command on namespace 'mathoid' for release 'staging' . [production]
09:00 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
09:00 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime [production]
08:57 <marostegui@cumin1001> dbctl commit (dc=all): 'Pool db1142 with minimum weight for the first time T252512', diff saved to https://phabricator.wikimedia.org/P11245 and previous config saved to /var/cache/conftool/dbconfig/20200520-085757-marostegui.json [production]
08:57 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime [production]
08:55 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
08:52 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime [production]
08:52 <dzahn@cumin1001> START - Cookbook sre.ganeti.makevm [production]
08:49 <_joe_> converting mw1266-1275 to use envoy T247389 [production]
08:46 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
08:44 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime [production]