4401-4450 of 10000 results (73ms)
2022-06-29 ยง
23:34 <cmjohnson@cumin1001> START - Cookbook sre.hosts.provision for host stat1009.mgmt.eqiad.wmnet with reboot policy FORCED [production]
23:30 <bking@cumin1001> END (PASS) - Cookbook sre.elasticsearch.rolling-operation (exit_code=0) Operation.RESTART (3 nodes at a time) for ElasticSearch cluster search_eqiad: eqiad cluster restart to pickup swift-s3 plugin - bking@cumin1001 - T309648 [production]
23:05 <pt1979@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2155.codfw.wmnet with reason: host reimage [production]
23:01 <pt1979@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on db2155.codfw.wmnet with reason: host reimage [production]
22:41 <pt1979@cumin2002> START - Cookbook sre.hosts.reimage for host db2155.codfw.wmnet with OS bullseye [production]
22:37 <cmjohnson@cumin1001> START - Cookbook sre.hosts.provision for host stat1009.mgmt.eqiad.wmnet with reboot policy FORCED [production]
22:34 <cmjohnson@cumin1001> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host cloudnet1006.mgmt.eqiad.wmnet with reboot policy FORCED [production]
22:31 <cmjohnson@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host mw1477.eqiad.wmnet with OS buster [production]
22:29 <cmjohnson@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
22:26 <pt1979@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2154.codfw.wmnet with OS bullseye [production]
22:25 <cmjohnson@cumin1001> START - Cookbook sre.dns.netbox [production]
22:16 <cmjohnson@cumin1001> START - Cookbook sre.hosts.provision for host cloudnet1006.mgmt.eqiad.wmnet with reboot policy FORCED [production]
22:08 <cmjohnson@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host mw1475.eqiad.wmnet with OS buster [production]
21:55 <cmjohnson@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host mw1496.eqiad.wmnet with OS buster [production]
21:54 <pt1979@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2154.codfw.wmnet with reason: host reimage [production]
21:52 <cmjohnson@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mw1477.eqiad.wmnet with reason: host reimage [production]
21:49 <pt1979@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on db2154.codfw.wmnet with reason: host reimage [production]
21:48 <cmjohnson@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on mw1477.eqiad.wmnet with reason: host reimage [production]
21:42 <cjming> end of UTC late backport window [production]
21:41 <cmjohnson@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudcephosd1031.eqiad.wmnet with OS buster [production]
21:41 <cjming@deploy1002> Synchronized php-1.39.0-wmf.18/extensions/DiscussionTools/modules/dt.ui.NewTopicController.less: Backport: [[gerrit:809691|New topic hint: Add clear:both (T311597)]] (duration: 03m 27s) [production]
21:39 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
21:39 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
21:39 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
21:38 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
21:37 <cmjohnson@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mw1475.eqiad.wmnet with reason: host reimage [production]
21:37 <cjming@deploy1002> Synchronized php-1.39.0-wmf.17/extensions/DiscussionTools/modules/dt.ui.NewTopicController.less: Backport: [[gerrit:809690|New topic hint: Add clear:both (T311597)]] (duration: 03m 24s) [production]
21:37 <cmjohnson@cumin1001> START - Cookbook sre.hosts.reimage for host mw1477.eqiad.wmnet with OS buster [production]
21:36 <cmjohnson@cumin1001> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host mw1477.mgmt.eqiad.wmnet with reboot policy FORCED [production]
21:36 <cmjohnson@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mw1496.eqiad.wmnet with reason: host reimage [production]
21:35 <cmjohnson@cumin1001> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host an-presto1006.eqiad.wmnet with OS bullseye [production]
21:34 <cmjohnson@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudcephosd1025.eqiad.wmnet with OS buster [production]
21:34 <cjming@deploy1002> Synchronized php-1.39.0-wmf.18/extensions/DiscussionTools/modules/NewTopicController.js: Backport: [[gerrit:809689|New topic hint: Avoid error about section editing when opened from diff (T311665)]] (duration: 03m 43s) [production]
21:33 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
21:33 <cmjohnson@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on mw1475.eqiad.wmnet with reason: host reimage [production]
21:32 <cmjohnson@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on mw1496.eqiad.wmnet with reason: host reimage [production]
21:32 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
21:32 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
21:31 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
21:30 <pt1979@cumin2002> START - Cookbook sre.hosts.reimage for host db2154.codfw.wmnet with OS bullseye [production]
21:30 <cjming@deploy1002> Synchronized php-1.39.0-wmf.17/extensions/DiscussionTools/modules/NewTopicController.js: Backport: [[gerrit:809688|New topic hint: Avoid error about section editing when opened from diff (T311665)]] (duration: 03m 35s) [production]
21:26 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
21:26 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
21:25 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
21:25 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
21:24 <pt1979@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2153.codfw.wmnet with OS bullseye [production]
21:23 <cmjohnson@cumin1001> START - Cookbook sre.hosts.provision for host mw1477.mgmt.eqiad.wmnet with reboot policy FORCED [production]
21:22 <cmjohnson@cumin1001> START - Cookbook sre.hosts.reimage for host mw1475.eqiad.wmnet with OS buster [production]
21:21 <cmjohnson@cumin1001> START - Cookbook sre.hosts.reimage for host mw1496.eqiad.wmnet with OS buster [production]
21:20 <cmjohnson@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]