2001-2050 of 10000 results (86ms)
2024-03-26 ยง
20:51 <catrope@deploy1002> aram and catrope: Continuing with sync [production]
20:51 <ladsgroup@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on 12 hosts with reason: Maint T352010 [production]
20:51 <ladsgroup@cumin1002> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on 12 hosts with reason: Maint T352010 [production]
20:50 <ladsgroup@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2113.codfw.wmnet with reason: Maintenance [production]
20:50 <ladsgroup@cumin1002> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2113.codfw.wmnet with reason: Maintenance [production]
20:49 <catrope@deploy1002> aram and catrope: Backport for [[gerrit:1010938|Add autopatrolled, rollbacker and suppressredirect user groups for ckbwiktionary (T360228)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
20:46 <ladsgroup@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on 13 hosts with reason: Maint T343718 [production]
20:46 <ladsgroup@cumin1002> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on 13 hosts with reason: Maint T343718 [production]
20:45 <catrope@deploy1002> Started scap: Backport for [[gerrit:1010938|Add autopatrolled, rollbacker and suppressredirect user groups for ckbwiktionary (T360228)]] [production]
20:43 <catrope@deploy1002> Finished scap: Backport for [[gerrit:1014456|CodexHTMLForm: Fix margins around links in login form (T360945)]] (duration: 22m 09s) [production]
20:41 <ladsgroup@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2129.codfw.wmnet with reason: Maintenance [production]
20:41 <ladsgroup@cumin1002> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2129.codfw.wmnet with reason: Maintenance [production]
20:41 <ladsgroup@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on 13 hosts with reason: Maint T352010 [production]
20:40 <ladsgroup@cumin1002> START - Cookbook sre.hosts.downtime for 12:00:00 on 13 hosts with reason: Maint T352010 [production]
20:30 <catrope@deploy1002> catrope: Continuing with sync [production]
20:27 <catrope@deploy1002> catrope: Backport for [[gerrit:1014456|CodexHTMLForm: Fix margins around links in login form (T360945)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
20:21 <catrope@deploy1002> Started scap: Backport for [[gerrit:1014456|CodexHTMLForm: Fix margins around links in login form (T360945)]] [production]
20:09 <ryankemper@cumin2002> START - Cookbook sre.elasticsearch.rolling-operation Operation.RESTART (3 nodes at a time) for ElasticSearch cluster search_codfw: cycle some masters - ryankemper@cumin2002 - T358882 [production]
19:48 <mutante> phabricator - added GMikesell-WMF to WMF-NDA because that goes together with the wmf LDAP group (https://wikitech.wikimedia.org/wiki/SRE/Clinic_Duty/Access_requests#WMF_Group) - T358922 [production]
19:46 <vriley@cumin1002> START - Cookbook sre.hosts.reimage for host dbprov1006.eqiad.wmnet with OS bullseye [production]
19:34 <logmsgbot> @deploy2002 helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
19:34 <logmsgbot> @deploy2002 helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply [production]
19:21 <ebernhardson@deploy2002> helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
19:21 <ebernhardson@deploy2002> helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply [production]
19:21 <pt1979@cumin2002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - pt1979@cumin2002" [production]
19:20 <ebernhardson@deploy2002> helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
19:20 <ebernhardson@deploy2002> helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply [production]
19:05 <pt1979@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on dbprov2005.codfw.wmnet with reason: host reimage [production]
19:02 <pt1979@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on dbprov2005.codfw.wmnet with reason: host reimage [production]
18:50 <ejegg> donorwiki upgraded from 27d326b7 to c7f1325c [production]
18:47 <pt1979@cumin2002> START - Cookbook sre.hosts.reimage for host dbprov2005.codfw.wmnet with OS bullseye [production]
18:47 <vriley@cumin1002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host dbprov1006.eqiad.wmnet with OS bullseye [production]
18:42 <cstone> civicrm upgraded from b8a84b22 to 143aa0bf [production]
18:36 <denisse@cumin2002> END (FAIL) - Cookbook sre.puppet.migrate-host (exit_code=99) for host alert2001.wikimedia.org [production]
18:36 <denisse@cumin2002> START - Cookbook sre.puppet.migrate-host for host alert2001.wikimedia.org [production]
18:34 <jhuneidi@deploy1002> rebuilt and synchronized wikiversions files: group0 wikis to 1.42.0-wmf.24 refs T360156 [production]
18:21 <hnowlan@deploy1002> helmfile [staging] DONE helmfile.d/services/mobileapps: apply [production]
18:21 <hnowlan@deploy1002> helmfile [staging] START helmfile.d/services/mobileapps: apply [production]
18:20 <hnowlan@deploy1002> helmfile [staging] DONE helmfile.d/services/mobileapps: apply [production]
18:20 <hnowlan@deploy1002> helmfile [staging] START helmfile.d/services/mobileapps: apply [production]
18:00 <pt1979@cumin2002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host dbprov2005.codfw.wmnet with OS bullseye [production]
17:45 <hnowlan@deploy1002> helmfile [staging] DONE helmfile.d/services/mobileapps: apply [production]
17:44 <hnowlan@deploy1002> helmfile [staging] START helmfile.d/services/mobileapps: apply [production]
17:22 <vriley@cumin1002> END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=1) upgrade firmware for hosts ['dbprov1006'] [production]
17:22 <vriley@cumin1002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['dbprov1006'] [production]
17:22 <vriley@cumin1002> END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=1) upgrade firmware for hosts ['dbprov1006'] [production]
17:22 <vriley@cumin1002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['dbprov1006'] [production]
17:21 <vriley@cumin1002> END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=1) upgrade firmware for hosts ['dbprov1006'] [production]
17:21 <vriley@cumin1002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['dbprov1006'] [production]
17:13 <vriley@cumin1002> START - Cookbook sre.hosts.reimage for host dbprov1006.eqiad.wmnet with OS bullseye [production]