1-50 of 10000 results (66ms)
2023-10-02 ยง
23:09 <eevans@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host restbase1032.eqiad.wmnet with OS bullseye [production]
22:46 <eevans@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on restbase1032.eqiad.wmnet with reason: host reimage [production]
22:43 <eevans@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on restbase1032.eqiad.wmnet with reason: host reimage [production]
22:30 <eevans@cumin1001> START - Cookbook sre.hosts.reimage for host restbase1032.eqiad.wmnet with OS bullseye [production]
22:30 <eevans@cumin1001> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host restbase1032.eqiad.wmnet with OS bullseye [production]
22:16 <eevans@cumin1001> START - Cookbook sre.hosts.reimage for host restbase1032.eqiad.wmnet with OS bullseye [production]
22:09 <eevans@cumin1001> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts restbase1032.eqiad.wmnet [production]
22:01 <eevans@cumin1001> END (ERROR) - Cookbook sre.hosts.reboot-single (exit_code=97) for host restbase1032.eqiad.wmnet [production]
22:01 <eevans@cumin1001> START - Cookbook sre.hosts.reboot-single for host restbase1032.eqiad.wmnet [production]
22:00 <eevans@cumin1001> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts restbase1032.eqiad.wmnet [production]
21:53 <maryum> Deployed patch for T347704 [production]
21:32 <kindrobot> end UTC late backport window [production]
21:28 <eevans@cumin1001> END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=99) upgrade firmware for hosts restbase1029.eqiad.wmnet [production]
21:28 <eevans@cumin1001> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts restbase1029.eqiad.wmnet [production]
21:23 <eevans@cumin1001> END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=99) upgrade firmware for hosts restbase1032.eqiad.wmnet [production]
21:23 <eevans@cumin1001> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts restbase1032.eqiad.wmnet [production]
21:22 <eevans@cumin1001> END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=99) upgrade firmware for hosts restbase1032.eqiad.wmnet [production]
21:22 <eevans@cumin1001> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts restbase1032.eqiad.wmnet [production]
21:21 <eevans@cumin1001> END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=99) upgrade firmware for hosts restbase1032.eqiad.wmnet [production]
21:21 <eevans@cumin1001> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts restbase1032.eqiad.wmnet [production]
21:21 <eevans@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host restbase1029.eqiad.wmnet with OS bullseye [production]
21:17 <kindrobot@deploy2002> Finished scap: Backport for [[gerrit:962212|Ignore <!--comment--> only site notices (T347645)]], [[gerrit:962213|HookUtils: Fix checking page props (T347878)]], [[gerrit:962214|Fix diff title escaping (T347578)]], [[gerrit:962215|Diff: Add missing .mw-diff-inline-moved selector]] (duration: 10m 06s) [production]
21:11 <kindrobot@deploy2002> kindrobot and matmarex: Continuing with sync [production]
21:08 <kindrobot@deploy2002> kindrobot and matmarex: Backport for [[gerrit:962212|Ignore <!--comment--> only site notices (T347645)]], [[gerrit:962213|HookUtils: Fix checking page props (T347878)]], [[gerrit:962214|Fix diff title escaping (T347578)]], [[gerrit:962215|Diff: Add missing .mw-diff-inline-moved selector]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
21:07 <kindrobot@deploy2002> Started scap: Backport for [[gerrit:962212|Ignore <!--comment--> only site notices (T347645)]], [[gerrit:962213|HookUtils: Fix checking page props (T347878)]], [[gerrit:962214|Fix diff title escaping (T347578)]], [[gerrit:962215|Diff: Add missing .mw-diff-inline-moved selector]] [production]
20:59 <ottomata> mw-page-content-change-enrich - CORRECTION - increase replicas to 20 to process backlog - T347676 [production]
20:58 <otto@deploy2002> helmfile [codfw] DONE helmfile.d/services/mw-page-content-change-enrich: apply [production]
20:58 <otto@deploy2002> helmfile [codfw] START helmfile.d/services/mw-page-content-change-enrich: apply [production]
20:57 <eevans@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on restbase1029.eqiad.wmnet with reason: host reimage [production]
20:57 <otto@deploy2002> helmfile [codfw] DONE helmfile.d/services/mw-page-content-change-enrich: apply [production]
20:57 <otto@deploy2002> helmfile [codfw] START helmfile.d/services/mw-page-content-change-enrich: apply [production]
20:56 <otto@deploy2002> helmfile [codfw] DONE helmfile.d/services/mw-page-content-change-enrich: apply [production]
20:56 <otto@deploy2002> helmfile [codfw] START helmfile.d/services/mw-page-content-change-enrich: apply [production]
20:56 <ottomata> mw-page-content-change-enrich - increase replicas to 24 to process backlog - T347676 [production]
20:54 <eevans@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on restbase1029.eqiad.wmnet with reason: host reimage [production]
20:42 <eevans@cumin1001> START - Cookbook sre.hosts.reimage for host restbase1029.eqiad.wmnet with OS bullseye [production]
20:40 <eevans@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host restbase1029.eqiad.wmnet with OS bullseye [production]
20:40 <eevans@cumin1001> START - Cookbook sre.hosts.reimage for host restbase1029.eqiad.wmnet with OS bullseye [production]
20:37 <eevans@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host restbase1029.eqiad.wmnet with OS bullseye [production]
20:37 <eevans@cumin1001> START - Cookbook sre.hosts.reimage for host restbase1029.eqiad.wmnet with OS bullseye [production]
20:36 <eevans@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host restbase1029.eqiad.wmnet with OS bullseye [production]
20:35 <eevans@cumin1001> START - Cookbook sre.hosts.reimage for host restbase1029.eqiad.wmnet with OS bullseye [production]
20:32 <eevans@cumin1001> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host restbase1029.eqiad.wmnet with OS bullseye [production]
20:31 <jclark@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host an-master1004.eqiad.wmnet with OS bullseye [production]
20:27 <ottomata> mw-page-content-change-enrich - increase replicas to 12 to process backlog - T347676 [production]
20:27 <kindrobot@deploy2002> Finished scap: Backport for [[gerrit:962105|Undeploy Reader Demographics 2 pilot survey (T345951)]], [[gerrit:962629|DiscussionTools: Disable timestamp links in production initially]] (duration: 08m 49s) [production]
20:27 <eevans@cumin1001> START - Cookbook sre.hosts.reimage for host restbase1029.eqiad.wmnet with OS bullseye [production]
20:22 <jclark@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host an-master1003.eqiad.wmnet with OS bullseye [production]
20:21 <kindrobot@deploy2002> esanders and dani and kindrobot: Continuing with sync [production]
20:19 <kindrobot@deploy2002> esanders and dani and kindrobot: Backport for [[gerrit:962105|Undeploy Reader Demographics 2 pilot survey (T345951)]], [[gerrit:962629|DiscussionTools: Disable timestamp links in production initially]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]