151-200 of 10000 results (91ms)
2024-08-16 §
07:40 <jayme@cumin1002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-main2007.codfw.wmnet with OS bullseye [production]
07:23 <jayme@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-main2007.codfw.wmnet with reason: host reimage [production]
07:20 <jayme@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-main2007.codfw.wmnet with reason: host reimage [production]
07:01 <jayme@cumin1002> START - Cookbook sre.hosts.reimage for host kafka-main2007.codfw.wmnet with OS bullseye [production]
06:56 <marostegui@cumin1002> dbctl commit (dc=all): 'Repool db2136 - running 10.11', diff saved to https://phabricator.wikimedia.org/P67345 and previous config saved to /var/cache/conftool/dbconfig/20240816-065606-marostegui.json [production]
06:42 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on db2152.codfw.wmnet with reason: Schema change [production]
06:42 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 2 days, 0:00:00 on db2152.codfw.wmnet with reason: Schema change [production]
2024-08-15 §
23:30 <jhancock@cumin2002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host cloudlb2004-dev.codfw.wmnet with OS bookworm [production]
23:10 <xSavitar> T372449 mwscript extensions/CentralAuth/maintenance/fixStuckGlobalRename.php --wiki=commonswiki --logwiki=metawiki 'Philip Federici' 'FilippoFederici' --ignorestatus [production]
22:42 <ryankemper@cumin2002> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) (T370754, transfer fresh wdqs-main journal to codfw host) xfer wikidata_main from wdqs1021.eqiad.wmnet -> wdqs2021.codfw.wmnet w/ force delete existing files, repooling neither afterwards [production]
22:10 <jhancock@cumin2002> START - Cookbook sre.hosts.reimage for host cloudlb2004-dev.codfw.wmnet with OS bookworm [production]
22:09 <jhancock@cumin2002> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['cloudlb2004-dev'] [production]
22:02 <jhancock@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['cloudlb2004-dev'] [production]
22:01 <jhancock@cumin2002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host cloudlb2004-dev.mgmt.codfw.wmnet with reboot policy FORCED [production]
21:54 <ryankemper@cumin2002> START - Cookbook sre.wdqs.data-transfer (T370754, transfer fresh wdqs-main journal to codfw host) xfer wikidata_main from wdqs1021.eqiad.wmnet -> wdqs2021.codfw.wmnet w/ force delete existing files, repooling neither afterwards [production]
21:53 <ryankemper@cumin2002> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) (T370754, transfer fresh wdqs-main journal to codfw host) xfer wikidata_main from wdqs1021.eqiad.wmnet -> wdqs2021.codfw.wmnet, repooling neither afterwards [production]
21:53 <ryankemper@cumin2002> START - Cookbook sre.wdqs.data-transfer (T370754, transfer fresh wdqs-main journal to codfw host) xfer wikidata_main from wdqs1021.eqiad.wmnet -> wdqs2021.codfw.wmnet, repooling neither afterwards [production]
21:47 <pfischer@deploy1003> helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
21:46 <pfischer@deploy1003> helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply [production]
21:46 <jhancock@cumin2002> START - Cookbook sre.hosts.provision for host cloudlb2004-dev.mgmt.codfw.wmnet with reboot policy FORCED [production]
21:43 <jhancock@cumin2002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
21:43 <jhancock@cumin2002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: adding cloudlb2004-dev to codfw - jhancock@cumin2002" [production]
21:43 <jhancock@cumin2002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: adding cloudlb2004-dev to codfw - jhancock@cumin2002" [production]
21:37 <jhancock@cumin2002> START - Cookbook sre.dns.netbox [production]
21:01 <ebernhardson> backport window complete [production]
20:55 <jhancock@cumin2002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
20:55 <jhancock@cumin2002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: adding ganeti2035 to codfw - jhancock@cumin2002" [production]
20:54 <jhancock@cumin2002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: adding ganeti2035 to codfw - jhancock@cumin2002" [production]
20:54 <ejegg> fundraising civicrm upgraded from eecbba5d to 56521963 [production]
20:49 <jhancock@cumin2002> START - Cookbook sre.dns.netbox [production]
20:45 <ebernhardson@deploy1003> Finished scap sync-world: Backport for [[gerrit:1063026|cirrus: Stop general writes to private wikis (T341332)]] (duration: 08m 25s) [production]
20:41 <ebernhardson@deploy1003> ebernhardson: Continuing with sync [production]
20:39 <ebernhardson@deploy1003> ebernhardson: Backport for [[gerrit:1063026|cirrus: Stop general writes to private wikis (T341332)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
20:37 <ebernhardson@deploy1003> Started scap sync-world: Backport for [[gerrit:1063026|cirrus: Stop general writes to private wikis (T341332)]] [production]
20:30 <ebernhardson@deploy1003> Finished scap sync-world: Backport for [[gerrit:1063041|Revert "CommentFormatter: Switch from deprecated addJsConfigVars to new setJsConfigVar" (T372499)]] (duration: 10m 06s) [production]
20:25 <ebernhardson@deploy1003> ebernhardson, matmarex: Continuing with sync [production]
20:23 <ebernhardson@deploy1003> ebernhardson, matmarex: Backport for [[gerrit:1063041|Revert "CommentFormatter: Switch from deprecated addJsConfigVars to new setJsConfigVar" (T372499)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
20:20 <ebernhardson@deploy1003> Started scap sync-world: Backport for [[gerrit:1063041|Revert "CommentFormatter: Switch from deprecated addJsConfigVars to new setJsConfigVar" (T372499)]] [production]
18:54 <tgr|away> running global rename cleanup script per T372006#10055573 [production]
18:15 <jhuneidi@deploy1003> rebuilt and synchronized wikiversions files: group2 to 1.43.0-wmf.18 refs T366963 [production]
18:02 <aokoth@cumin1002> END (PASS) - Cookbook sre.vrts.upgrade (exit_code=0) on VRTS host vrts1001.eqiad.wmnet [production]
18:00 <aokoth@cumin1002> START - Cookbook sre.vrts.upgrade on VRTS host vrts1001.eqiad.wmnet [production]
17:45 <dwisehaupt@cumin1002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
17:44 <dwisehaupt@cumin1002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update mgmt dns for civi2002 frpig2002 - dwisehaupt@cumin1002" [production]
17:44 <dwisehaupt@cumin1002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update mgmt dns for civi2002 frpig2002 - dwisehaupt@cumin1002" [production]
17:41 <dwisehaupt@cumin1002> START - Cookbook sre.dns.netbox [production]
17:22 <sukhe@cumin1002> END (PASS) - Cookbook sre.dns.admin (exit_code=0) DNS admin: pool site ulsfo [reason: testing done, T369366] [production]
17:22 <sukhe@cumin1002> START - Cookbook sre.dns.admin DNS admin: pool site ulsfo [reason: testing done, T369366] [production]
17:13 <jayme@cumin1002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-main2008.codfw.wmnet with OS bullseye [production]
17:07 <sukhe@cumin1002> END (PASS) - Cookbook sre.dns.admin (exit_code=0) DNS admin: depool site ulsfo [reason: testing live change, T369366] [production]