1001-1050 of 10000 results (78ms)
2023-12-19 ยง
16:12 <mvernon@cumin2002> START - Cookbook sre.hosts.downtime for 14 days, 0:00:00 on moss-be[2001-2003].codfw.wmnet with reason: not in service, being used to test a destructive cookbook [production]
16:04 <ayounsi@cumin1001> END (FAIL) - Cookbook sre.network.peering (exit_code=99) with action 'email' for AS: 327700 [production]
16:04 <ayounsi@cumin1001> START - Cookbook sre.network.peering with action 'email' for AS: 327700 [production]
16:02 <ayounsi@cumin1001> END (FAIL) - Cookbook sre.network.peering (exit_code=99) with action 'email' for AS: 139901 [production]
16:00 <ayounsi@cumin1001> START - Cookbook sre.network.peering with action 'email' for AS: 139901 [production]
16:00 <ayounsi@cumin1001> END (PASS) - Cookbook sre.network.peering (exit_code=0) with action 'email' for AS: 15133 [production]
15:58 <ayounsi@cumin1001> START - Cookbook sre.network.peering with action 'email' for AS: 15133 [production]
15:55 <ayounsi@cumin1001> END (PASS) - Cookbook sre.network.peering (exit_code=0) with action 'email' for AS: 5398 [production]
15:55 <cgoubert@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 7 days, 0:00:00 on mw2448.codfw.wmnet with reason: hw failure [production]
15:55 <cgoubert@cumin2002> START - Cookbook sre.hosts.downtime for 7 days, 0:00:00 on mw2448.codfw.wmnet with reason: hw failure [production]
15:55 <ayounsi@cumin1001> START - Cookbook sre.network.peering with action 'email' for AS: 5398 [production]
15:42 <lucaswerkmeister-wmde@deploy2002> Finished scap: Backport for [[gerrit:983758|Change virtual domain of botpassword to plural (T351559)]] (duration: 07m 01s) [production]
15:38 <moritzm> installing gnutls28 security updates on bookworm [production]
15:37 <lucaswerkmeister-wmde@deploy2002> lucaswerkmeister-wmde and ladsgroup: Continuing with sync [production]
15:37 <lucaswerkmeister-wmde@deploy2002> lucaswerkmeister-wmde and ladsgroup: Backport for [[gerrit:983758|Change virtual domain of botpassword to plural (T351559)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
15:35 <lucaswerkmeister-wmde@deploy2002> Started scap: Backport for [[gerrit:983758|Change virtual domain of botpassword to plural (T351559)]] [production]
15:33 <lucaswerkmeister-wmde@deploy2002> Finished scap: Backport for [[gerrit:984174|Use main replica DB in importExistingFilesToScanTable.php]] (duration: 07m 47s) [production]
15:27 <lucaswerkmeister-wmde@deploy2002> kharlan and lucaswerkmeister-wmde: Continuing with sync [production]
15:27 <lucaswerkmeister-wmde@deploy2002> kharlan and lucaswerkmeister-wmde: Backport for [[gerrit:984174|Use main replica DB in importExistingFilesToScanTable.php]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
15:25 <lucaswerkmeister-wmde@deploy2002> Started scap: Backport for [[gerrit:984174|Use main replica DB in importExistingFilesToScanTable.php]] [production]
15:23 <taavi@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 30 days, 0:00:00 on cloudvirt1063.eqiad.wmnet with reason: host is down, downtiming in icinga too [production]
15:23 <taavi@cumin1001> START - Cookbook sre.hosts.downtime for 30 days, 0:00:00 on cloudvirt1063.eqiad.wmnet with reason: host is down, downtiming in icinga too [production]
15:22 <lucaswerkmeister-wmde@deploy2002> Finished scap: Backport for [[gerrit:984172|Make SearchEntitiesIntegrationTest an ApiTestCase (T353334)]], [[gerrit:984173|Use link batch in search APIs (T353334)]] (duration: 08m 49s) [production]
15:16 <lucaswerkmeister-wmde@deploy2002> lucaswerkmeister-wmde: Continuing with sync [production]
15:15 <moritzm> installing exim4 bugfix updates from Bookworm point release [production]
15:15 <lucaswerkmeister-wmde@deploy2002> lucaswerkmeister-wmde: Backport for [[gerrit:984172|Make SearchEntitiesIntegrationTest an ApiTestCase (T353334)]], [[gerrit:984173|Use link batch in search APIs (T353334)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
15:13 <lucaswerkmeister-wmde@deploy2002> Started scap: Backport for [[gerrit:984172|Make SearchEntitiesIntegrationTest an ApiTestCase (T353334)]], [[gerrit:984173|Use link batch in search APIs (T353334)]] [production]
15:10 <moritzm> installing nagios-plugins-contrib bugfix updates from Bookworm point release [production]
14:44 <hnowlan@deploy2002> helmfile [eqiad] DONE helmfile.d/services/changeprop-jobqueue: apply [production]
14:43 <hnowlan@deploy2002> helmfile [eqiad] START helmfile.d/services/changeprop-jobqueue: apply [production]
14:43 <hnowlan@deploy2002> helmfile [codfw] DONE helmfile.d/services/changeprop-jobqueue: apply [production]
14:42 <hnowlan@deploy2002> helmfile [codfw] START helmfile.d/services/changeprop-jobqueue: apply [production]
14:33 <jgiannelos@deploy2002> helmfile [eqiad] DONE helmfile.d/services/proton: apply [production]
14:32 <jgiannelos@deploy2002> helmfile [eqiad] START helmfile.d/services/proton: apply [production]
14:31 <jgiannelos@deploy2002> helmfile [codfw] DONE helmfile.d/services/proton: apply [production]
14:30 <jgiannelos@deploy2002> helmfile [codfw] START helmfile.d/services/proton: apply [production]
14:29 <jgiannelos@deploy2002> helmfile [staging] DONE helmfile.d/services/proton: apply [production]
14:29 <jgiannelos@deploy2002> helmfile [staging] START helmfile.d/services/proton: apply [production]
14:25 <Lucas_WMDE> UTC afternoon backport+config window done [production]
14:24 <lucaswerkmeister-wmde@deploy2002> Finished scap: Backport for [[gerrit:984166|Send PhotoDNA the mime type of the thumbnail and not original file (T351401)]], [[gerrit:984169|Add maintenance script to scan files in the mediamoderation_scan table (T351399)]] (duration: 07m 53s) [production]
14:24 <kamila@deploy2002> helmfile [staging] DONE helmfile.d/services/rdf-streaming-updater: apply [production]
14:24 <kamila@deploy2002> helmfile [staging] START helmfile.d/services/rdf-streaming-updater: apply [production]
14:24 <kamila@deploy2002> helmfile [staging] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
14:24 <kamila@deploy2002> helmfile [staging] START helmfile.d/services/cirrus-streaming-updater: apply [production]
14:22 <kamila@deploy2002> helmfile [staging] DONE helmfile.d/services/proton: apply [production]
14:21 <kamila@deploy2002> helmfile [staging] START helmfile.d/services/proton: apply [production]
14:21 <kamila@deploy2002> helmfile [staging] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
14:19 <kamila@deploy2002> helmfile [staging] START helmfile.d/services/cirrus-streaming-updater: apply [production]
14:19 <lucaswerkmeister-wmde@deploy2002> lucaswerkmeister-wmde and kharlan: Continuing with sync [production]
14:18 <lucaswerkmeister-wmde@deploy2002> lucaswerkmeister-wmde and kharlan: Backport for [[gerrit:984166|Send PhotoDNA the mime type of the thumbnail and not original file (T351401)]], [[gerrit:984169|Add maintenance script to scan files in the mediamoderation_scan table (T351399)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]