5901-5950 of 10000 results (132ms)
2024-11-19 ยง
09:38 <dcausse@deploy2002> helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply [production]
09:35 <arnaudb@cumin1002> START - Cookbook sre.switchdc.databases.prepare for the switch from codfw to eqiad [production]
09:33 <dcausse@deploy2002> helmfile [codfw] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
09:32 <dcausse@deploy2002> helmfile [codfw] START helmfile.d/services/cirrus-streaming-updater: apply [production]
09:19 <aklapper@deploy2002> rebuilt and synchronized wikiversions files: group0 to 1.44.0-wmf.4 refs T375663 [production]
09:18 <arnaudb@cumin1002> END (PASS) - Cookbook sre.switchdc.databases.prepare (exit_code=0) for the switch from codfw to eqiad [production]
09:18 <arnaudb@cumin1002> START - Cookbook sre.switchdc.databases.prepare for the switch from codfw to eqiad [production]
08:59 <urbanecm@deploy2002> Finished scap sync-world: Backport for [[gerrit:1092752|Add + to nowiki in core-Permissions.php (T380252)]] (duration: 10m 17s) [production]
08:54 <urbanecm@deploy2002> urbanecm, jhsoby: Continuing with sync [production]
08:54 <urbanecm@deploy2002> urbanecm, jhsoby: Backport for [[gerrit:1092752|Add + to nowiki in core-Permissions.php (T380252)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
08:49 <urbanecm@deploy2002> Started scap sync-world: Backport for [[gerrit:1092752|Add + to nowiki in core-Permissions.php (T380252)]] [production]
08:48 <urbanecm@deploy2002> Finished scap sync-world: Backport for [[gerrit:1092741|fix tours by finishing partial variable rename (T380071)]], [[gerrit:1092364|affcom contactpages: Fix Letter of intent and logo field labels (T375392)]], [[gerrit:1092743|Add nowiki to commonsuploads dblist (T380252)]] (duration: 14m 29s) [production]
08:43 <urbanecm@deploy2002> ammarpad, migr, jhsoby, urbanecm: Continuing with sync [production]
08:39 <urbanecm@deploy2002> ammarpad, migr, jhsoby, urbanecm: Backport for [[gerrit:1092741|fix tours by finishing partial variable rename (T380071)]], [[gerrit:1092364|affcom contactpages: Fix Letter of intent and logo field labels (T375392)]], [[gerrit:1092743|Add nowiki to commonsuploads dblist (T380252)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
08:34 <urbanecm@deploy2002> Started scap sync-world: Backport for [[gerrit:1092741|fix tours by finishing partial variable rename (T380071)]], [[gerrit:1092364|affcom contactpages: Fix Letter of intent and logo field labels (T375392)]], [[gerrit:1092743|Add nowiki to commonsuploads dblist (T380252)]] [production]
08:29 <urbanecm@deploy2002> Finished scap sync-world: Backport for [[gerrit:1082726|Translate Event Logging: Enable using $wgTranslateEnableEventLogging (T364460)]], [[gerrit:1092258|CirrusSearch: enable offloading weighted tags via EventBus (T378983 T377150)]], [[gerrit:1091197|[GrowthExperiments] Add virtual domain config (T354939)]] (duration: 24m 42s) [production]
08:22 <urbanecm@deploy2002> urbanecm, wangombe, pfischer: Continuing with sync [production]
08:12 <urbanecm@deploy2002> urbanecm, wangombe, pfischer: Backport for [[gerrit:1082726|Translate Event Logging: Enable using $wgTranslateEnableEventLogging (T364460)]], [[gerrit:1092258|CirrusSearch: enable offloading weighted tags via EventBus (T378983 T377150)]], [[gerrit:1091197|[GrowthExperiments] Add virtual domain config (T354939)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
08:04 <urbanecm@deploy2002> Started scap sync-world: Backport for [[gerrit:1082726|Translate Event Logging: Enable using $wgTranslateEnableEventLogging (T364460)]], [[gerrit:1092258|CirrusSearch: enable offloading weighted tags via EventBus (T378983 T377150)]], [[gerrit:1091197|[GrowthExperiments] Add virtual domain config (T354939)]] [production]
07:45 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2202.codfw.wmnet with reason: sad [production]
07:45 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2202.codfw.wmnet with reason: sad [production]
07:41 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 7 days, 0:00:00 on db1246.eqiad.wmnet with reason: T374215 - hw maintenance [production]
07:40 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 7 days, 0:00:00 on db1246.eqiad.wmnet with reason: T374215 - hw maintenance [production]
07:32 <jmm@cumin2002> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti1016.eqiad.wmnet [production]
07:31 <jmm@cumin2002> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti1016.eqiad.wmnet [production]
07:24 <jmm@cumin2002> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti1016.eqiad.wmnet [production]
05:01 <mwpresync@deploy2002> Pruned MediaWiki: 1.44.0-wmf.1 (duration: 01m 18s) [production]
04:52 <mwpresync@deploy2002> Finished scap sync-world: testwikis to 1.44.0-wmf.4 refs T375663 (duration: 49m 01s) [production]
04:16 <andrew@cumin1002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudvirt1062.eqiad.wmnet with OS bookworm [production]
04:03 <mwpresync@deploy2002> Started scap sync-world: testwikis to 1.44.0-wmf.4 refs T375663 [production]
04:00 <ejegg> fundraising civicrm upgraded from 463a12c5 to e29243f0 [production]
03:51 <andrew@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudvirt1062.eqiad.wmnet with reason: host reimage [production]
03:48 <andrew@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on cloudvirt1062.eqiad.wmnet with reason: host reimage [production]
03:33 <andrew@cumin1002> START - Cookbook sre.hosts.reimage for host cloudvirt1062.eqiad.wmnet with OS bookworm [production]
03:09 <ejegg> payments-wiki upgraded from 459f259b to c4463536 [production]
02:31 <jclark@cumin1002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-jumbo1018.eqiad.wmnet with OS bullseye [production]
02:30 <jclark@cumin1002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jclark@cumin1002" [production]
02:30 <jclark@cumin1002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jclark@cumin1002" [production]
02:23 <ejegg> standalone (IPN listener) SmashPig upgraded from 601405dc to 131e92a5 [production]
02:12 <jclark@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-jumbo1018.eqiad.wmnet with reason: host reimage [production]
02:08 <jclark@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-jumbo1018.eqiad.wmnet with reason: host reimage [production]
01:54 <jclark@cumin1002> START - Cookbook sre.hosts.reimage for host kafka-jumbo1018.eqiad.wmnet with OS bullseye [production]
01:54 <jclark@cumin1002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host kafka-jumbo1018.eqiad.wmnet with OS bullseye [production]
01:51 <jclark@cumin1002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-jumbo1016.eqiad.wmnet with OS bullseye [production]
01:51 <jclark@cumin1002> END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jclark@cumin1002" [production]
01:50 <jclark@cumin1002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-jumbo1017.eqiad.wmnet with OS bullseye [production]
01:50 <jclark@cumin1002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jclark@cumin1002" [production]
01:40 <jclark@cumin1002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jclark@cumin1002" [production]
01:24 <jclark@cumin1002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jclark@cumin1002" [production]
01:24 <jclark@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-jumbo1017.eqiad.wmnet with reason: host reimage [production]