5401-5450 of 10000 results (97ms)
2023-11-08 §
07:58 <jmm@cumin2002> START - Cookbook sre.puppet.migrate-role for role: analytics_cluster::turnilo::staging [production]
07:56 <jmm@cumin2002> END (PASS) - Cookbook sre.puppet.migrate-role (exit_code=0) for role: zookeeper::test [production]
07:42 <jmm@cumin2002> START - Cookbook sre.puppet.migrate-role for role: zookeeper::test [production]
00:16 <urbanecm> mwmaint2002: Stop T315510#9312431 instances of extensions/DiscussionTools/maintenance/persistRevisionThreadItems.php (T315510) [production]
2023-11-07 §
23:09 <ladsgroup@deploy2002> Finished scap: Backport for [[gerrit:972488|styles: Fix stylesheet validation issues]] (duration: 07m 14s) [production]
23:04 <ladsgroup@deploy2002> ladsgroup and volker-e: Continuing with sync [production]
23:03 <ladsgroup@deploy2002> ladsgroup and volker-e: Backport for [[gerrit:972488|styles: Fix stylesheet validation issues]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
23:02 <ladsgroup@deploy2002> Started scap: Backport for [[gerrit:972488|styles: Fix stylesheet validation issues]] [production]
22:19 <bking@deploy2002> helmfile [staging] DONE helmfile.d/services/rdf-streaming-updater: apply [production]
22:19 <bking@deploy2002> helmfile [staging] START helmfile.d/services/rdf-streaming-updater: apply [production]
22:19 <bking@deploy2002> helmfile [staging] DONE helmfile.d/services/rdf-streaming-updater: apply [production]
22:19 <bking@deploy2002> helmfile [staging] START helmfile.d/services/rdf-streaming-updater: apply [production]
22:18 <ladsgroup@deploy2002> Finished scap: Backport for [[gerrit:971604|Replace WikimediaUI Base with Codex design tokens (T331403 T334934)]] (duration: 09m 15s) [production]
22:13 <ladsgroup@deploy2002> ladsgroup and volker-e: Continuing with sync [production]
22:10 <ladsgroup@deploy2002> ladsgroup and volker-e: Backport for [[gerrit:971604|Replace WikimediaUI Base with Codex design tokens (T331403 T334934)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
22:09 <ladsgroup@deploy2002> Started scap: Backport for [[gerrit:971604|Replace WikimediaUI Base with Codex design tokens (T331403 T334934)]] [production]
22:00 <bking@cumin2002> END (PASS) - Cookbook sre.elasticsearch.rolling-operation (exit_code=0) Operation.RESTART (1 nodes at a time) for ElasticSearch cluster cloudelastic: cloudelastic cluster restart - bking@cumin2002 - T350703 [production]
21:58 <tgr@deploy2002> Finished scap: Backport for [[gerrit:972262|Fix centralauthtoken key schema migration (T347223 T350723)]] (duration: 13m 17s) [production]
21:53 <tgr@deploy2002> tgr: Continuing with sync [production]
21:46 <tgr@deploy2002> tgr: Backport for [[gerrit:972262|Fix centralauthtoken key schema migration (T347223 T350723)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
21:45 <tgr@deploy2002> Started scap: Backport for [[gerrit:972262|Fix centralauthtoken key schema migration (T347223 T350723)]] [production]
21:37 <tgr@deploy2002> Finished scap: Backport for [[gerrit:972435|CentralAuth: Clear domain cookie when setting non-domain cookie (T350695)]] (duration: 20m 27s) [production]
21:36 <bking@cumin2002> START - Cookbook sre.elasticsearch.rolling-operation Operation.RESTART (1 nodes at a time) for ElasticSearch cluster cloudelastic: cloudelastic cluster restart - bking@cumin2002 - T350703 [production]
21:35 <tzatziki> changing email for User:Rlayton-WMF [production]
21:32 <tgr@deploy2002> tgr: Continuing with sync [production]
21:18 <tgr@deploy2002> tgr: Backport for [[gerrit:972435|CentralAuth: Clear domain cookie when setting non-domain cookie (T350695)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
21:17 <tgr@deploy2002> Started scap: Backport for [[gerrit:972435|CentralAuth: Clear domain cookie when setting non-domain cookie (T350695)]] [production]
21:14 <tgr@deploy2002> Finished scap: Backport for [[gerrit:972462|Enable edit check on fonwiki (T350634)]] (duration: 09m 45s) [production]
21:09 <tgr@deploy2002> tgr and kemayo: Continuing with sync [production]
21:06 <tgr@deploy2002> tgr and kemayo: Backport for [[gerrit:972462|Enable edit check on fonwiki (T350634)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
21:05 <tgr@deploy2002> Started scap: Backport for [[gerrit:972462|Enable edit check on fonwiki (T350634)]] [production]
20:47 <dzahn@cumin1001> END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) for new host stewards1001.eqiad.wmnet [production]
20:47 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host stewards1001.eqiad.wmnet with OS bookworm [production]
20:34 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on stewards1001.eqiad.wmnet with reason: host reimage [production]
20:31 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on stewards1001.eqiad.wmnet with reason: host reimage [production]
20:20 <dzahn@cumin1001> START - Cookbook sre.hosts.reimage for host stewards1001.eqiad.wmnet with OS bookworm [production]
20:19 <dzahn@cumin1001> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.ganeti.makevm: created new VM stewards1001.eqiad.wmnet - dzahn@cumin1001" [production]
20:18 <dzahn@cumin1001> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.ganeti.makevm: created new VM stewards1001.eqiad.wmnet - dzahn@cumin1001" [production]
20:18 <dzahn@cumin1001> END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) stewards1001.eqiad.wmnet on all recursors [production]
20:18 <dzahn@cumin1001> START - Cookbook sre.dns.wipe-cache stewards1001.eqiad.wmnet on all recursors [production]
20:18 <dzahn@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
20:18 <dzahn@cumin1001> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM stewards1001.eqiad.wmnet - dzahn@cumin1001" [production]
20:16 <dzahn@cumin1001> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM stewards1001.eqiad.wmnet - dzahn@cumin1001" [production]
19:58 <wfan> payments-wiki change from 1d66a20f to 8c073c23, config revision changed from c841729a to 5c00d761 [production]
19:54 <marostegui@cumin1001> dbctl commit (dc=all): 'db1192 (re)pooling @ 100%: Change binlog format', diff saved to https://phabricator.wikimedia.org/P53158 and previous config saved to /var/cache/conftool/dbconfig/20231107-195420-root.json [production]
19:39 <marostegui@cumin1001> dbctl commit (dc=all): 'db1192 (re)pooling @ 75%: Change binlog format', diff saved to https://phabricator.wikimedia.org/P53157 and previous config saved to /var/cache/conftool/dbconfig/20231107-193915-root.json [production]
19:32 <bking@deploy2002> helmfile [staging] DONE helmfile.d/services/rdf-streaming-updater: apply [production]
19:32 <bking@deploy2002> helmfile [staging] START helmfile.d/services/rdf-streaming-updater: apply [production]
19:26 <jbond@cumin1001> END (PASS) - Cookbook sre.puppet.migrate-role (exit_code=0) for role: mail::mx [production]
19:25 <bking@deploy2002> helmfile [staging] DONE helmfile.d/services/rdf-streaming-updater: apply [production]