1-50 of 10000 results (46ms)
2024-02-21 ยง
15:24 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1236', diff saved to https://phabricator.wikimedia.org/P57575 and previous config saved to /var/cache/conftool/dbconfig/20240221-152420-arnaudb.json [production]
15:21 <cmooney@cumin1002> START - Cookbook sre.hosts.reimage for host db2137.codfw.wmnet with OS bookworm [production]
15:19 <marostegui@cumin1002> dbctl commit (dc=all): 'es2026 (re)pooling @ 5%: After migration to 10.6', diff saved to https://phabricator.wikimedia.org/P57574 and previous config saved to /var/cache/conftool/dbconfig/20240221-151909-root.json [production]
15:12 <hnowlan@deploy2002> helmfile [codfw] DONE helmfile.d/services/thumbor: apply [production]
15:12 <hnowlan@deploy2002> helmfile [codfw] START helmfile.d/services/thumbor: apply [production]
15:10 <cmooney@cumin1002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
15:10 <cmooney@cumin1002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: update dns entries for private1-b-codfw - cmooney@cumin1002" [production]
14:55 <TheresNoTime> UTC afternoon backport window done [production]
14:54 <hnowlan@deploy2002> helmfile [codfw] START helmfile.d/services/thumbor: apply [production]
14:54 <cmooney@cumin1002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
14:54 <cmooney@cumin1002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: update dns entries for private1-a-codfw - cmooney@cumin1002" [production]
14:54 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1236 (T357189)', diff saved to https://phabricator.wikimedia.org/P57570 and previous config saved to /var/cache/conftool/dbconfig/20240221-145407-arnaudb.json [production]
14:53 <cmooney@cumin1002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: update dns entries for private1-a-codfw - cmooney@cumin1002" [production]
14:53 <hnowlan@deploy2002> helmfile [codfw] DONE helmfile.d/services/thumbor: apply [production]
14:52 <hnowlan@deploy2002> helmfile [codfw] START helmfile.d/services/thumbor: apply [production]
14:49 <hnowlan@deploy2002> helmfile [codfw] DONE helmfile.d/services/thumbor: apply [production]
14:48 <hnowlan@deploy2002> helmfile [codfw] START helmfile.d/services/thumbor: apply [production]
14:47 <TheresNoTime> [samtar@mwmaint2002 ~]$ mwscript namespaceDupes.php --wiki hewikinews --fix #T349581 [production]
14:47 <arnaudb@cumin1002> dbctl commit (dc=all): 'Depooling db1236 (T357189)', diff saved to https://phabricator.wikimedia.org/P57569 and previous config saved to /var/cache/conftool/dbconfig/20240221-144702-arnaudb.json [production]
14:46 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1236.eqiad.wmnet with reason: Maintenance [production]
14:46 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1236.eqiad.wmnet with reason: Maintenance [production]
14:46 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1227 (T357189)', diff saved to https://phabricator.wikimedia.org/P57568 and previous config saved to /var/cache/conftool/dbconfig/20240221-144641-arnaudb.json [production]
14:46 <samtar@deploy2002> Finished scap: Backport for [[gerrit:1005085|cswiki, commonswiki, enwiki: Lift IP cap for WikiGap Editathon]], [[gerrit:990077|mywiki: create portal and draft namespace (T352424)]] (duration: 20m 23s) [production]
14:45 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2170', diff saved to https://phabricator.wikimedia.org/P57567 and previous config saved to /var/cache/conftool/dbconfig/20240221-144536-marostegui.json [production]
14:44 <cmooney@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on es2026.codfw.wmnet with reason: host reimage [production]
14:44 <cmooney@cumin1002> START - Cookbook sre.dns.netbox [production]
14:43 <hnowlan@deploy2002> helmfile [codfw] DONE helmfile.d/services/thumbor: apply [production]
14:42 <cmooney@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on es2026.codfw.wmnet with reason: host reimage [production]
14:40 <aborrero@cumin1002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudvirt1033.eqiad.wmnet with OS bookworm [production]
14:38 <cmooney@cumin1002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
14:38 <cmooney@cumin1002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: update dns entries for private1-a-codfw - cmooney@cumin1002" [production]
14:37 <cmooney@cumin1002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: update dns entries for private1-a-codfw - cmooney@cumin1002" [production]
14:37 <samtar@deploy2002> samtar and anzx: Continuing with sync [production]
14:34 <cmooney@cumin1002> START - Cookbook sre.dns.netbox [production]
14:33 <hnowlan@deploy2002> helmfile [codfw] START helmfile.d/services/thumbor: apply [production]
14:33 <hnowlan@deploy2002> helmfile [codfw] DONE helmfile.d/services/thumbor: apply [production]
14:33 <hnowlan@deploy2002> helmfile [codfw] START helmfile.d/services/thumbor: apply [production]
14:32 <arnaudb@cumin1002> dbctl commit (dc=all): 'db1180 (re)pooling @ 100%: Maintenance done', diff saved to https://phabricator.wikimedia.org/P57566 and previous config saved to /var/cache/conftool/dbconfig/20240221-143239-arnaudb.json [production]
14:31 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1227', diff saved to https://phabricator.wikimedia.org/P57565 and previous config saved to /var/cache/conftool/dbconfig/20240221-143133-arnaudb.json [production]
14:31 <arnaudb@cumin1002> dbctl commit (dc=all): 'db1213 (re)pooling @ 100%: Maintenance done', diff saved to https://phabricator.wikimedia.org/P57564 and previous config saved to /var/cache/conftool/dbconfig/20240221-143120-arnaudb.json [production]
14:30 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2170', diff saved to https://phabricator.wikimedia.org/P57563 and previous config saved to /var/cache/conftool/dbconfig/20240221-143030-marostegui.json [production]
14:27 <samtar@deploy2002> samtar and anzx: Backport for [[gerrit:1005085|cswiki, commonswiki, enwiki: Lift IP cap for WikiGap Editathon]], [[gerrit:990077|mywiki: create portal and draft namespace (T352424)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
14:26 <samtar@deploy2002> Started scap: Backport for [[gerrit:1005085|cswiki, commonswiki, enwiki: Lift IP cap for WikiGap Editathon]], [[gerrit:990077|mywiki: create portal and draft namespace (T352424)]] [production]
14:24 <cmooney@cumin1002> START - Cookbook sre.hosts.reimage for host es2026.codfw.wmnet with OS bookworm [production]
14:23 <samtar@deploy2002> Finished scap: Backport for [[gerrit:1005109|zhwiki: Create group ipblock-exempt-grantor (T357991)]] (duration: 11m 05s) [production]
14:22 <jmm@cumin2002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "new apt server in codfw - jmm@cumin2002 - T331613" [production]
14:20 <jmm@cumin2002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "new apt server in codfw - jmm@cumin2002 - T331613" [production]
14:17 <arnaudb@cumin1002> dbctl commit (dc=all): 'db1180 (re)pooling @ 75%: Maintenance done', diff saved to https://phabricator.wikimedia.org/P57562 and previous config saved to /var/cache/conftool/dbconfig/20240221-141734-arnaudb.json [production]
14:16 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1227', diff saved to https://phabricator.wikimedia.org/P57561 and previous config saved to /var/cache/conftool/dbconfig/20240221-141627-arnaudb.json [production]
14:16 <arnaudb@cumin1002> dbctl commit (dc=all): 'db1213 (re)pooling @ 75%: Maintenance done', diff saved to https://phabricator.wikimedia.org/P57560 and previous config saved to /var/cache/conftool/dbconfig/20240221-141615-arnaudb.json [production]