901-950 of 10000 results (23ms)
2025-05-27 §
07:44 <ayounsi@cumin1002> START - Cookbook sre.network.peering with action 'email' for AS: 45287 [production]
07:44 <hashar@deploy1003> hashar, bunnypranav, samwilson, ahonc: Backport for [[gerrit:1149805|core-Namespaces: Update Malay wiki (mswiki) namespace aliases (T394603)]], [[gerrit:1150778|Add user group extendedmover to ukwiki (T395285)]], [[gerrit:1150629|InitialiseSettings: wgTemplateDataEnableDiscovery on plwiki and arwiki (T377975)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can [production]
07:43 <taavi@cumin1002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "set cloudcephmon1004 as active after disk replacement - taavi@cumin1002" [production]
07:43 <ayounsi@cumin1002> END (PASS) - Cookbook sre.network.peering (exit_code=0) with action 'email' for AS: 209453 [production]
07:43 <ayounsi@cumin1002> START - Cookbook sre.network.peering with action 'email' for AS: 209453 [production]
07:42 <taavi@cumin1002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "set cloudcephmon1004 as active after disk replacement - taavi@cumin1002" [production]
07:42 <ayounsi@cumin1002> END (PASS) - Cookbook sre.network.peering (exit_code=0) with action 'email' for AS: 134823 [production]
07:42 <ayounsi@cumin1002> START - Cookbook sre.network.peering with action 'email' for AS: 134823 [production]
07:39 <hashar@deploy1003> Started scap sync-world: Backport for [[gerrit:1149805|core-Namespaces: Update Malay wiki (mswiki) namespace aliases (T394603)]], [[gerrit:1150778|Add user group extendedmover to ukwiki (T395285)]], [[gerrit:1150629|InitialiseSettings: wgTemplateDataEnableDiscovery on plwiki and arwiki (T377975)]] [production]
07:39 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host sretest1002.eqiad.wmnet [production]
07:38 <ayounsi@cumin1002> END (PASS) - Cookbook sre.network.peering (exit_code=0) with action 'email' for AS: 269180 [production]
07:37 <ayounsi@cumin1002> START - Cookbook sre.network.peering with action 'email' for AS: 269180 [production]
07:35 <ayounsi@cumin1002> END (ERROR) - Cookbook sre.network.peering (exit_code=97) with action 'email' for AS: 52468 [production]
07:35 <ayounsi@cumin1002> START - Cookbook sre.network.peering with action 'email' for AS: 52468 [production]
07:34 <jmm@cumin2002> END (PASS) - Cookbook sre.ganeti.changedisk (exit_code=0) for changing disk type of doh7002.wikimedia.org to plain [production]
07:33 <jmm@cumin2002> START - Cookbook sre.ganeti.changedisk for changing disk type of doh7002.wikimedia.org to plain [production]
07:33 <ayounsi@cumin1002> END (PASS) - Cookbook sre.network.peering (exit_code=0) with action 'email' for AS: 15932 [production]
07:33 <ayounsi@cumin1002> START - Cookbook sre.network.peering with action 'email' for AS: 15932 [production]
07:32 <ayounsi@cumin1002> END (PASS) - Cookbook sre.network.peering (exit_code=0) with action 'email' for AS: 10089 [production]
07:32 <ayounsi@cumin1002> START - Cookbook sre.network.peering with action 'email' for AS: 10089 [production]
07:32 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host mc-misc2002.codfw.wmnet [production]
07:32 <jmm@cumin2002> END (PASS) - Cookbook sre.ganeti.changedisk (exit_code=0) for changing disk type of durum7002.magru.wmnet to plain [production]
07:31 <jmm@cumin2002> START - Cookbook sre.ganeti.changedisk for changing disk type of durum7002.magru.wmnet to plain [production]
07:30 <ayounsi@cumin1002> END (PASS) - Cookbook sre.network.peering (exit_code=0) with action 'email' for AS: 17806 [production]
07:30 <ayounsi@cumin1002> START - Cookbook sre.network.peering with action 'email' for AS: 17806 [production]
07:23 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host mc-misc2002.codfw.wmnet [production]
07:19 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host mc-misc2002.codfw.wmnet [production]
07:10 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host mc-misc2002.codfw.wmnet [production]
07:07 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet [production]
07:05 <ayounsi@cumin1002> END (PASS) - Cookbook sre.network.peering (exit_code=0) with action 'email' for AS: 15133 [production]
07:05 <ayounsi@cumin1002> START - Cookbook sre.network.peering with action 'email' for AS: 15133 [production]
07:01 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet [production]
06:58 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet [production]
06:38 <moritzm> failover Ganeti master in magru/B4 to ganeti7004 T394263 [production]
06:31 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet [production]
04:01 <mwpresync@deploy1003> Pruned MediaWiki: 1.44.0-wmf.28 (duration: 01m 50s) [production]
03:49 <mwpresync@deploy1003> Finished scap sync-world: testwikis to 1.45.0-wmf.3 refs T392173 (duration: 46m 49s) [production]
03:02 <mwpresync@deploy1003> Started scap sync-world: testwikis to 1.45.0-wmf.3 refs T392173 [production]
00:20 <dani@deploy1003> helmfile [codfw] DONE helmfile.d/services/miscweb: apply [production]
00:20 <dani@deploy1003> helmfile [codfw] START helmfile.d/services/miscweb: apply [production]
00:20 <dani@deploy1003> helmfile [eqiad] DONE helmfile.d/services/miscweb: apply [production]
00:20 <dani@deploy1003> helmfile [eqiad] START helmfile.d/services/miscweb: apply [production]
00:20 <dani@deploy1003> helmfile [staging] DONE helmfile.d/services/miscweb: apply [production]
00:20 <dani@deploy1003> helmfile [staging] START helmfile.d/services/miscweb: apply [production]
2025-05-26 §
21:56 <fceratto@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2228 (T395241)', diff saved to https://phabricator.wikimedia.org/P76461 and previous config saved to /var/cache/conftool/dbconfig/20250526-215649-fceratto.json [production]
21:41 <fceratto@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2228', diff saved to https://phabricator.wikimedia.org/P76460 and previous config saved to /var/cache/conftool/dbconfig/20250526-214142-fceratto.json [production]
21:26 <fceratto@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2228', diff saved to https://phabricator.wikimedia.org/P76459 and previous config saved to /var/cache/conftool/dbconfig/20250526-212634-fceratto.json [production]
21:11 <fceratto@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2228 (T395241)', diff saved to https://phabricator.wikimedia.org/P76458 and previous config saved to /var/cache/conftool/dbconfig/20250526-211127-fceratto.json [production]
21:04 <fceratto@cumin1002> dbctl commit (dc=all): 'Depooling db2228 (T395241)', diff saved to https://phabricator.wikimedia.org/P76457 and previous config saved to /var/cache/conftool/dbconfig/20250526-210445-fceratto.json [production]
21:04 <fceratto@cumin1002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on db2186.codfw.wmnet with reason: Maintenance [production]