1-50 of 10000 results (57ms)
2024-01-14 §
15:47 <taavi@deploy2002> Finished scap: Backport for [[gerrit:990396|Log IpReputation channel as debug (T354928)]] (duration: 26m 49s) [production]
15:36 <taavi@deploy2002> taavi: Continuing with sync [production]
15:35 <taavi@deploy2002> taavi: Backport for [[gerrit:990396|Log IpReputation channel as debug (T354928)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
15:20 <taavi@deploy2002> Started scap: Backport for [[gerrit:990396|Log IpReputation channel as debug (T354928)]] [production]
15:01 <andrewbogott> manually emptying /srv/mediawiki/images/wikitech/archive on wikitech-static; the maintenance script didn't do it and the host is failing due to a full disk [production]
15:01 <andrewbogott> running deleteArchivedFiles.php on wikitech-static [production]
2024-01-12 §
23:49 <dzahn@cumin1001> END (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Conniecc1 out of all services on: 2213 hosts [production]
23:47 <dzahn@cumin1001> START - Cookbook sre.idm.logout Logging Conniecc1 out of all services on: 2213 hosts [production]
22:52 <dzahn@cumin1001> END (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Conniecc1 out of all services on: 2213 hosts [production]
22:51 <dzahn@cumin1001> START - Cookbook sre.idm.logout Logging Conniecc1 out of all services on: 2213 hosts [production]
22:29 <dzahn@cumin1001> END (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Conniecc1 out of all services on: 2213 hosts [production]
22:28 <dzahn@cumin1001> START - Cookbook sre.idm.logout Logging Conniecc1 out of all services on: 2213 hosts [production]
18:07 <mutante> aphlict1002 - systemctl start logrotate [production]
17:18 <tchanders@deploy2002> helmfile [codfw] DONE helmfile.d/services/ipoid: apply [production]
17:18 <tchanders@deploy2002> helmfile [codfw] START helmfile.d/services/ipoid: apply [production]
17:17 <tchanders@deploy2002> helmfile [eqiad] DONE helmfile.d/services/ipoid: apply [production]
17:16 <tchanders@deploy2002> helmfile [eqiad] START helmfile.d/services/ipoid: apply [production]
17:10 <tchanders@deploy2002> helmfile [staging] DONE helmfile.d/services/ipoid: apply [production]
17:09 <tchanders@deploy2002> helmfile [staging] START helmfile.d/services/ipoid: apply [production]
16:52 <cgoubert@deploy2002> helmfile [codfw] [main] DONE helmfile.d/services/mw-jobrunner : sync [production]
16:52 <cgoubert@deploy2002> helmfile [codfw] [main] START helmfile.d/services/mw-jobrunner : sync [production]
16:51 <cgoubert@deploy2002> helmfile [eqiad] [main] DONE helmfile.d/services/mw-jobrunner : sync [production]
16:51 <cgoubert@deploy2002> helmfile [eqiad] [main] START helmfile.d/services/mw-jobrunner : sync [production]
16:20 <cgoubert@deploy2002> helmfile [codfw] DONE helmfile.d/services/mw-api-int: apply [production]
16:20 <cgoubert@deploy2002> helmfile [codfw] START helmfile.d/services/mw-api-int: apply [production]
16:20 <cgoubert@deploy2002> helmfile [eqiad] DONE helmfile.d/services/mw-api-int: apply [production]
16:19 <cgoubert@deploy2002> helmfile [eqiad] START helmfile.d/services/mw-api-int: apply [production]
15:46 <klausman@deploy2002> helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'llm' for release 'main' . [production]
15:37 <klausman@deploy2002> helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'llm' for release 'main' . [production]
15:14 <klausman@deploy2002> helmfile [ml-staging-codfw] DONE helmfile.d/admin 'apply'. [production]
15:14 <klausman@deploy2002> helmfile [ml-staging-codfw] START helmfile.d/admin 'apply'. [production]
14:32 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db2114.codfw.wmnet with reason: Maintenance [production]
14:31 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 8:00:00 on db2114.codfw.wmnet with reason: Maintenance [production]
14:04 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2193 (T354336)', diff saved to https://phabricator.wikimedia.org/P54714 and previous config saved to /var/cache/conftool/dbconfig/20240112-140423-marostegui.json [production]
13:49 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2193', diff saved to https://phabricator.wikimedia.org/P54713 and previous config saved to /var/cache/conftool/dbconfig/20240112-134916-marostegui.json [production]
13:34 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2193', diff saved to https://phabricator.wikimedia.org/P54712 and previous config saved to /var/cache/conftool/dbconfig/20240112-133410-marostegui.json [production]
13:19 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2193 (T354336)', diff saved to https://phabricator.wikimedia.org/P54711 and previous config saved to /var/cache/conftool/dbconfig/20240112-131904-marostegui.json [production]
12:59 <marostegui@cumin1002> dbctl commit (dc=all): 'Depooling db2193 (T354336)', diff saved to https://phabricator.wikimedia.org/P54710 and previous config saved to /var/cache/conftool/dbconfig/20240112-125944-marostegui.json [production]
12:59 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db2193.codfw.wmnet with reason: Maintenance [production]
12:59 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 8:00:00 on db2193.codfw.wmnet with reason: Maintenance [production]
12:59 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2180 (T354336)', diff saved to https://phabricator.wikimedia.org/P54709 and previous config saved to /var/cache/conftool/dbconfig/20240112-125921-marostegui.json [production]
12:44 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2180', diff saved to https://phabricator.wikimedia.org/P54708 and previous config saved to /var/cache/conftool/dbconfig/20240112-124416-marostegui.json [production]
12:33 <urbanecm> [urbanecm@mwmaint2002 ~]$ mwscript extensions/CentralAuth/maintenance/fixStuckGlobalRename.php --wiki=dewiki --logwiki=metawiki 'Osip Knecht' 'Artquichotte39' [production]
12:29 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2180', diff saved to https://phabricator.wikimedia.org/P54707 and previous config saved to /var/cache/conftool/dbconfig/20240112-122909-marostegui.json [production]
12:14 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2180 (T354336)', diff saved to https://phabricator.wikimedia.org/P54706 and previous config saved to /var/cache/conftool/dbconfig/20240112-121402-marostegui.json [production]
12:11 <marostegui@cumin1002> dbctl commit (dc=all): 'Depooling db2180 (T354336)', diff saved to https://phabricator.wikimedia.org/P54704 and previous config saved to /var/cache/conftool/dbconfig/20240112-121150-marostegui.json [production]
12:11 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db2180.codfw.wmnet with reason: Maintenance [production]
12:11 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 8:00:00 on db2180.codfw.wmnet with reason: Maintenance [production]
12:11 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2171:3316 (T354336)', diff saved to https://phabricator.wikimedia.org/P54703 and previous config saved to /var/cache/conftool/dbconfig/20240112-121127-marostegui.json [production]
12:06 <brouberol@deploy2002> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/spark-history: apply [production]