6551-6600 of 10000 results (110ms)
2024-01-15 §
09:48 <ladsgroup@deploy2002> ladsgroup: Backport for [[gerrit:990424|SecurePoll: Adding updated voterlist files (T349263)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
09:46 <ladsgroup@deploy2002> Started scap: Backport for [[gerrit:990424|SecurePoll: Adding updated voterlist files (T349263)]] [production]
09:16 <pfischer@deploy2002> helmfile [staging] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
09:16 <pfischer@deploy2002> helmfile [staging] START helmfile.d/services/cirrus-streaming-updater: apply [production]
09:15 <pfischer@deploy2002> helmfile [staging] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
09:15 <pfischer@deploy2002> helmfile [staging] START helmfile.d/services/cirrus-streaming-updater: apply [production]
09:15 <pfischer@deploy2002> helmfile [staging] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
09:14 <pfischer@deploy2002> helmfile [staging] START helmfile.d/services/cirrus-streaming-updater: apply [production]
08:45 <filippo@deploy2002> Finished deploy [performance/arc-lamp@67389a0]: (no justification provided) (duration: 00m 05s) [production]
08:45 <filippo@deploy2002> Started deploy [performance/arc-lamp@67389a0]: (no justification provided) [production]
08:23 <dcausse@deploy2002> Finished scap: Backport for [[gerrit:990029|enable page_rerender for 5th batch of wikis (T351503)]] (duration: 11m 40s) [production]
08:17 <dcausse@deploy2002> pfischer and dcausse: Continuing with sync [production]
08:13 <dcausse@deploy2002> pfischer and dcausse: Backport for [[gerrit:990029|enable page_rerender for 5th batch of wikis (T351503)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
08:12 <dcausse@deploy2002> Started scap: Backport for [[gerrit:990029|enable page_rerender for 5th batch of wikis (T351503)]] [production]
04:57 <andrewbogott> restarting wikitech-static, oom [production]
2024-01-14 §
15:47 <taavi@deploy2002> Finished scap: Backport for [[gerrit:990396|Log IpReputation channel as debug (T354928)]] (duration: 26m 49s) [production]
15:36 <taavi@deploy2002> taavi: Continuing with sync [production]
15:35 <taavi@deploy2002> taavi: Backport for [[gerrit:990396|Log IpReputation channel as debug (T354928)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
15:20 <taavi@deploy2002> Started scap: Backport for [[gerrit:990396|Log IpReputation channel as debug (T354928)]] [production]
15:01 <andrewbogott> manually emptying /srv/mediawiki/images/wikitech/archive on wikitech-static; the maintenance script didn't do it and the host is failing due to a full disk [production]
15:01 <andrewbogott> running deleteArchivedFiles.php on wikitech-static [production]
2024-01-12 §
23:49 <dzahn@cumin1001> END (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Conniecc1 out of all services on: 2213 hosts [production]
23:47 <dzahn@cumin1001> START - Cookbook sre.idm.logout Logging Conniecc1 out of all services on: 2213 hosts [production]
22:52 <dzahn@cumin1001> END (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Conniecc1 out of all services on: 2213 hosts [production]
22:51 <dzahn@cumin1001> START - Cookbook sre.idm.logout Logging Conniecc1 out of all services on: 2213 hosts [production]
22:29 <dzahn@cumin1001> END (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Conniecc1 out of all services on: 2213 hosts [production]
22:28 <dzahn@cumin1001> START - Cookbook sre.idm.logout Logging Conniecc1 out of all services on: 2213 hosts [production]
18:07 <mutante> aphlict1002 - systemctl start logrotate [production]
17:18 <tchanders@deploy2002> helmfile [codfw] DONE helmfile.d/services/ipoid: apply [production]
17:18 <tchanders@deploy2002> helmfile [codfw] START helmfile.d/services/ipoid: apply [production]
17:17 <tchanders@deploy2002> helmfile [eqiad] DONE helmfile.d/services/ipoid: apply [production]
17:16 <tchanders@deploy2002> helmfile [eqiad] START helmfile.d/services/ipoid: apply [production]
17:10 <tchanders@deploy2002> helmfile [staging] DONE helmfile.d/services/ipoid: apply [production]
17:09 <tchanders@deploy2002> helmfile [staging] START helmfile.d/services/ipoid: apply [production]
16:52 <cgoubert@deploy2002> helmfile [codfw] [main] DONE helmfile.d/services/mw-jobrunner : sync [production]
16:52 <cgoubert@deploy2002> helmfile [codfw] [main] START helmfile.d/services/mw-jobrunner : sync [production]
16:51 <cgoubert@deploy2002> helmfile [eqiad] [main] DONE helmfile.d/services/mw-jobrunner : sync [production]
16:51 <cgoubert@deploy2002> helmfile [eqiad] [main] START helmfile.d/services/mw-jobrunner : sync [production]
16:20 <cgoubert@deploy2002> helmfile [codfw] DONE helmfile.d/services/mw-api-int: apply [production]
16:20 <cgoubert@deploy2002> helmfile [codfw] START helmfile.d/services/mw-api-int: apply [production]
16:20 <cgoubert@deploy2002> helmfile [eqiad] DONE helmfile.d/services/mw-api-int: apply [production]
16:19 <cgoubert@deploy2002> helmfile [eqiad] START helmfile.d/services/mw-api-int: apply [production]
15:46 <klausman@deploy2002> helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'llm' for release 'main' . [production]
15:37 <klausman@deploy2002> helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'llm' for release 'main' . [production]
15:14 <klausman@deploy2002> helmfile [ml-staging-codfw] DONE helmfile.d/admin 'apply'. [production]
15:14 <klausman@deploy2002> helmfile [ml-staging-codfw] START helmfile.d/admin 'apply'. [production]
14:32 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db2114.codfw.wmnet with reason: Maintenance [production]
14:31 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 8:00:00 on db2114.codfw.wmnet with reason: Maintenance [production]
14:04 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2193 (T354336)', diff saved to https://phabricator.wikimedia.org/P54714 and previous config saved to /var/cache/conftool/dbconfig/20240112-140423-marostegui.json [production]
13:49 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2193', diff saved to https://phabricator.wikimedia.org/P54713 and previous config saved to /var/cache/conftool/dbconfig/20240112-134916-marostegui.json [production]