6001-6050 of 10000 results (86ms)
2022-11-01 §
10:42 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1096:3316 (T318955)', diff saved to https://phabricator.wikimedia.org/P37286 and previous config saved to /var/cache/conftool/dbconfig/20221101-104154-ladsgroup.json [production]
10:42 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 8:00:00 on db2117.codfw.wmnet with reason: Maintenance [production]
10:41 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db1096.eqiad.wmnet with reason: Maintenance [production]
10:41 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 8:00:00 on db1096.eqiad.wmnet with reason: Maintenance [production]
10:39 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2098.codfw.wmnet with reason: Maintenance [production]
10:39 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1099:3318 (T318605)', diff saved to https://phabricator.wikimedia.org/P37285 and previous config saved to /var/cache/conftool/dbconfig/20221101-103934-ladsgroup.json [production]
10:39 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1099.eqiad.wmnet with reason: Maintenance [production]
10:39 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2098.codfw.wmnet with reason: Maintenance [production]
10:39 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1099.eqiad.wmnet with reason: Maintenance [production]
09:43 <moritzm> imported quickstack 20161026-1+deb12u1 to apt.wikimedia.org/bookworm-wikimedia T321783 [production]
08:32 <moritzm> draining ganeti1028 for eventual reimage T311687 [production]
08:29 <jmm@cumin2002> END (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Faidon Liambotis out of all services on: 1203 hosts [production]
08:29 <jmm@cumin2002> START - Cookbook sre.idm.logout Logging Faidon Liambotis out of all services on: 1203 hosts [production]
08:29 <jmm@cumin2002> END (PASS) - Cookbook sre.idm.logout (exit_code=0) Logging Faidon Liambotis out of all services on: 802 hosts [production]
08:27 <jmm@cumin2002> START - Cookbook sre.idm.logout Logging Faidon Liambotis out of all services on: 802 hosts [production]
03:51 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mw-debug: apply [production]
03:45 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mw-debug: apply [production]
03:45 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mw-debug: apply [production]
03:38 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mw-debug: apply [production]
03:36 <mwpresync@deploy1002> Finished scap: testwikis wikis to 1.40.0-wmf.8 refs T320513 (duration: 33m 56s) [production]
03:08 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mw-debug: apply [production]
03:07 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mw-debug: apply [production]
03:07 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mw-debug: apply [production]
03:06 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mw-debug: apply [production]
03:02 <mwpresync@deploy1002> Started scap: testwikis wikis to 1.40.0-wmf.8 refs T320513 [production]
02:31 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mw-debug: apply [production]
02:30 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mw-debug: apply [production]
02:30 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mw-debug: apply [production]
02:29 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mw-debug: apply [production]
02:09 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mw-debug: apply [production]
02:08 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mw-debug: apply [production]
02:08 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mw-debug: apply [production]
02:07 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mw-debug: apply [production]
2022-10-31 §
22:23 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on dbstore1007.eqiad.wmnet with reason: Maintenance [production]
22:22 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on dbstore1007.eqiad.wmnet with reason: Maintenance [production]
22:21 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1197 (T318605)', diff saved to https://phabricator.wikimedia.org/P37282 and previous config saved to /var/cache/conftool/dbconfig/20221031-222151-ladsgroup.json [production]
22:06 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1197', diff saved to https://phabricator.wikimedia.org/P37281 and previous config saved to /var/cache/conftool/dbconfig/20221031-220645-ladsgroup.json [production]
21:51 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1197', diff saved to https://phabricator.wikimedia.org/P37280 and previous config saved to /var/cache/conftool/dbconfig/20221031-215138-ladsgroup.json [production]
21:36 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1197 (T318605)', diff saved to https://phabricator.wikimedia.org/P37279 and previous config saved to /var/cache/conftool/dbconfig/20221031-213632-ladsgroup.json [production]
21:27 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1197 (T318605)', diff saved to https://phabricator.wikimedia.org/P37278 and previous config saved to /var/cache/conftool/dbconfig/20221031-212749-ladsgroup.json [production]
21:27 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1197.eqiad.wmnet with reason: Maintenance [production]
21:27 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1197.eqiad.wmnet with reason: Maintenance [production]
21:27 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1188 (T318605)', diff saved to https://phabricator.wikimedia.org/P37277 and previous config saved to /var/cache/conftool/dbconfig/20221031-212717-ladsgroup.json [production]
21:12 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1188', diff saved to https://phabricator.wikimedia.org/P37276 and previous config saved to /var/cache/conftool/dbconfig/20221031-211210-ladsgroup.json [production]
20:57 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1188', diff saved to https://phabricator.wikimedia.org/P37275 and previous config saved to /var/cache/conftool/dbconfig/20221031-205703-ladsgroup.json [production]
20:41 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1188 (T318605)', diff saved to https://phabricator.wikimedia.org/P37274 and previous config saved to /var/cache/conftool/dbconfig/20221031-204157-ladsgroup.json [production]
20:33 <otto@deploy1002> Synchronized wmf-config/InitialiseSettings.php: No-op sync of InitialiseSettings.php to declare stream rc0.mediawiki.page_change. This stream is disabled everywhere by default, and only enabled in beta for now. - T311129 (duration: 03m 42s) [production]
20:33 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1188 (T318605)', diff saved to https://phabricator.wikimedia.org/P37273 and previous config saved to /var/cache/conftool/dbconfig/20221031-203319-ladsgroup.json [production]
20:33 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1188.eqiad.wmnet with reason: Maintenance [production]
20:33 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1188.eqiad.wmnet with reason: Maintenance [production]