1-50 of 10000 results (58ms)
2022-10-31 ยง
22:23 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on dbstore1007.eqiad.wmnet with reason: Maintenance [production]
22:22 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on dbstore1007.eqiad.wmnet with reason: Maintenance [production]
22:21 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1197 (T318605)', diff saved to https://phabricator.wikimedia.org/P37282 and previous config saved to /var/cache/conftool/dbconfig/20221031-222151-ladsgroup.json [production]
22:06 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1197', diff saved to https://phabricator.wikimedia.org/P37281 and previous config saved to /var/cache/conftool/dbconfig/20221031-220645-ladsgroup.json [production]
21:51 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1197', diff saved to https://phabricator.wikimedia.org/P37280 and previous config saved to /var/cache/conftool/dbconfig/20221031-215138-ladsgroup.json [production]
21:36 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1197 (T318605)', diff saved to https://phabricator.wikimedia.org/P37279 and previous config saved to /var/cache/conftool/dbconfig/20221031-213632-ladsgroup.json [production]
21:27 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1197 (T318605)', diff saved to https://phabricator.wikimedia.org/P37278 and previous config saved to /var/cache/conftool/dbconfig/20221031-212749-ladsgroup.json [production]
21:27 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1197.eqiad.wmnet with reason: Maintenance [production]
21:27 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1197.eqiad.wmnet with reason: Maintenance [production]
21:27 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1188 (T318605)', diff saved to https://phabricator.wikimedia.org/P37277 and previous config saved to /var/cache/conftool/dbconfig/20221031-212717-ladsgroup.json [production]
21:12 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1188', diff saved to https://phabricator.wikimedia.org/P37276 and previous config saved to /var/cache/conftool/dbconfig/20221031-211210-ladsgroup.json [production]
20:57 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1188', diff saved to https://phabricator.wikimedia.org/P37275 and previous config saved to /var/cache/conftool/dbconfig/20221031-205703-ladsgroup.json [production]
20:41 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1188 (T318605)', diff saved to https://phabricator.wikimedia.org/P37274 and previous config saved to /var/cache/conftool/dbconfig/20221031-204157-ladsgroup.json [production]
20:33 <otto@deploy1002> Synchronized wmf-config/InitialiseSettings.php: No-op sync of InitialiseSettings.php to declare stream rc0.mediawiki.page_change. This stream is disabled everywhere by default, and only enabled in beta for now. - T311129 (duration: 03m 42s) [production]
20:33 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1188 (T318605)', diff saved to https://phabricator.wikimedia.org/P37273 and previous config saved to /var/cache/conftool/dbconfig/20221031-203319-ladsgroup.json [production]
20:33 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1188.eqiad.wmnet with reason: Maintenance [production]
20:33 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1188.eqiad.wmnet with reason: Maintenance [production]
20:32 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1182 (T318605)', diff saved to https://phabricator.wikimedia.org/P37272 and previous config saved to /var/cache/conftool/dbconfig/20221031-203258-ladsgroup.json [production]
20:32 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mw-debug: apply [production]
20:32 <cjming> end of UTC late backport window [production]
20:32 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mw-debug: apply [production]
20:31 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mw-debug: apply [production]
20:31 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mw-debug: apply [production]
20:26 <cjming@deploy1002> Finished scap: Backport for [[gerrit:843521|cirrus: Correct comments in ProductionServices.php (T262630)]] (duration: 05m 57s) [production]
20:25 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mw-debug: apply [production]
20:25 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mw-debug: apply [production]
20:24 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mw-debug: apply [production]
20:24 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mw-debug: apply [production]
20:20 <cjming@deploy1002> cjming and ebernhardson: Backport for [[gerrit:843521|cirrus: Correct comments in ProductionServices.php (T262630)]] synced to the testservers: mwdebug1001.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2001.codfw.wmnet [production]
20:20 <cjming@deploy1002> Started scap: Backport for [[gerrit:843521|cirrus: Correct comments in ProductionServices.php (T262630)]] [production]
20:19 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mw-debug: apply [production]
20:18 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mw-debug: apply [production]
20:18 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mw-debug: apply [production]
20:17 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1182', diff saved to https://phabricator.wikimedia.org/P37271 and previous config saved to /var/cache/conftool/dbconfig/20221031-201751-ladsgroup.json [production]
20:17 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mw-debug: apply [production]
20:17 <cjming@deploy1002> Finished scap: Backport for [[gerrit:851109|Update sample rate for edit attempt stream to 1 for group 0. (T312016)]] (duration: 04m 17s) [production]
20:13 <cjming@deploy1002> cjming and cjming: Backport for [[gerrit:851109|Update sample rate for edit attempt stream to 1 for group 0. (T312016)]] synced to the testservers: mwdebug2001.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2002.codfw.wmnet [production]
20:12 <cjming@deploy1002> Started scap: Backport for [[gerrit:851109|Update sample rate for edit attempt stream to 1 for group 0. (T312016)]] [production]
20:12 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mw-debug: apply [production]
20:11 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mw-debug: apply [production]
20:11 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mw-debug: apply [production]
20:10 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mw-debug: apply [production]
20:09 <cjming@deploy1002> Finished scap: Backport for [[gerrit:844073|Disable wgParserEnableLegacyMediaDOM on itwiki (T314318)]] (duration: 05m 44s) [production]
20:03 <cjming@deploy1002> cjming and arlolra: Backport for [[gerrit:844073|Disable wgParserEnableLegacyMediaDOM on itwiki (T314318)]] synced to the testservers: mwdebug2001.codfw.wmnet, mwdebug2002.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug1002.eqiad.wmnet [production]
20:03 <cjming@deploy1002> Started scap: Backport for [[gerrit:844073|Disable wgParserEnableLegacyMediaDOM on itwiki (T314318)]] [production]
20:02 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1182', diff saved to https://phabricator.wikimedia.org/P37270 and previous config saved to /var/cache/conftool/dbconfig/20221031-200245-ladsgroup.json [production]
19:59 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mw-debug: apply [production]
19:59 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mw-debug: apply [production]
19:58 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mw-debug: apply [production]
19:57 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mw-debug: apply [production]