8551-8600 of 10000 results (72ms)
2022-04-12 §
02:28 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
02:28 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
02:03 <eileen> config revision changed from c4cab5b1 to c8fee00c [production]
02:03 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
02:03 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
02:03 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
02:03 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
02:02 <eileen> civicrm revision changed from a90a6709 to 7de7ddd4 [production]
01:54 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1133.eqiad.wmnet with reason: Maintenance [production]
01:54 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1133.eqiad.wmnet with reason: Maintenance [production]
01:04 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1139.eqiad.wmnet with reason: Maintenance [production]
01:04 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1139.eqiad.wmnet with reason: Maintenance [production]
01:04 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1106 (T298565)', diff saved to https://phabricator.wikimedia.org/P24478 and previous config saved to /var/cache/conftool/dbconfig/20220412-010438-ladsgroup.json [production]
00:49 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1106', diff saved to https://phabricator.wikimedia.org/P24477 and previous config saved to /var/cache/conftool/dbconfig/20220412-004933-ladsgroup.json [production]
00:34 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1106', diff saved to https://phabricator.wikimedia.org/P24476 and previous config saved to /var/cache/conftool/dbconfig/20220412-003428-ladsgroup.json [production]
00:19 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1106 (T298565)', diff saved to https://phabricator.wikimedia.org/P24475 and previous config saved to /var/cache/conftool/dbconfig/20220412-001923-ladsgroup.json [production]
2022-04-11 §
23:21 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1106 (T298565)', diff saved to https://phabricator.wikimedia.org/P24474 and previous config saved to /var/cache/conftool/dbconfig/20220411-232102-ladsgroup.json [production]
23:20 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on clouddb[1013,1017,1021].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance [production]
23:20 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 12:00:00 on clouddb[1013,1017,1021].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance [production]
23:20 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1106.eqiad.wmnet with reason: Maintenance [production]
23:20 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1106.eqiad.wmnet with reason: Maintenance [production]
23:20 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1169 (T298565)', diff saved to https://phabricator.wikimedia.org/P24473 and previous config saved to /var/cache/conftool/dbconfig/20220411-232045-ladsgroup.json [production]
23:05 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1169', diff saved to https://phabricator.wikimedia.org/P24472 and previous config saved to /var/cache/conftool/dbconfig/20220411-230540-ladsgroup.json [production]
22:50 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1169', diff saved to https://phabricator.wikimedia.org/P24471 and previous config saved to /var/cache/conftool/dbconfig/20220411-225035-ladsgroup.json [production]
22:35 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1169 (T298565)', diff saved to https://phabricator.wikimedia.org/P24470 and previous config saved to /var/cache/conftool/dbconfig/20220411-223530-ladsgroup.json [production]
21:44 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1169 (T298565)', diff saved to https://phabricator.wikimedia.org/P24469 and previous config saved to /var/cache/conftool/dbconfig/20220411-214408-ladsgroup.json [production]
21:44 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1169.eqiad.wmnet with reason: Maintenance [production]
21:44 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1169.eqiad.wmnet with reason: Maintenance [production]
21:44 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1164 (T298565)', diff saved to https://phabricator.wikimedia.org/P24468 and previous config saved to /var/cache/conftool/dbconfig/20220411-214400-ladsgroup.json [production]
21:28 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1164', diff saved to https://phabricator.wikimedia.org/P24467 and previous config saved to /var/cache/conftool/dbconfig/20220411-212855-ladsgroup.json [production]
21:13 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1164', diff saved to https://phabricator.wikimedia.org/P24466 and previous config saved to /var/cache/conftool/dbconfig/20220411-211350-ladsgroup.json [production]
21:02 <urbanecm> [urbanecm@mwmaint1002 ~]$ mwscript extensions/GlobalBlocking/maintenance/PopulateCentralId.php --wiki=metawiki # END, T305014 [production]
20:58 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1164 (T298565)', diff saved to https://phabricator.wikimedia.org/P24463 and previous config saved to /var/cache/conftool/dbconfig/20220411-205844-ladsgroup.json [production]
20:45 <urbanecm> [urbanecm@mwmaint1002 ~]$ mwscript extensions/GlobalBlocking/maintenance/PopulateCentralId.php --wiki=metawiki # START, T305014, running in a tmux under my account at mwmaint1002 [production]
20:24 <urbanecm@deploy1002> Synchronized wmf-config/InitialiseSettings.php: 17c8c17: Start writing to cuc_actor in guwwiki and shnwikivoyage (T233004) (duration: 00m 51s) [production]
20:24 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
20:24 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
20:24 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
20:24 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
20:19 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
20:19 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
20:19 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
20:19 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
20:15 <urbanecm@deploy1002> Synchronized wmf-config/CommonSettings.php: 8455fa0: Stop writing to $wmfUsingKubernetes (T45956) (duration: 00m 51s) [production]
20:11 <urbanecm@deploy1002> Synchronized wmf-config/: d4ff32f: Migrate $wmfUsingKubernetes to $wmgUsingKubernetes (T45956) (duration: 00m 53s) [production]
20:08 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
20:08 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
20:08 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
20:08 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
20:03 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1164 (T298565)', diff saved to https://phabricator.wikimedia.org/P24461 and previous config saved to /var/cache/conftool/dbconfig/20220411-200301-ladsgroup.json [production]