3651-3700 of 10000 results (72ms)
2022-04-26 ยง
22:25 <tgr@deploy1002> Started scap: backport with i18n changes: [[gerrit:785944]], [[gerrit:785941]] [production]
22:17 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1135', diff saved to https://phabricator.wikimedia.org/P26631 and previous config saved to /var/cache/conftool/dbconfig/20220426-221739-ladsgroup.json [production]
22:02 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1135 (T306560)', diff saved to https://phabricator.wikimedia.org/P26630 and previous config saved to /var/cache/conftool/dbconfig/20220426-220234-ladsgroup.json [production]
22:00 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1135 (T306560)', diff saved to https://phabricator.wikimedia.org/P26629 and previous config saved to /var/cache/conftool/dbconfig/20220426-220001-ladsgroup.json [production]
22:00 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1135.eqiad.wmnet with reason: Maintenance [production]
21:59 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1135.eqiad.wmnet with reason: Maintenance [production]
21:59 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1134 (T306560)', diff saved to https://phabricator.wikimedia.org/P26628 and previous config saved to /var/cache/conftool/dbconfig/20220426-215953-ladsgroup.json [production]
21:44 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1134', diff saved to https://phabricator.wikimedia.org/P26627 and previous config saved to /var/cache/conftool/dbconfig/20220426-214448-ladsgroup.json [production]
21:38 <aqu@deploy1002> Finished deploy [airflow-dags/analytics@e5fecc9]: Fix typo in mediarequest/hourly sensor [airflow-dags/analytics@e5fecc9] (duration: 00m 07s) [production]
21:37 <aqu@deploy1002> Started deploy [airflow-dags/analytics@e5fecc9]: Fix typo in mediarequest/hourly sensor [airflow-dags/analytics@e5fecc9] [production]
21:29 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1134', diff saved to https://phabricator.wikimedia.org/P26626 and previous config saved to /var/cache/conftool/dbconfig/20220426-212943-ladsgroup.json [production]
21:14 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1134 (T306560)', diff saved to https://phabricator.wikimedia.org/P26625 and previous config saved to /var/cache/conftool/dbconfig/20220426-211437-ladsgroup.json [production]
21:12 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1134 (T306560)', diff saved to https://phabricator.wikimedia.org/P26624 and previous config saved to /var/cache/conftool/dbconfig/20220426-211204-ladsgroup.json [production]
21:12 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1134.eqiad.wmnet with reason: Maintenance [production]
21:12 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1134.eqiad.wmnet with reason: Maintenance [production]
21:11 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1184 (T306560)', diff saved to https://phabricator.wikimedia.org/P26623 and previous config saved to /var/cache/conftool/dbconfig/20220426-211156-ladsgroup.json [production]
21:05 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
21:05 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
21:05 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
21:05 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
21:03 <dzahn@cumin2002> conftool action : set/weight=30; selector: dc=codfw,name=mw2418.codfw.wmnet [production]
21:03 <dzahn@cumin2002> conftool action : set/weight=30; selector: dc=codfw,name=mw2417.codfw.wmnet [production]
21:02 <dzahn@cumin2002> conftool action : set/weight=30; selector: dc=codfw,name=mw2416.codfw.wmnet [production]
21:01 <urbanecm@deploy1002> Synchronized wmf-config/CommonSettings.php: cab00628da0ba6226ff162cfc848bea35a35783a: fix wmgVectorMaxWidthOptionsNamespaces (T300182) (duration: 01m 00s) [production]
21:00 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
21:00 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
21:00 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
21:00 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
21:00 <mutante> mw2416, mw2417, mw2418 - scap pull [production]
20:56 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1184', diff saved to https://phabricator.wikimedia.org/P26621 and previous config saved to /var/cache/conftool/dbconfig/20220426-205651-ladsgroup.json [production]
20:50 <aqu@deploy1002> Finished deploy [airflow-dags/analytics@e177d87]: Bump jar dependency to 0.1.27 in mediarequest/hourly [airflow-dags/analytics@e177d87] (duration: 00m 07s) [production]
20:50 <aqu@deploy1002> Started deploy [airflow-dags/analytics@e177d87]: Bump jar dependency to 0.1.27 in mediarequest/hourly [airflow-dags/analytics@e177d87] [production]
20:49 <urbanecm@deploy1002> Synchronized wmf-config/SearchSettingsForWikidata.php: f76bc806157a3f4c88d44cd467de347b4b471f4e: Correct wbsearchentities profiles (duration: 00m 57s) [production]
20:41 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1184', diff saved to https://phabricator.wikimedia.org/P26620 and previous config saved to /var/cache/conftool/dbconfig/20220426-204146-ladsgroup.json [production]
20:40 <dzahn@cumin2002> conftool action : set/weight=30; selector: dc=codfw,name=mw2415.codfw.wmnet [production]
20:40 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
20:39 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
20:39 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
20:39 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
20:39 <dzahn@cumin2002> conftool action : set/weight=30; selector: dc=codfw,name=mw2414.codfw.wmnet [production]
20:39 <dzahn@cumin2002> conftool action : set/weight=30; selector: dc=codfw,name=mw2413.codfw.wmnet [production]
20:38 <dzahn@cumin2002> conftool action : set/weight=30; selector: dc=codfw,name=mw2412.codfw.wmnet [production]
20:37 <urbanecm@deploy1002> Synchronized php-1.39.0-wmf.9/skins/Vector/resources/skins.vector.styles/: 019a812176bb940383ddeb22f8a74b5d0f447bf1: [ToC] Increase threshold for ToC collapsing to 1000px (T306904) (duration: 00m 50s) [production]
20:36 <urbanecm@deploy1002> Synchronized php-1.39.0-wmf.8/skins/Vector/resources/skins.vector.styles/: 31ed884d6eda998f8625a88be0f4aa5fd67aef4b: [ToC] Increase threshold for ToC collapsing to 1000px (T306904) (duration: 00m 50s) [production]
20:34 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
20:34 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
20:34 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
20:34 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
20:33 <mutante> mw2412, mw2413, mw2414, mw2415 - scap pull, get into production the first time [production]
20:29 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]