1301-1350 of 10000 results (32ms)
2022-03-08 ยง
14:42 <marostegui@cumin2002> dbctl commit (dc=all): 'Repooling after maintenance db1110', diff saved to https://phabricator.wikimedia.org/P22110 and previous config saved to /var/cache/conftool/dbconfig/20220308-144256-marostegui.json [production]
14:33 <urbanecm> UTC afternoon B&C window done [production]
14:32 <urbanecm@deploy1002> Synchronized php-1.38.0-wmf.24/extensions/DiscussionTools/includes/Notifications/DiscussionToolsEventTrait.php: 23939c7: Fix logic for finding the oldest comment in a bundle (T302014) (duration: 00m 50s) [production]
14:28 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
14:27 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
14:27 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
14:27 <marostegui@cumin2002> dbctl commit (dc=all): 'Repooling after maintenance db1110 (T298294)', diff saved to https://phabricator.wikimedia.org/P22109 and previous config saved to /var/cache/conftool/dbconfig/20220308-142721-marostegui.json [production]
14:26 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
14:24 <marostegui@cumin2002> dbctl commit (dc=all): 'Depooling db1110 (T298294)', diff saved to https://phabricator.wikimedia.org/P22108 and previous config saved to /var/cache/conftool/dbconfig/20220308-142412-marostegui.json [production]
14:23 <marostegui@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db1110.eqiad.wmnet with reason: Maintenance [production]
14:23 <marostegui@cumin2002> START - Cookbook sre.hosts.downtime for 8:00:00 on db1110.eqiad.wmnet with reason: Maintenance [production]
14:23 <marostegui@cumin2002> dbctl commit (dc=all): 'Repooling after maintenance db1144:3315 (T298294)', diff saved to https://phabricator.wikimedia.org/P22107 and previous config saved to /var/cache/conftool/dbconfig/20220308-142332-marostegui.json [production]
14:16 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
14:15 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
14:15 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
14:14 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
14:09 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
14:08 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
14:08 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
14:07 <marostegui@cumin2002> dbctl commit (dc=all): 'Repooling after maintenance db1144:3315', diff saved to https://phabricator.wikimedia.org/P22104 and previous config saved to /var/cache/conftool/dbconfig/20220308-140758-marostegui.json [production]
14:06 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
14:06 <dcaro@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cloudcontrol1003.wikimedia.org [production]
14:06 <urbanecm@deploy1002> Synchronized wmf-config/InitialiseSettings.php: 75465dd0288b998ee6d4668e87e57b0d7961471a: fawiki: Add patrolmarks right to autopatrolled group (T303269) (duration: 00m 49s) [production]
13:56 <aqu@deploy1002> Finished deploy [airflow-dags/analytics@d1c8ae0]: Fix wikidata_item_page_link destination table after tests (duration: 00m 07s) [production]
13:56 <aqu@deploy1002> Started deploy [airflow-dags/analytics@d1c8ae0]: Fix wikidata_item_page_link destination table after tests [production]
13:52 <marostegui@cumin2002> dbctl commit (dc=all): 'Repooling after maintenance db1144:3315', diff saved to https://phabricator.wikimedia.org/P22103 and previous config saved to /var/cache/conftool/dbconfig/20220308-135223-marostegui.json [production]
13:48 <elukey@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ml-serve1004.eqiad.wmnet [production]
13:46 <dcaro@cumin2002> START - Cookbook sre.hosts.reboot-single for host cloudcontrol1003.wikimedia.org [production]
13:40 <aqu@deploy1002> Finished deploy [airflow-dags/analytics@725f528]: Set wikidata/item_page_link/weekly start date in production (duration: 00m 07s) [production]
13:40 <aqu@deploy1002> Started deploy [airflow-dags/analytics@725f528]: Set wikidata/item_page_link/weekly start date in production [production]
13:40 <elukey@cumin1001> START - Cookbook sre.hosts.reboot-single for host ml-serve1004.eqiad.wmnet [production]
13:39 <elukey@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ml-serve1003.eqiad.wmnet [production]
13:36 <marostegui@cumin2002> dbctl commit (dc=all): 'Repooling after maintenance db1144:3315 (T298294)', diff saved to https://phabricator.wikimedia.org/P22102 and previous config saved to /var/cache/conftool/dbconfig/20220308-133647-marostegui.json [production]
13:34 <btullis@cumin2002> END (PASS) - Cookbook sre.druid.roll-restart-workers (exit_code=0) for Druid analytics cluster: Roll restart of Druid jvm daemons. [production]
13:33 <marostegui@cumin2002> dbctl commit (dc=all): 'Depooling db1144:3315 (T298294)', diff saved to https://phabricator.wikimedia.org/P22101 and previous config saved to /var/cache/conftool/dbconfig/20220308-133335-marostegui.json [production]
13:33 <marostegui@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db1144.eqiad.wmnet with reason: Maintenance [production]
13:33 <marostegui@cumin2002> START - Cookbook sre.hosts.downtime for 8:00:00 on db1144.eqiad.wmnet with reason: Maintenance [production]
13:32 <marostegui@cumin2002> dbctl commit (dc=all): 'Repooling after maintenance db1096:3315 (T298294)', diff saved to https://phabricator.wikimedia.org/P22100 and previous config saved to /var/cache/conftool/dbconfig/20220308-133255-marostegui.json [production]
13:31 <elukey@cumin1001> START - Cookbook sre.hosts.reboot-single for host ml-serve1003.eqiad.wmnet [production]
13:26 <elukey@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ml-serve1002.eqiad.wmnet [production]
13:17 <aborrero@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cloudcontrol1004.wikimedia.org [production]
13:17 <elukey@cumin1001> START - Cookbook sre.hosts.reboot-single for host ml-serve1002.eqiad.wmnet [production]
13:17 <marostegui@cumin2002> dbctl commit (dc=all): 'Repooling after maintenance db1096:3315', diff saved to https://phabricator.wikimedia.org/P22099 and previous config saved to /var/cache/conftool/dbconfig/20220308-131720-marostegui.json [production]
13:16 <elukey@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ml-serve1001.eqiad.wmnet [production]
13:14 <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db1181 (T300775)', diff saved to https://phabricator.wikimedia.org/P22098 and previous config saved to /var/cache/conftool/dbconfig/20220308-131420-marostegui.json [production]
13:14 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on db1181.eqiad.wmnet with reason: Maintenance [production]
13:14 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 2 days, 0:00:00 on db1181.eqiad.wmnet with reason: Maintenance [production]
13:13 <marostegui@cumin1001> dbctl commit (dc=all): 'db1174 (re)pooling @ 100%: After schema change', diff saved to https://phabricator.wikimedia.org/P22097 and previous config saved to /var/cache/conftool/dbconfig/20220308-131309-root.json [production]
13:09 <elukey@cumin1001> START - Cookbook sre.hosts.reboot-single for host ml-serve1001.eqiad.wmnet [production]
13:07 <aborrero@cumin2002> START - Cookbook sre.hosts.reboot-single for host cloudcontrol1004.wikimedia.org [production]