2151-2200 of 10000 results (85ms)
2023-05-08 ยง
21:37 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1187', diff saved to https://phabricator.wikimedia.org/P47947 and previous config saved to /var/cache/conftool/dbconfig/20230508-213754-ladsgroup.json [production]
21:24 <mstyles@deploy1002> Finished scap: Backport for [[gerrit:916913|Disable translation memory on collabwiki (T313241)]] (duration: 06m 45s) [production]
21:22 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1187', diff saved to https://phabricator.wikimedia.org/P47946 and previous config saved to /var/cache/conftool/dbconfig/20230508-212248-ladsgroup.json [production]
21:21 <mforns@deploy1002> Finished deploy [airflow-dags/analytics@a6a3ceb]: (no justification provided) (duration: 00m 09s) [production]
21:21 <mforns@deploy1002> Started deploy [airflow-dags/analytics@a6a3ceb]: (no justification provided) [production]
21:18 <mstyles@deploy1002> mstyles and sbassett: Backport for [[gerrit:916913|Disable translation memory on collabwiki (T313241)]] synced to the testservers: mwdebug2002.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2001.codfw.wmnet, mwdebug1001.eqiad.wmnet [production]
21:17 <mstyles@deploy1002> Started scap: Backport for [[gerrit:916913|Disable translation memory on collabwiki (T313241)]] [production]
21:09 <herron@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host kafka-logging1001.eqiad.wmnet [production]
21:07 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1187 (T335845)', diff saved to https://phabricator.wikimedia.org/P47945 and previous config saved to /var/cache/conftool/dbconfig/20230508-210742-ladsgroup.json [production]
21:02 <herron@cumin1001> START - Cookbook sre.hosts.reboot-single for host kafka-logging1001.eqiad.wmnet [production]
21:01 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1187 (T335845)', diff saved to https://phabricator.wikimedia.org/P47944 and previous config saved to /var/cache/conftool/dbconfig/20230508-210119-ladsgroup.json [production]
21:01 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1187.eqiad.wmnet with reason: Maintenance [production]
21:01 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1187.eqiad.wmnet with reason: Maintenance [production]
21:00 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1180 (T335845)', diff saved to https://phabricator.wikimedia.org/P47943 and previous config saved to /var/cache/conftool/dbconfig/20230508-210056-ladsgroup.json [production]
20:59 <herron@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host kafka-logging1002.eqiad.wmnet [production]
20:53 <herron@cumin1001> START - Cookbook sre.hosts.reboot-single for host kafka-logging1002.eqiad.wmnet [production]
20:49 <herron@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host kafka-logging1004.eqiad.wmnet [production]
20:45 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1180', diff saved to https://phabricator.wikimedia.org/P47942 and previous config saved to /var/cache/conftool/dbconfig/20230508-204549-ladsgroup.json [production]
20:43 <mutante> miscweb2003 - rebooting [production]
20:41 <herron@cumin1001> START - Cookbook sre.hosts.reboot-single for host kafka-logging1004.eqiad.wmnet [production]
20:41 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 0:15:00 on miscweb2003.codfw.wmnet with reason: reboot [production]
20:41 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime for 0:15:00 on miscweb2003.codfw.wmnet with reason: reboot [production]
20:39 <herron@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host kafka-logging1003.eqiad.wmnet [production]
20:38 <taavi@deploy1002> Finished scap: Backport for [[gerrit:917395|Deploy fixed width indicator to wikis (T335307)]] (duration: 08m 22s) [production]
20:36 <aokoth@cumin1001> END (PASS) - Cookbook sre.ganeti.reimage (exit_code=0) for host vrts1001.eqiad.wmnet with OS bullseye [production]
20:33 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 0:15:00 on miscweb2003.codfw.wmnet with reason: reboot [production]
20:32 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime for 0:15:00 on miscweb2003.codfw.wmnet with reason: reboot [production]
20:32 <herron@cumin1001> START - Cookbook sre.hosts.reboot-single for host kafka-logging1003.eqiad.wmnet [production]
20:31 <taavi@deploy1002> jdlrobson and taavi: Backport for [[gerrit:917395|Deploy fixed width indicator to wikis (T335307)]] synced to the testservers: mwdebug2001.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2002.codfw.wmnet [production]
20:30 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1180', diff saved to https://phabricator.wikimedia.org/P47941 and previous config saved to /var/cache/conftool/dbconfig/20230508-203043-ladsgroup.json [production]
20:30 <taavi@deploy1002> Started scap: Backport for [[gerrit:917395|Deploy fixed width indicator to wikis (T335307)]] [production]
20:29 <taavi@deploy1002> Finished scap: Backport for [[gerrit:917162|Ensure page load popupNotification is closed when the toggle button is clicked (T335153)]] (duration: 07m 58s) [production]
20:25 <bking@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host wdqs2007.codfw.wmnet [production]
20:24 <aokoth@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on vrts1001.eqiad.wmnet with reason: host reimage [production]
20:23 <taavi@deploy1002> jdlrobson and taavi: Backport for [[gerrit:917162|Ensure page load popupNotification is closed when the toggle button is clicked (T335153)]] synced to the testservers: mwdebug1001.eqiad.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2001.codfw.wmnet, mwdebug2002.codfw.wmnet [production]
20:21 <taavi@deploy1002> Started scap: Backport for [[gerrit:917162|Ensure page load popupNotification is closed when the toggle button is clicked (T335153)]] [production]
20:21 <aokoth@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on vrts1001.eqiad.wmnet with reason: host reimage [production]
20:20 <taavi@deploy1002> Finished scap: Backport for [[gerrit:917160|Update a/b test code for visual enhancements a/b test (T333715)]], [[gerrit:916903|Enable DiscussionTools visual enhancements a/b test (T302358)]] (duration: 11m 54s) [production]
20:15 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1180 (T335845)', diff saved to https://phabricator.wikimedia.org/P47939 and previous config saved to /var/cache/conftool/dbconfig/20230508-201537-ladsgroup.json [production]
20:11 <aokoth@cumin1001> START - Cookbook sre.ganeti.reimage for host vrts1001.eqiad.wmnet with OS bullseye [production]
20:09 <taavi@deploy1002> kemayo and taavi: Backport for [[gerrit:917160|Update a/b test code for visual enhancements a/b test (T333715)]], [[gerrit:916903|Enable DiscussionTools visual enhancements a/b test (T302358)]] synced to the testservers: mwdebug2002.codfw.wmnet, mwdebug2001.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug1002.eqiad.wmnet [production]
20:08 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1180 (T335845)', diff saved to https://phabricator.wikimedia.org/P47938 and previous config saved to /var/cache/conftool/dbconfig/20230508-200825-ladsgroup.json [production]
20:08 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1180.eqiad.wmnet with reason: Maintenance [production]
20:08 <taavi@deploy1002> Started scap: Backport for [[gerrit:917160|Update a/b test code for visual enhancements a/b test (T333715)]], [[gerrit:916903|Enable DiscussionTools visual enhancements a/b test (T302358)]] [production]
20:08 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1180.eqiad.wmnet with reason: Maintenance [production]
20:08 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1173 (T335845)', diff saved to https://phabricator.wikimedia.org/P47937 and previous config saved to /var/cache/conftool/dbconfig/20230508-200802-ladsgroup.json [production]
20:05 <aokoth@cumin1001> END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) for new host vrts1001.eqiad.wmnet [production]
20:05 <aokoth@cumin1001> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.ganeti.makevm: created new VM vrts1001.eqiad.wmnet - aokoth@cumin1001" [production]
20:04 <aokoth@cumin1001> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.ganeti.makevm: created new VM vrts1001.eqiad.wmnet - aokoth@cumin1001" [production]
20:04 <aokoth@cumin1001> END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) vrts1001.eqiad.wmnet on all recursors [production]