401-450 of 10000 results (65ms)
2022-11-15 ยง
23:10 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2130', diff saved to https://phabricator.wikimedia.org/P39854 and previous config saved to /var/cache/conftool/dbconfig/20221115-231043-ladsgroup.json [production]
23:10 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1107', diff saved to https://phabricator.wikimedia.org/P39853 and previous config saved to /var/cache/conftool/dbconfig/20221115-231025-ladsgroup.json [production]
23:02 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2109', diff saved to https://phabricator.wikimedia.org/P39852 and previous config saved to /var/cache/conftool/dbconfig/20221115-230240-marostegui.json [production]
22:55 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2130 (T318605)', diff saved to https://phabricator.wikimedia.org/P39851 and previous config saved to /var/cache/conftool/dbconfig/20221115-225537-ladsgroup.json [production]
22:55 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1107', diff saved to https://phabricator.wikimedia.org/P39850 and previous config saved to /var/cache/conftool/dbconfig/20221115-225518-ladsgroup.json [production]
22:52 <ejegg> re-enabled civicrm dedupe jobs [production]
22:51 <mutante> phab1004 - running public_task_dump.py [production]
22:51 <ejegg> civicrm upgraded from d85589e8 to fa71f219 [production]
22:47 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2109 (T321130)', diff saved to https://phabricator.wikimedia.org/P39849 and previous config saved to /var/cache/conftool/dbconfig/20221115-224733-marostegui.json [production]
22:40 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1107 (T318605)', diff saved to https://phabricator.wikimedia.org/P39848 and previous config saved to /var/cache/conftool/dbconfig/20221115-224011-ladsgroup.json [production]
22:12 <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db2109 (T321130)', diff saved to https://phabricator.wikimedia.org/P39847 and previous config saved to /var/cache/conftool/dbconfig/20221115-221247-marostegui.json [production]
22:12 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2109.codfw.wmnet with reason: Maintenance [production]
22:12 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 5:00:00 on db2109.codfw.wmnet with reason: Maintenance [production]
22:12 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2105 (T321130)', diff saved to https://phabricator.wikimedia.org/P39846 and previous config saved to /var/cache/conftool/dbconfig/20221115-221225-marostegui.json [production]
21:57 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2105', diff saved to https://phabricator.wikimedia.org/P39845 and previous config saved to /var/cache/conftool/dbconfig/20221115-215719-marostegui.json [production]
21:42 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2105', diff saved to https://phabricator.wikimedia.org/P39844 and previous config saved to /var/cache/conftool/dbconfig/20221115-214212-marostegui.json [production]
21:40 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2175 (T321126)', diff saved to https://phabricator.wikimedia.org/P39843 and previous config saved to /var/cache/conftool/dbconfig/20221115-214022-marostegui.json [production]
21:33 <cjming> end of UTC late backport window [production]
21:27 <cmooney@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
21:27 <cjming@deploy1002> Finished scap: Backport for [[gerrit:857040|tnwiki: Set timezone to Africa/Gaborone (UTC+2) (T318208)]] (duration: 06m 14s) [production]
21:27 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2105 (T321130)', diff saved to https://phabricator.wikimedia.org/P39842 and previous config saved to /var/cache/conftool/dbconfig/20221115-212706-marostegui.json [production]
21:25 <cmooney@cumin1001> START - Cookbook sre.dns.netbox [production]
21:25 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2175', diff saved to https://phabricator.wikimedia.org/P39841 and previous config saved to /var/cache/conftool/dbconfig/20221115-212516-marostegui.json [production]
21:21 <cjming@deploy1002> cjming and stang: Backport for [[gerrit:857040|tnwiki: Set timezone to Africa/Gaborone (UTC+2) (T318208)]] synced to the testservers: mwdebug2002.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2001.codfw.wmnet, mwdebug1001.eqiad.wmnet [production]
21:21 <cjming@deploy1002> Started scap: Backport for [[gerrit:857040|tnwiki: Set timezone to Africa/Gaborone (UTC+2) (T318208)]] [production]
21:19 <cjming@deploy1002> Finished scap: Backport for [[gerrit:857030|logos: Remove duplicated code (T307705)]] (duration: 04m 31s) [production]
21:15 <cjming@deploy1002> cjming and stang: Backport for [[gerrit:857030|logos: Remove duplicated code (T307705)]] synced to the testservers: mwdebug1001.eqiad.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2001.codfw.wmnet, mwdebug2002.codfw.wmnet [production]
21:14 <cjming@deploy1002> Started scap: Backport for [[gerrit:857030|logos: Remove duplicated code (T307705)]] [production]
21:13 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db2130 (T318605)', diff saved to https://phabricator.wikimedia.org/P39840 and previous config saved to /var/cache/conftool/dbconfig/20221115-211314-ladsgroup.json [production]
21:13 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2130.codfw.wmnet with reason: Maintenance [production]
21:12 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2130.codfw.wmnet with reason: Maintenance [production]
21:12 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2116 (T318605)', diff saved to https://phabricator.wikimedia.org/P39839 and previous config saved to /var/cache/conftool/dbconfig/20221115-211253-ladsgroup.json [production]
21:10 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2175', diff saved to https://phabricator.wikimedia.org/P39838 and previous config saved to /var/cache/conftool/dbconfig/20221115-211009-marostegui.json [production]
21:09 <jhathaway@deploy1002> helmfile [aux-k8s-eqiad] DONE helmfile.d/admin 'sync'. [production]
21:09 <jhathaway@deploy1002> helmfile [aux-k8s-eqiad] START helmfile.d/admin 'sync'. [production]
21:08 <cjming@deploy1002> Finished scap: Backport for [[gerrit:854570|EditAttemptStep sampling rate to 1 everywhere (T312016)]] (duration: 04m 45s) [production]
21:07 <jhathaway@deploy1002> helmfile [aux-k8s-eqiad] DONE helmfile.d/admin 'sync'. [production]
21:07 <jhathaway@deploy1002> helmfile [aux-k8s-eqiad] START helmfile.d/admin 'sync'. [production]
21:04 <cjming@deploy1002> cjming and phuedx: Backport for [[gerrit:854570|EditAttemptStep sampling rate to 1 everywhere (T312016)]] synced to the testservers: mwdebug1002.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug2001.codfw.wmnet, mwdebug1001.eqiad.wmnet [production]
21:03 <cjming@deploy1002> Started scap: Backport for [[gerrit:854570|EditAttemptStep sampling rate to 1 everywhere (T312016)]] [production]
21:02 <jhathaway@deploy1002> helmfile [aux-k8s-eqiad] DONE helmfile.d/admin 'sync'. [production]
21:01 <jhathaway@deploy1002> helmfile [aux-k8s-eqiad] START helmfile.d/admin 'sync'. [production]
20:59 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1107 (T318605)', diff saved to https://phabricator.wikimedia.org/P39837 and previous config saved to /var/cache/conftool/dbconfig/20221115-205929-ladsgroup.json [production]
20:59 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1107.eqiad.wmnet with reason: Maintenance [production]
20:59 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1107.eqiad.wmnet with reason: Maintenance [production]
20:59 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1106 (T318605)', diff saved to https://phabricator.wikimedia.org/P39836 and previous config saved to /var/cache/conftool/dbconfig/20221115-205907-ladsgroup.json [production]
20:57 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2116', diff saved to https://phabricator.wikimedia.org/P39835 and previous config saved to /var/cache/conftool/dbconfig/20221115-205746-ladsgroup.json [production]
20:55 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2175 (T321126)', diff saved to https://phabricator.wikimedia.org/P39834 and previous config saved to /var/cache/conftool/dbconfig/20221115-205503-marostegui.json [production]
20:52 <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db2175 (T321126)', diff saved to https://phabricator.wikimedia.org/P39833 and previous config saved to /var/cache/conftool/dbconfig/20221115-205222-marostegui.json [production]
20:52 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2175.codfw.wmnet with reason: Maintenance [production]