5551-5600 of 10000 results (96ms)
2023-07-26 ยง
17:25 <jforrester@deploy1002> Started scap: Backport for [[gerrit:941974|wgNoFollowDomainExceptions: Add wikifunctions.org (T275945)]] [production]
17:13 <jforrester@deploy1002> Finished scap: Backport for [[gerrit:941970|MWMultiVersion: Alert this code to wikifunctions.org existing (T275945)]] (duration: 08m 40s) [production]
17:12 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db2169:3316 (T342617)', diff saved to https://phabricator.wikimedia.org/P49730 and previous config saved to /var/cache/conftool/dbconfig/20230726-171244-ladsgroup.json [production]
17:12 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2169.codfw.wmnet with reason: Maintenance [production]
17:12 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2169.codfw.wmnet with reason: Maintenance [production]
17:12 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2158 (T342617)', diff saved to https://phabricator.wikimedia.org/P49729 and previous config saved to /var/cache/conftool/dbconfig/20230726-171223-ladsgroup.json [production]
17:06 <jforrester@deploy1002> jforrester: Backport for [[gerrit:941970|MWMultiVersion: Alert this code to wikifunctions.org existing (T275945)]] synced to the testservers mwdebug1001.eqiad.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2001.codfw.wmnet, mwdebug2002.codfw.wmnet, and mw-debug kubernetes deployment (accessible via k8s-experimental XWD option) [production]
17:04 <jforrester@deploy1002> Started scap: Backport for [[gerrit:941970|MWMultiVersion: Alert this code to wikifunctions.org existing (T275945)]] [production]
16:57 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2158', diff saved to https://phabricator.wikimedia.org/P49728 and previous config saved to /var/cache/conftool/dbconfig/20230726-165717-ladsgroup.json [production]
16:53 <jforrester@deploy1002> Finished scap: Backport for [[gerrit:941965|docroot: Add wikifunctions.org (T275945)]] (duration: 08m 05s) [production]
16:47 <jforrester@deploy1002> jforrester: Backport for [[gerrit:941965|docroot: Add wikifunctions.org (T275945)]] synced to the testservers mwdebug1002.eqiad.wmnet, mwdebug2001.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug2002.codfw.wmnet, and mw-debug kubernetes deployment (accessible via k8s-experimental XWD option) [production]
16:45 <jforrester@deploy1002> Started scap: Backport for [[gerrit:941965|docroot: Add wikifunctions.org (T275945)]] [production]
16:44 <fabfur> end reboot of lvs1018 (T335835) [production]
16:43 <fabfur@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host lvs1018.eqiad.wmnet [production]
16:42 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2158', diff saved to https://phabricator.wikimedia.org/P49727 and previous config saved to /var/cache/conftool/dbconfig/20230726-164211-ladsgroup.json [production]
16:40 <fabfur@cumin1001> START - Cookbook sre.hosts.reboot-single for host lvs1018.eqiad.wmnet [production]
16:27 <jforrester@deploy1002> Finished scap: Initial deploy of wikifunctionswiki in locked-down mode for T275945 (duration: 07m 49s) [production]
16:27 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2158 (T342617)', diff saved to https://phabricator.wikimedia.org/P49726 and previous config saved to /var/cache/conftool/dbconfig/20230726-162705-ladsgroup.json [production]
16:19 <jforrester@deploy1002> Started scap: Initial deploy of wikifunctionswiki in locked-down mode for T275945 [production]
16:18 <fabfur> begin reboot of lvs1018 (T335835) [production]
16:15 <jforrester@deploy1002> Finished scap: Backport for [[gerrit:771623|Add wikifunctions.org to prod wgLocalVirtualHosts (T275945)]] (duration: 09m 07s) [production]
16:08 <jforrester@deploy1002> jforrester: Backport for [[gerrit:771623|Add wikifunctions.org to prod wgLocalVirtualHosts (T275945)]] synced to the testservers mwdebug1002.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug2001.codfw.wmnet, mwdebug1001.eqiad.wmnet, and mw-debug kubernetes deployment (accessible via k8s-experimental XWD option) [production]
16:07 <fabfur> end reboot of lvs1017 (T335835) [production]
16:06 <jforrester@deploy1002> Started scap: Backport for [[gerrit:771623|Add wikifunctions.org to prod wgLocalVirtualHosts (T275945)]] [production]
16:03 <jnuche@deploy1002> Synchronized php: group1 wikis to 1.41.0-wmf.19 refs T340247 (duration: 06m 56s) [production]
15:56 <jnuche@deploy1002> rebuilt and synchronized wikiversions files: group1 wikis to 1.41.0-wmf.19 refs T340247 [production]
15:55 <fabfur@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host lvs1017.eqiad.wmnet [production]
15:52 <fabfur@cumin1001> START - Cookbook sre.hosts.reboot-single for host lvs1017.eqiad.wmnet [production]
15:47 <jforrester@deploy1002> Finished scap: Backport for [[gerrit:941882|Load RescoreFunctions from the ExtensionRegistry (T342744)]] (duration: 09m 39s) [production]
15:42 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db2158 (T342617)', diff saved to https://phabricator.wikimedia.org/P49725 and previous config saved to /var/cache/conftool/dbconfig/20230726-154245-ladsgroup.json [production]
15:42 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on db2187.codfw.wmnet with reason: Maintenance [production]
15:42 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 2 days, 0:00:00 on db2187.codfw.wmnet with reason: Maintenance [production]
15:42 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2158.codfw.wmnet with reason: Maintenance [production]
15:42 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2158.codfw.wmnet with reason: Maintenance [production]
15:42 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2151 (T342617)', diff saved to https://phabricator.wikimedia.org/P49724 and previous config saved to /var/cache/conftool/dbconfig/20230726-154209-ladsgroup.json [production]
15:39 <jforrester@deploy1002> dcausse and jforrester: Backport for [[gerrit:941882|Load RescoreFunctions from the ExtensionRegistry (T342744)]] synced to the testservers mwdebug1001.eqiad.wmnet, mwdebug2001.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2002.codfw.wmnet, and mw-debug kubernetes deployment (accessible via k8s-experimental XWD option) [production]
15:38 <derick@deploy1002> helmfile [codfw] DONE helmfile.d/services/proton: apply [production]
15:37 <jforrester@deploy1002> Started scap: Backport for [[gerrit:941882|Load RescoreFunctions from the ExtensionRegistry (T342744)]] [production]
15:37 <derick@deploy1002> helmfile [codfw] START helmfile.d/services/proton: apply [production]
15:37 <derick@deploy1002> helmfile [eqiad] DONE helmfile.d/services/proton: apply [production]
15:35 <derick@deploy1002> helmfile [eqiad] START helmfile.d/services/proton: apply [production]
15:34 <derick@deploy1002> helmfile [staging] DONE helmfile.d/services/proton: apply [production]
15:34 <derick@deploy1002> helmfile [staging] START helmfile.d/services/proton: apply [production]
15:32 <derick@deploy1002> helmfile [staging] DONE helmfile.d/services/proton: apply [production]
15:32 <derick@deploy1002> helmfile [staging] START helmfile.d/services/proton: apply [production]
15:30 <fabfur> begin reboot of lvs1017 (T335835) [production]
15:27 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2151', diff saved to https://phabricator.wikimedia.org/P49723 and previous config saved to /var/cache/conftool/dbconfig/20230726-152703-ladsgroup.json [production]
15:26 <fabfur> end reboot of lvs1020 (T335835) [production]
15:25 <fabfur@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host lvs1020.eqiad.wmnet [production]
15:21 <fabfur@cumin1001> START - Cookbook sre.hosts.reboot-single for host lvs1020.eqiad.wmnet [production]