|
2022-06-30
ยง
|
| 14:25 |
<pt1979@cumin2002> |
START - Cookbook sre.dns.netbox |
[production] |
| 14:23 |
<pt1979@cumin2002> |
START - Cookbook sre.hosts.provision for host db2156.mgmt.codfw.wmnet with reboot policy FORCED |
[production] |
| 14:22 |
<pt1979@cumin2002> |
END (ERROR) - Cookbook sre.hosts.provision (exit_code=97) for host db2156.mgmt.codfw.wmnet with reboot policy FORCED |
[production] |
| 14:20 |
<pt1979@cumin2002> |
START - Cookbook sre.hosts.provision for host db2156.mgmt.codfw.wmnet with reboot policy FORCED |
[production] |
| 14:14 |
<pt1979@cumin2002> |
START - Cookbook sre.hosts.reimage for host db2160.codfw.wmnet with OS bullseye |
[production] |
| 14:10 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1175', diff saved to https://phabricator.wikimedia.org/P30664 and previous config saved to /var/cache/conftool/dbconfig/20220630-141014-ladsgroup.json |
[production] |
| 14:02 |
<pt1979@cumin2002> |
END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2154.codfw.wmnet with OS bullseye |
[production] |
| 13:55 |
<moritzm> |
installing firejail security updates on stretch |
[production] |
| 13:55 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1175 (T307525)', diff saved to https://phabricator.wikimedia.org/P30663 and previous config saved to /var/cache/conftool/dbconfig/20220630-135509-ladsgroup.json |
[production] |
| 13:52 |
<pt1979@cumin2002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2154.codfw.wmnet with reason: host reimage |
[production] |
| 13:48 |
<cmjohnson@cumin1001> |
END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host stat1009.eqiad.wmnet with OS bullseye |
[production] |
| 13:48 |
<pt1979@cumin2002> |
START - Cookbook sre.hosts.downtime for 2:00:00 on db2154.codfw.wmnet with reason: host reimage |
[production] |
| 13:47 |
<pt1979@cumin2002> |
START - Cookbook sre.hosts.reimage for host db2154.codfw.wmnet with OS bullseye |
[production] |
| 13:40 |
<Amir1> |
killed refreshLinkRecommendations.php on arzwiki (T299021) |
[production] |
| 13:39 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host poolcounter2003.codfw.wmnet |
[production] |
| 13:39 |
<urbanecm> |
[urbanecm@mwmaint1002 /srv/mediawiki/php]$ mwscript extensions/CentralAuth/maintenance/attachAccount.php --wiki=metawiki --userlist /home/urbanecm/users.txt # T311012, users.txt has `New user message` only |
[production] |
| 13:39 |
<urbanecm> |
run `CentralAuthUser::importLocalNames` for `New user message` (T311012) |
[production] |
| 13:38 |
<bmansurov@deploy1002> |
Finished deploy [airflow-dags/research@b3fe77c]: (no justification provided) (duration: 00m 08s) |
[production] |
| 13:38 |
<bmansurov@deploy1002> |
Started deploy [airflow-dags/research@b3fe77c]: (no justification provided) |
[production] |
| 13:38 |
<bmansurov@deploy1002> |
deploy aborted: (no justification provided) (duration: 00m 03s) |
[production] |
| 13:38 |
<bmansurov@deploy1002> |
Started deploy [airflow-dags/research@b3fe77c]: (no justification provided) |
[production] |
| 13:37 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db1175 (T307525)', diff saved to https://phabricator.wikimedia.org/P30662 and previous config saved to /var/cache/conftool/dbconfig/20220630-133743-ladsgroup.json |
[production] |
| 13:37 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1175.eqiad.wmnet with reason: Maintenance |
[production] |
| 13:37 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 6:00:00 on db1175.eqiad.wmnet with reason: Maintenance |
[production] |
| 13:36 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db1128 (T309311)', diff saved to https://phabricator.wikimedia.org/P30661 and previous config saved to /var/cache/conftool/dbconfig/20220630-133619-ladsgroup.json |
[production] |
| 13:36 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1128.eqiad.wmnet with reason: Maintenance |
[production] |
| 13:36 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 6:00:00 on db1128.eqiad.wmnet with reason: Maintenance |
[production] |
| 13:36 |
<jmm@cumin2002> |
START - Cookbook sre.hosts.reboot-single for host poolcounter2003.codfw.wmnet |
[production] |
| 13:35 |
<urbanecm> |
[urbanecm@mwmaint1002 /srv/mediawiki/php]$ mwscript extensions/CentralAuth/maintenance/attachAccount.php --wiki=rowiki --userlist /home/urbanecm/users.txt # T275935, users.txt has `MediaWiki message delivery` only |
[production] |
| 13:35 |
<urbanecm> |
run `CentralAuthUser::importLocalNames` for `MediaWiki message delivery` (T275935) |
[production] |
| 13:35 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host poolcounter2004.codfw.wmnet |
[production] |
| 13:34 |
<slyngshede@cumin1001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest1001.eqiad.wmnet |
[production] |
| 13:34 |
<cmjohnson@cumin1001> |
START - Cookbook sre.hosts.reimage for host stat1009.eqiad.wmnet with OS bullseye |
[production] |
| 13:34 |
<pt1979@cumin2002> |
END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2155.codfw.wmnet with OS bullseye |
[production] |
| 13:33 |
<cmjohnson@cumin1001> |
END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host stat1009.eqiad.wmnet with OS buster |
[production] |
| 13:32 |
<urbanecm> |
[urbanecm@mwmaint1002 /srv/mediawiki/php]$ mwscript extensions/CentralAuth/maintenance/attachAccount.php --wiki=rowiki --userlist /home/urbanecm/users.txt # T311012, users.txt has `New user message` only |
[production] |
| 13:31 |
<jmm@cumin2002> |
START - Cookbook sre.hosts.reboot-single for host poolcounter2004.codfw.wmnet |
[production] |
| 13:29 |
<slyngshede@cumin1001> |
START - Cookbook sre.hosts.reboot-single for host sretest1001.eqiad.wmnet |
[production] |
| 13:28 |
<cmjohnson@cumin1001> |
START - Cookbook sre.hosts.reimage for host stat1009.eqiad.wmnet with OS buster |
[production] |
| 13:26 |
<bmansurov@deploy1002> |
Finished deploy [airflow-dags/research@b3fe77c]: (no justification provided) (duration: 00m 08s) |
[production] |
| 13:26 |
<bmansurov@deploy1002> |
Started deploy [airflow-dags/research@b3fe77c]: (no justification provided) |
[production] |
| 13:25 |
<urbanecm@deploy1002> |
Synchronized php-1.39.0-wmf.18/skins/Vector/resources/skins.vector.styles/layouts/screen.less: a927e6fbf56f031c42737cd9710eb0531bab43e1: Fixes Content sub unreadable in Vector 22 (T311564) (duration: 03m 18s) |
[production] |
| 13:25 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/mwdebug: apply |
[production] |
| 13:25 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] START helmfile.d/services/mwdebug: apply |
[production] |
| 13:24 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply |
[production] |
| 13:24 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] START helmfile.d/services/mwdebug: apply |
[production] |
| 13:23 |
<pt1979@cumin2002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2155.codfw.wmnet with reason: host reimage |
[production] |
| 13:23 |
<bmansurov@deploy1002> |
Finished deploy [airflow-dags/research@b3fe77c]: (no justification provided) (duration: 00m 08s) |
[production] |
| 13:23 |
<bmansurov@deploy1002> |
Started deploy [airflow-dags/research@b3fe77c]: (no justification provided) |
[production] |
| 13:22 |
<bmansurov@deploy1002> |
Finished deploy [airflow-dags/research@b3fe77c]: (no justification provided) (duration: 00m 09s) |
[production] |