|
2024-01-04
ยง
|
| 14:45 |
<tchanders@deploy2002> |
pfischer and tchanders: Backport for [[gerrit:984810|enable page_rerender for 2nd batch: dewiki, frwiktionary, and kuwiktionary]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) |
[production] |
| 14:42 |
<tchanders@deploy2002> |
Started scap: Backport for [[gerrit:984810|enable page_rerender for 2nd batch: dewiki, frwiktionary, and kuwiktionary]] |
[production] |
| 14:40 |
<tchanders@deploy2002> |
Finished scap: Backport for [[gerrit:987726|Attempt to send original file to PhotoDNA if no thumbnail (T353854)]] (duration: 09m 25s) |
[production] |
| 14:34 |
<tchanders@deploy2002> |
tchanders and dreamyjazz: Continuing with sync |
[production] |
| 14:34 |
<tchanders@deploy2002> |
tchanders and dreamyjazz: Backport for [[gerrit:987726|Attempt to send original file to PhotoDNA if no thumbnail (T353854)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) |
[production] |
| 14:30 |
<tchanders@deploy2002> |
Started scap: Backport for [[gerrit:987726|Attempt to send original file to PhotoDNA if no thumbnail (T353854)]] |
[production] |
| 14:25 |
<tchanders@deploy2002> |
Finished scap: Backport for [[gerrit:987485|Attempt to send original file to PhotoDNA if no thumbnail (T353854)]] (duration: 09m 24s) |
[production] |
| 14:20 |
<tchanders@deploy2002> |
dreamyjazz and tchanders: Continuing with sync |
[production] |
| 14:20 |
<tchanders@deploy2002> |
dreamyjazz and tchanders: Backport for [[gerrit:987485|Attempt to send original file to PhotoDNA if no thumbnail (T353854)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) |
[production] |
| 14:16 |
<tchanders@deploy2002> |
Started scap: Backport for [[gerrit:987485|Attempt to send original file to PhotoDNA if no thumbnail (T353854)]] |
[production] |
| 14:12 |
<pfischer@deploy2002> |
helmfile [staging] DONE helmfile.d/services/cirrus-streaming-updater: apply |
[production] |
| 14:12 |
<pfischer@deploy2002> |
helmfile [staging] START helmfile.d/services/cirrus-streaming-updater: apply |
[production] |
| 14:09 |
<pfischer@deploy2002> |
helmfile [staging] DONE helmfile.d/services/cirrus-streaming-updater: apply |
[production] |
| 14:09 |
<pfischer@deploy2002> |
helmfile [staging] START helmfile.d/services/cirrus-streaming-updater: apply |
[production] |
| 14:09 |
<pfischer@deploy2002> |
helmfile [staging] DONE helmfile.d/services/cirrus-streaming-updater: apply |
[production] |
| 14:08 |
<pfischer@deploy2002> |
helmfile [staging] START helmfile.d/services/cirrus-streaming-updater: apply |
[production] |
| 14:08 |
<pfischer@deploy2002> |
helmfile [staging] DONE helmfile.d/services/cirrus-streaming-updater: apply |
[production] |
| 14:08 |
<pfischer@deploy2002> |
helmfile [staging] START helmfile.d/services/cirrus-streaming-updater: apply |
[production] |
| 14:06 |
<pfischer@deploy2002> |
helmfile [staging] DONE helmfile.d/services/cirrus-streaming-updater: apply |
[production] |
| 14:03 |
<XioNoX> |
repool drmrs - T354340 |
[production] |
| 14:01 |
<pfischer@deploy2002> |
helmfile [staging] START helmfile.d/services/cirrus-streaming-updater: apply |
[production] |
| 14:00 |
<pfischer@deploy2002> |
helmfile [staging] DONE helmfile.d/services/cirrus-streaming-updater: apply |
[production] |
| 14:00 |
<pfischer@deploy2002> |
helmfile [staging] START helmfile.d/services/cirrus-streaming-updater: apply |
[production] |
| 13:57 |
<ayounsi@cumin1002> |
END (PASS) - Cookbook sre.network.peering (exit_code=0) with action 'configure' for AS: 2686 |
[production] |
| 13:56 |
<ayounsi@cumin1002> |
START - Cookbook sre.network.peering with action 'configure' for AS: 2686 |
[production] |
| 13:53 |
<moritzm> |
installing libssh security updates |
[production] |
| 13:24 |
<dcausse> |
restarting blazegraph on wdqs1019 (stuck with high thread count) |
[production] |
| 13:06 |
<zabe@deploy2002> |
Finished scap: Backport for [[gerrit:987483|Revert "Get blocks from DatabaseBlockStore instead of doing our own query" (T353620)]], [[gerrit:987482|Revert "Support new block schema" (T354298)]] (duration: 10m 06s) |
[production] |
| 13:02 |
<kamila@cumin1002> |
END (FAIL) - Cookbook sre.hosts.reboot-single (exit_code=99) for host mw1377.eqiad.wmnet |
[production] |
| 13:02 |
<XioNoX> |
depool drmrs for router work - T354340 |
[production] |
| 13:01 |
<zabe@deploy2002> |
zabe: Continuing with sync |
[production] |
| 13:00 |
<zabe@deploy2002> |
zabe: Backport for [[gerrit:987483|Revert "Get blocks from DatabaseBlockStore instead of doing our own query" (T353620)]], [[gerrit:987482|Revert "Support new block schema" (T354298)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) |
[production] |
| 12:56 |
<zabe@deploy2002> |
Started scap: Backport for [[gerrit:987483|Revert "Get blocks from DatabaseBlockStore instead of doing our own query" (T353620)]], [[gerrit:987482|Revert "Support new block schema" (T354298)]] |
[production] |
| 12:53 |
<ayounsi@cumin1002> |
END (PASS) - Cookbook sre.network.peering (exit_code=0) with action 'configure' for AS: 63296 |
[production] |
| 12:52 |
<ayounsi@cumin1002> |
START - Cookbook sre.network.peering with action 'configure' for AS: 63296 |
[production] |
| 12:10 |
<kamila@cumin1002> |
START - Cookbook sre.hosts.reboot-single for host mw1377.eqiad.wmnet |
[production] |
| 12:04 |
<moritzm> |
installing lua5.3 security updates |
[production] |
| 11:52 |
<moritzm> |
installing libde265 security updates |
[production] |
| 11:35 |
<kamila@cumin1002> |
END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host mw1379.eqiad.wmnet with OS bullseye |
[production] |
| 11:19 |
<kamila@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mw1379.eqiad.wmnet with reason: host reimage |
[production] |
| 11:16 |
<kamila@cumin1002> |
START - Cookbook sre.hosts.downtime for 2:00:00 on mw1379.eqiad.wmnet with reason: host reimage |
[production] |
| 11:01 |
<kamila@cumin1002> |
START - Cookbook sre.hosts.reimage for host mw1379.eqiad.wmnet with OS bullseye |
[production] |
| 10:51 |
<akosiaris@deploy2002> |
helmfile [codfw] DONE helmfile.d/admin 'apply'. |
[production] |
| 10:33 |
<akosiaris@deploy2002> |
helmfile [codfw] START helmfile.d/admin 'apply'. |
[production] |
| 10:32 |
<akosiaris@deploy2002> |
helmfile [eqiad] DONE helmfile.d/admin 'apply'. |
[production] |
| 10:17 |
<akosiaris> |
bump memory limits for calico-node in wikikube codfw/eqiad by 25% (i.e from 400Mi to 500Mi) take #3 |
[production] |
| 10:17 |
<akosiaris@deploy2002> |
helmfile [eqiad] START helmfile.d/admin 'apply'. |
[production] |
| 09:57 |
<akosiaris@deploy2002> |
helmfile [eqiad] DONE helmfile.d/admin 'apply'. |
[production] |
| 09:38 |
<akosiaris> |
delete mw1377-mw1383 from eqiad wikikube nodes |
[production] |
| 09:38 |
<akosiaris> |
bump memory limits for calico-node in wikikube codfw/eqiad by 25% (i.e from 400Mi to 500Mi) take #2 |
[production] |