|
2021-11-10
ยง
|
| 19:49 |
<mutante> |
- removing manually added things in Horizon Hiera that were already in the repo, please don't keep adding in web UI, we don't want to repeat the same thing we did in deployment-prep |
[devtools] |
| 19:46 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 19:43 |
<cjming> |
end of UTC evening backport & config window |
[production] |
| 19:42 |
<cjming> |
end of UTC late backport & config window |
[production] |
| 19:41 |
<cjming@deploy1002> |
Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:737814|Lower mobile web click tracking rate (T295432)]] (duration: 00m 55s) |
[production] |
| 19:36 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 19:35 |
<cjming@deploy1002> |
Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:737814|Lower mobile web click tracking rate (T295432)]] (duration: 00m 57s) |
[production] |
| 19:33 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 19:23 |
<legoktm> |
uploaded php-pcov_1.0.6-4+wmf1~buster1_amd64.changes to apt.wm.o (T243847) |
[production] |
| 18:57 |
<mutante> |
removing mediawiki font packages from parsoid hosts - T294378 |
[production] |
| 18:37 |
<bblack@cumin1001> |
START - Cookbook sre.hosts.reimage for host dns6002.wikimedia.org with OS buster |
[production] |
| 18:37 |
<bblack@cumin1001> |
START - Cookbook sre.hosts.reimage for host dns6001.wikimedia.org with OS buster |
[production] |
| 18:20 |
<btullis> |
btullis@an-launcher1002:~$ sudo systemctl reset-failed monitor_refine_event_sanitized_analytics_delayed.service |
[analytics] |
| 18:19 |
<dancy@deploy1002> |
Finished scap: Config: [[gerrit:737976|Get rid of obsolete train-versions.json file]] (duration: 15m 57s) |
[production] |
| 18:09 |
<bblack> |
drmrs - rebooting a bunch of hosts to bios for further settings, please ignore any accidental alerts - they do *look* like they're alert-disabled) |
[production] |
| 18:08 |
<vgutierrez> |
restart haproxy on cp4026 and cp5006 to enable hitless reloads - T290005 |
[production] |
| 18:07 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 18:03 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 18:03 |
<dancy@deploy1002> |
Started scap: Config: [[gerrit:737976|Get rid of obsolete train-versions.json file]] |
[production] |
| 17:10 |
<bblack@cumin1001> |
END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host dns6001.wikimedia.org with OS buster |
[production] |
| 17:08 |
<dpifke> |
Cherry-picking https://gerrit.wikimedia.org/r/c/operations/puppet/+/737970 in deployment-prep. Should only affect deployment-webperf11. |
[releng] |
| 16:49 |
<bblack@cumin1001> |
END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host dns6002.wikimedia.org with OS buster |
[production] |
| 16:47 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 16:44 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 16:34 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 16:32 |
<ebernhardson@deploy1002> |
Synchronized wmf-config/InitialiseSettings.php: T295480: Move all cirrussearch traffic to codfw (duration: 00m 55s) |
[production] |
| 16:30 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 16:28 |
<elukey> |
move atskafka to the new CA bundle - T291905 |
[production] |
| 16:26 |
<elukey> |
move kafkatee instances (analytics-test,centralog) to the new CA bundle - T291905 |
[production] |
| 16:21 |
<_joe_> |
cherry-picking puppet patches for mediawiki on php 7.4 to deployment-prep |
[releng] |
| 16:14 |
<bblack@cumin1001> |
START - Cookbook sre.hosts.reimage for host dns6002.wikimedia.org with OS buster |
[production] |
| 16:12 |
<bblack@cumin1001> |
START - Cookbook sre.hosts.reimage for host dns6001.wikimedia.org with OS buster |
[production] |
| 16:07 |
<hashar> |
reloading Zuul for https://gerrit.wikimedia.org/r/c/integration/config/+/737677 |
[releng] |
| 15:52 |
<ebernhardson@deploy1002> |
Synchronized wmf-config/InitialiseSettings.php: T295480: Move all cirrussearch traffic to codfw (duration: 00m 56s) |
[production] |
| 15:35 |
<majavah> |
delete shut down deployment-logstash* and deployment-mwmaint01 instances |
[releng] |
| 14:09 |
<legoktm> |
restarted mailman3/mailman3-web to pick up new DNS for m5-master |
[production] |
| 14:08 |
<elukey@cumin1001> |
END (PASS) - Cookbook sre.kafka.roll-restart-brokers (exit_code=0) for Kafka A:kafka-test-eqiad cluster: Roll restart of jvm daemons for openjdk upgrade. - elukey@cumin1001 |
[production] |
| 14:02 |
<ayounsi@cumin1001> |
END (PASS) - Cookbook sre.dns.netbox (exit_code=0) |
[production] |
| 13:48 |
<elukey@cumin1001> |
START - Cookbook sre.kafka.roll-restart-brokers for Kafka A:kafka-test-eqiad cluster: Roll restart of jvm daemons for openjdk upgrade. - elukey@cumin1001 |
[production] |
| 13:47 |
<ayounsi@cumin1001> |
START - Cookbook sre.dns.netbox |
[production] |
| 13:46 |
<elukey@cumin1001> |
END (PASS) - Cookbook sre.kafka.roll-restart-mirror-maker (exit_code=0) restart MirrorMaker for Kafka A:kafka-mirror-maker-test-eqiad cluster: Roll restart of jvm daemons. - elukey@cumin1001 |
[production] |
| 13:36 |
<elukey@cumin1001> |
START - Cookbook sre.kafka.roll-restart-mirror-maker restart MirrorMaker for Kafka A:kafka-mirror-maker-test-eqiad cluster: Roll restart of jvm daemons. - elukey@cumin1001 |
[production] |
| 13:13 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 13:10 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 13:03 |
<Lucas_WMDE> |
UTC morning backport+config window done |
[production] |
| 13:01 |
<lucaswerkmeister-wmde@deploy1002> |
Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:737696|Enable the visual editor on the 2022 namespace on Wikimania wiki (T295267)]] (duration: 00m 55s) |
[production] |
| 12:59 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 12:56 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 12:53 |
<lucaswerkmeister-wmde@deploy1002> |
Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:737695|Update $wgNamespacesToBeSearchedDefault for Wikimania 2022 (T295267)]] (duration: 00m 55s) |
[production] |
| 12:46 |
<XioNoX> |
delete route6 object for 2a02:ec80::/32 (split in two /48s) |
[production] |