|
2021-08-18
§
|
| 14:26 |
<effie> |
enable puppet on alert* |
[production] |
| 14:11 |
<effie> |
disable puppet on alerts* to avoid alert flood due to 713494 |
[production] |
| 14:01 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 13:57 |
<jiji@deploy1002> |
Synchronized wmf-config/ProductionServices.php: Config: [[gerrit:713619|ProductionServices: change rdb* servers in eqiad and codfw (T280582)]] (duration: 01m 51s) |
[production] |
| 13:57 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 13:41 |
<godog> |
bounce logstash on logstash100[89] |
[production] |
| 13:33 |
<jgiannelos@deploy1002> |
helmfile [staging] Ran 'sync' command on namespace 'tegola-vector-tiles' for release 'main' . |
[production] |
| 13:24 |
<effie> |
mw2383 is depooled - T286463 |
[production] |
| 13:01 |
<kormat> |
Deploying wmfmariadbpy 0.7.2 T289139 |
[production] |
| 13:01 |
<kormat> |
uploaded wmfmariadbpy 0.7.2 to apt.wm.o |
[production] |
| 11:38 |
<zpapierski@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'rdf-streaming-updater' for release 'main' . |
[production] |
| 11:36 |
<zpapierski@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'rdf-streaming-updater' for release 'main' . |
[production] |
| 11:35 |
<zpapierski@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'rdf-streaming-updater' for release 'main' . |
[production] |
| 11:12 |
<jiji@cumin1001> |
conftool action : set/pooled=inactive; selector: name=mw2383.codfw.wmnet |
[production] |
| 11:03 |
<zpapierski@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'rdf-streaming-updater' for release 'main' . |
[production] |
| 10:47 |
<effie> |
pooling mw2383 - T286463 |
[production] |
| 10:41 |
<zpapierski@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'rdf-streaming-updater' for release 'main' . |
[production] |
| 10:18 |
<hnowlan@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 3 days, 0:00:00 on maps1004.eqiad.wmnet with reason: Awaiting decommissioning |
[production] |
| 10:18 |
<hnowlan@cumin1001> |
START - Cookbook sre.hosts.downtime for 3 days, 0:00:00 on maps1004.eqiad.wmnet with reason: Awaiting decommissioning |
[production] |
| 10:13 |
<jiji@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mw2383.codfw.wmnet with reason: REIMAGE |
[production] |
| 10:10 |
<jiji@cumin1001> |
START - Cookbook sre.hosts.downtime for 2:00:00 on mw2383.codfw.wmnet with reason: REIMAGE |
[production] |
| 09:36 |
<joal@deploy1002> |
Finished deploy [analytics/refinery@88c6618] (hadoop-test): Regular analytics weekly train TEST [analytics/refinery@88c6618] (duration: 05m 48s) |
[production] |
| 09:30 |
<joal@deploy1002> |
Started deploy [analytics/refinery@88c6618] (hadoop-test): Regular analytics weekly train TEST [analytics/refinery@88c6618] |
[production] |
| 09:30 |
<joal@deploy1002> |
Finished deploy [analytics/refinery@88c6618] (thin): Regular analytics weekly train THIN [analytics/refinery@88c6618] (duration: 00m 07s) |
[production] |
| 09:30 |
<joal@deploy1002> |
Started deploy [analytics/refinery@88c6618] (thin): Regular analytics weekly train THIN [analytics/refinery@88c6618] |
[production] |
| 09:29 |
<joal@deploy1002> |
Finished deploy [analytics/refinery@88c6618]: Regular analytics weekly train [analytics/refinery@88c6618] (duration: 32m 29s) |
[production] |
| 08:57 |
<joal@deploy1002> |
Started deploy [analytics/refinery@88c6618]: Regular analytics weekly train [analytics/refinery@88c6618] |
[production] |
| 04:38 |
<marostegui> |
Drop user2 from s6 - T289051 |
[production] |
| 02:03 |
<rzl@cumin2001> |
conftool action : get/pooled; selector: service=docker-registry |
[production] |
| 00:39 |
<dpifke@deploy1002> |
Finished deploy [performance/navtiming@88f12a0]: Revert CpuBenchmark again (T281243) (duration: 00m 05s) |
[production] |
| 00:39 |
<dpifke@deploy1002> |
Started deploy [performance/navtiming@88f12a0]: Revert CpuBenchmark again (T281243) |
[production] |
| 00:38 |
<dpifke@deploy1002> |
Finished deploy [performance/navtiming@88f12a0]: Re-deploy fixed CpuBenchmark (T281243) (duration: 00m 06s) |
[production] |
| 00:38 |
<dpifke@deploy1002> |
Started deploy [performance/navtiming@88f12a0]: Re-deploy fixed CpuBenchmark (T281243) |
[production] |
|
2021-08-17
§
|
| 23:40 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 23:33 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 23:32 |
<ebernhardson@deploy1002> |
Synchronized php-1.37.0-wmf.19/extensions/CirrusSearch/maintenance/UpdateSuggesterIndex.php: T288233: Work around cache failure for wikitech (duration: 01m 28s) |
[production] |
| 23:05 |
<tzatziki> |
resetting email for vanished user |
[production] |
| 21:51 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 21:44 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 21:44 |
<urbanecm> |
Deploy security patch for T289063 |
[production] |
| 20:30 |
<brennen> |
running scap pull on mw2383 |
[production] |
| 20:29 |
<brennen@deploy1002> |
Pruned MediaWiki: 1.37.0-wmf.16 (duration: 02m 01s) |
[production] |
| 20:20 |
<brennen@deploy1002> |
Pruned MediaWiki: 1.37.0-wmf.15 (duration: 06m 51s) |
[production] |
| 20:14 |
<brennen> |
pruning 1.37.0-wmf.15 and .16 (T281160) |
[production] |
| 20:06 |
<urbanecm@deploy1002> |
Synchronized php-1.37.0-wmf.18/includes/block/BlockUser.php: d377d4fae704640c81172a6fa94b12b2efdba42c: BlockUser: Restore blocking autoblocked IP addresses (T287798) (duration: 01m 08s) |
[production] |
| 19:05 |
<brennen@deploy1002> |
rebuilt and synchronized wikiversions files: group0 wikis to 1.37.0-wmf.19 |
[production] |
| 19:02 |
<brennen> |
1.37.0-wmf.19 train status: no current blockers, proceeding to group0 (T281160) |
[production] |
| 17:44 |
<ladsgroup@deploy1002> |
Synchronized php-1.37.0-wmf.18/includes/: Backport: [[gerrit:713506|Revert "objectcache: make use of new `modtoken` field in SqlBagOStuff" (T288998)]] (duration: 01m 13s) |
[production] |
| 17:41 |
<urbanecm> |
[urbanecm@mw2383 ~]$ scap pull # to clear an icinga alert |
[production] |
| 17:39 |
<ladsgroup@deploy1002> |
Synchronized php-1.37.0-wmf.19/includes/: Backport: [[gerrit:713365|Revert "objectcache: make use of new `modtoken` field in SqlBagOStuff" (T288998)]] (duration: 01m 14s) |
[production] |