3851-3900 of 10000 results (45ms)
2021-11-08 §
08:24 <elukey@deploy1002> helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revscoring-editquality' for release 'main' . [production]
05:52 <rzl> rebooted wikitech-static via rackspace web UI - T295266 [production]
2021-11-06 §
01:43 <dduvall@deploy1002> Synchronized php-1.38.0-wmf.7/includes/parser/ParserOutput.php: Backport: [[gerrit:737079|Regression fix: do language conversion on ToC in ParserOutput::getText() (T295187)]] (duration: 00m 56s) [production]
01:42 <dduvall> emergency backport https://gerrit.wikimedia.org/r/c/mediawiki/core/+/737079 deployed and verified on mwdebug1002. syncing to all targets [production]
01:40 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
01:37 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
01:33 <dduvall> performing emergency backport deployment of https://gerrit.wikimedia.org/r/c/mediawiki/core/+/737079 [production]
2021-11-05 §
23:26 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
23:19 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
23:05 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
22:58 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
22:48 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
22:45 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
22:35 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
22:32 <dduvall> re-rolling 1.38.0-wmf.7 to all wikis due to a better of two evil regressions UBN T295187 (refs T293948) [production]
22:32 <dduvall@deploy1002> rebuilt and synchronized wikiversions files: all wikis to 1.38.0-wmf.7 refs T293948 [production]
22:31 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
22:21 <dduvall@deploy1002> rebuilt and synchronized wikiversions files: Revert "group0/group1 to 1.38.0-wmf.7 refs T293948" [production]
22:19 <dduvall> rolling back 1.38.0-wmf.7 from group1 and group0 due to UBN T295187 (refs T293948) [production]
20:17 <dduvall@deploy1002> rebuilt and synchronized wikiversions files: Revert "all wikis to 1.38.0-wmf.7 refs T293948" [production]
20:09 <dduvall> rolling back 1.38.0-wmf.7 from all wikis due to UBN T295187 (refs T293948) [production]
18:41 <mutante> removing mediawiki font packages from labweb* (wikitech wiki) [production]
18:35 <XioNoX> cr2-codfw> request chassis fpc online slot 0 - T294789 [production]
18:20 <legoktm> upgrading scap to 4.0.3 everywhere (T294966) [production]
18:01 <mmandere@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host ganeti6001.drmrs.wmnet with OS buster [production]
17:22 <mmandere@cumin1001> START - Cookbook sre.hosts.reimage for host ganeti6001.drmrs.wmnet with OS buster [production]
16:52 <mmandere@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host ganeti6001.drmrs.wmnet with OS buster [production]
16:30 <mmandere@cumin1001> START - Cookbook sre.hosts.reimage for host ganeti6001.drmrs.wmnet with OS buster [production]
16:21 <elukey@cumin1001> END (PASS) - Cookbook sre.kafka.roll-restart-brokers (exit_code=0) for Kafka A:kafka-test-eqiad cluster: Roll restart of jvm daemons for openjdk upgrade. - elukey@cumin1001 [production]
16:01 <elukey@cumin1001> START - Cookbook sre.kafka.roll-restart-brokers for Kafka A:kafka-test-eqiad cluster: Roll restart of jvm daemons for openjdk upgrade. - elukey@cumin1001 [production]
15:38 <hnowlan@deploy1002> helmfile [staging] Ran 'sync' command on namespace 'api-gateway' for release 'staging' . [production]
15:38 <hnowlan@deploy1002> helmfile [staging] Ran 'sync' command on namespace 'api-gateway' for release 'production' . [production]
14:30 <jayme> published docker-registry.discovery.wmnet/golang1.17:1.17-1 [production]
13:42 <jmm@cumin2002> END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) for new host testvm2001.codfw.wmnet [production]
13:28 <jmm@cumin2002> START - Cookbook sre.ganeti.makevm for new host testvm2001.codfw.wmnet [production]
12:50 <mmandere@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host ganeti6001.drmrs.wmnet with OS buster [production]
12:22 <moritzm> renamed Ganeti group of test cluster from "default" to "row_A" (following conventions in main DCs) T286206 [production]
12:10 <mmandere@cumin1001> START - Cookbook sre.hosts.reimage for host ganeti6001.drmrs.wmnet with OS buster [production]
12:01 <jmm@cumin2002> END (FAIL) - Cookbook sre.ganeti.makevm (exit_code=99) for new host testvm2001.codfw.wmnet [production]
11:40 <jmm@cumin2002> START - Cookbook sre.ganeti.makevm for new host testvm2001.codfw.wmnet [production]
11:09 <mmandere@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host ganeti6001.drmrs.wmnet with OS buster [production]
10:29 <mmandere@cumin1001> START - Cookbook sre.hosts.reimage for host ganeti6001.drmrs.wmnet with OS buster [production]
09:53 <ema> cp[4033-4036]: upgrade varnish to 6.0.8-1wm2 T295120 [production]
09:43 <jmm@cumin2002> END (FAIL) - Cookbook sre.ganeti.makevm (exit_code=99) for new host testvm2002.codfw.wmnet [production]
09:39 <mmandere@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host ganeti6001.drmrs.wmnet with OS buster [production]
09:27 <jmm@cumin2002> START - Cookbook sre.ganeti.makevm for new host testvm2002.codfw.wmnet [production]
09:27 <jmm@cumin2002> END (FAIL) - Cookbook sre.ganeti.makevm (exit_code=99) for new host testvm2001.codfw.wmnet [production]
09:19 <Amir1> Upgrade db1151 T295026 [production]
09:09 <jmm@cumin2002> START - Cookbook sre.ganeti.makevm for new host testvm2001.codfw.wmnet [production]
09:01 <ema> apt.wm.org: remove varnish 6.0.8-1wm1 from component main of buster-wikimedia, we use component/varnish6 instead [production]