4301-4350 of 10000 results (44ms)
2021-11-08 §
10:49 <hnowlan@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'api-gateway' for release 'production' . [production]
10:49 <hnowlan@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'api-gateway' for release 'staging' . [production]
10:49 <vgutierrez@cumin1001> START - Cookbook sre.hosts.reimage for host cp4026.ulsfo.wmnet with OS buster [production]
10:27 <vgutierrez> depool cp4026 to be reimaged as a haproxy-tls test node - T290005 [production]
10:27 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2003.codfw.wmnet [production]
10:17 <Lucas_WMDE> Deployed patch for T294693 [production]
09:47 <XioNoX> all core routers: add drmrs to prefix lists + confed [production]
09:47 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host ganeti-test2003.codfw.wmnet [production]
09:46 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2002.codfw.wmnet [production]
09:23 <elukey@deploy1002> helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revscoring-articlequality' for release 'main' . [production]
09:22 <elukey@deploy1002> helmfile [ml-serve-eqiad] DONE helmfile.d/admin 'sync'. [production]
09:22 <elukey@deploy1002> helmfile [ml-serve-eqiad] START helmfile.d/admin 'sync'. [production]
09:21 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host ganeti-test2002.codfw.wmnet [production]
09:21 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2001.codfw.wmnet [production]
09:16 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host ganeti-test2001.codfw.wmnet [production]
08:51 <elukey@deploy1002> helmfile [ml-serve-eqiad] DONE helmfile.d/admin 'sync'. [production]
08:51 <elukey@deploy1002> helmfile [ml-serve-eqiad] START helmfile.d/admin 'sync'. [production]
08:24 <elukey@deploy1002> helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revscoring-editquality' for release 'main' . [production]
05:52 <rzl> rebooted wikitech-static via rackspace web UI - T295266 [production]
2021-11-06 §
01:43 <dduvall@deploy1002> Synchronized php-1.38.0-wmf.7/includes/parser/ParserOutput.php: Backport: [[gerrit:737079|Regression fix: do language conversion on ToC in ParserOutput::getText() (T295187)]] (duration: 00m 56s) [production]
01:42 <dduvall> emergency backport https://gerrit.wikimedia.org/r/c/mediawiki/core/+/737079 deployed and verified on mwdebug1002. syncing to all targets [production]
01:40 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
01:37 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
01:33 <dduvall> performing emergency backport deployment of https://gerrit.wikimedia.org/r/c/mediawiki/core/+/737079 [production]
2021-11-05 §
23:26 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
23:19 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
23:05 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
22:58 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
22:48 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
22:45 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
22:35 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
22:32 <dduvall> re-rolling 1.38.0-wmf.7 to all wikis due to a better of two evil regressions UBN T295187 (refs T293948) [production]
22:32 <dduvall@deploy1002> rebuilt and synchronized wikiversions files: all wikis to 1.38.0-wmf.7 refs T293948 [production]
22:31 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
22:21 <dduvall@deploy1002> rebuilt and synchronized wikiversions files: Revert "group0/group1 to 1.38.0-wmf.7 refs T293948" [production]
22:19 <dduvall> rolling back 1.38.0-wmf.7 from group1 and group0 due to UBN T295187 (refs T293948) [production]
20:17 <dduvall@deploy1002> rebuilt and synchronized wikiversions files: Revert "all wikis to 1.38.0-wmf.7 refs T293948" [production]
20:09 <dduvall> rolling back 1.38.0-wmf.7 from all wikis due to UBN T295187 (refs T293948) [production]
18:41 <mutante> removing mediawiki font packages from labweb* (wikitech wiki) [production]
18:35 <XioNoX> cr2-codfw> request chassis fpc online slot 0 - T294789 [production]
18:20 <legoktm> upgrading scap to 4.0.3 everywhere (T294966) [production]
18:01 <mmandere@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host ganeti6001.drmrs.wmnet with OS buster [production]
17:22 <mmandere@cumin1001> START - Cookbook sre.hosts.reimage for host ganeti6001.drmrs.wmnet with OS buster [production]
16:52 <mmandere@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host ganeti6001.drmrs.wmnet with OS buster [production]
16:30 <mmandere@cumin1001> START - Cookbook sre.hosts.reimage for host ganeti6001.drmrs.wmnet with OS buster [production]
16:21 <elukey@cumin1001> END (PASS) - Cookbook sre.kafka.roll-restart-brokers (exit_code=0) for Kafka A:kafka-test-eqiad cluster: Roll restart of jvm daemons for openjdk upgrade. - elukey@cumin1001 [production]
16:01 <elukey@cumin1001> START - Cookbook sre.kafka.roll-restart-brokers for Kafka A:kafka-test-eqiad cluster: Roll restart of jvm daemons for openjdk upgrade. - elukey@cumin1001 [production]
15:38 <hnowlan@deploy1002> helmfile [staging] Ran 'sync' command on namespace 'api-gateway' for release 'staging' . [production]
15:38 <hnowlan@deploy1002> helmfile [staging] Ran 'sync' command on namespace 'api-gateway' for release 'production' . [production]
14:30 <jayme> published docker-registry.discovery.wmnet/golang1.17:1.17-1 [production]