1451-1500 of 10000 results (15ms)
2020-04-14 ยง
17:23 <ppchelko@deploy1001> Started deploy [changeprop/deploy@354ae2d]: Rollback removing k8s rules, again [production]
17:12 <ppchelko@deploy1001> Finished deploy [changeprop/deploy@354ae2d]: Remove rules enabled in k8s T248677 attempt 2 (duration: 00m 25s) [production]
17:12 <ppchelko@deploy1001> Started deploy [changeprop/deploy@354ae2d]: Remove rules enabled in k8s T248677 attempt 2 [production]
17:08 <hnowlan@deploy1001> helmfile [CODFW] Ran 'sync' command on namespace 'changeprop' for release 'production' . [production]
17:07 <hnowlan@deploy1001> helmfile [EQIAD] Ran 'sync' command on namespace 'changeprop' for release 'production' . [production]
17:05 <hnowlan@deploy1001> helmfile [STAGING] Ran 'sync' command on namespace 'changeprop' for release 'staging' . [production]
16:52 <jforrester@deploy1001> Started scap: Testwikis to php-1.35.0-wmf.28 and rebuild i18n cache for T247775 [production]
16:49 <jforrester@deploy1001> sync aborted: testwikis wikis to 1.35.0-wmf.28 (duration: 00m 05s) [production]
16:49 <jforrester@deploy1001> Started scap: testwikis wikis to 1.35.0-wmf.28 [production]
16:38 <akosiaris> stop all ganeti components (VMs are fine) on all ganeti2* hosts for key/cert rollover [production]
16:37 <jforrester@deploy1001> Pruned MediaWiki: 1.35.0-wmf.25 (duration: 17m 20s) [production]
16:20 <James_F> Scap cleaning 1.35.0-wmf.25 T247775 [production]
16:07 <ariel@deploy1001> Finished deploy [dumps/dumps@90cbab0]: fix listing of input files for 7z recompression, retry (duration: 00m 04s) [production]
16:06 <ariel@deploy1001> Started deploy [dumps/dumps@90cbab0]: fix listing of input files for 7z recompression, retry [production]
16:06 <ppchelko@deploy1001> Finished deploy [changeprop/deploy@baf0a4b]: Rollback removing k8s rules (duration: 01m 20s) [production]
16:06 <ejegg> disabled new recurring payments charge job [production]
16:05 <ppchelko@deploy1001> Started deploy [changeprop/deploy@baf0a4b]: Rollback removing k8s rules [production]
16:04 <ariel@deploy1001> Finished deploy [dumps/dumps@90cbab0]: fix listing of input files for 7z recompression (duration: 00m 04s) [production]
16:04 <ariel@deploy1001> Started deploy [dumps/dumps@90cbab0]: fix listing of input files for 7z recompression [production]
15:52 <ema> cp3050: suspend purged testing, varnish-frontend-restart to clear mailbox lag T249583 [production]
15:50 <hnowlan@deploy1001> helmfile [EQIAD] Ran 'sync' command on namespace 'changeprop' for release 'production' . [production]
15:49 <James_F> 1.35.0-wmf.28 was branched at ded5b87df12cea88d94dde0fa22cac13227f8e92 for T247775 [production]
15:47 <hnowlan@deploy1001> helmfile [CODFW] Ran 'sync' command on namespace 'changeprop' for release 'production' . [production]
15:19 <hnowlan@deploy1001> helmfile [EQIAD] Ran 'sync' command on namespace 'changeprop' for release 'production' . [production]
15:17 <hnowlan@deploy1001> helmfile [EQIAD] Ran 'sync' command on namespace 'changeprop' for release 'production' . [production]
15:15 <vgutierrez> update to ats 8.0.7-rc0-1wm2 on cp[4026,4032] - T249335 [production]
15:13 <vgutierrez> upload trafficserver 8.0.7-rc0-1wm2 to apt.wm.o (buster) - T249335 [production]
15:12 <hnowlan@deploy1001> helmfile [CODFW] Ran 'sync' command on namespace 'changeprop' for release 'production' . [production]
15:11 <hnowlan@deploy1001> helmfile [CODFW] Ran 'sync' command on namespace 'changeprop' for release 'production' . [production]
14:44 <ppchelko@deploy1001> Finished deploy [changeprop/deploy@354ae2d]: Remove rules enabled in k8s T248677 (duration: 01m 58s) [production]
14:42 <ppchelko@deploy1001> Started deploy [changeprop/deploy@354ae2d]: Remove rules enabled in k8s T248677 [production]
14:34 <godog> power down ms-be1023 - T249174 [production]
14:33 <filippo@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
14:33 <filippo@cumin1001> START - Cookbook sre.hosts.downtime [production]
14:33 <filippo@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
14:33 <filippo@cumin1001> START - Cookbook sre.hosts.downtime [production]
14:15 <elukey> enable TLS between weblog1001,mwlog2001.codfw.wmnet,mwlog1001 and Kafka Jumbo/Logging - T250147 [production]
14:15 <hashar> Rebasing mediawiki-config on deploy1001 for a deployment-prep config change ( https://gerrit.wikimedia.org/r/#/c/operations/mediawiki-config/+/588706/ ) [production]
14:12 <ema> cp3050: resume purged testing T249583 [production]
13:55 <ema> upload purged 0.4 to buster-wikimedia T249583 [production]
13:21 <hashar> Starting zuul-merger on contint2001 [production]
12:50 <vgutierrez> Enable inbound TLSv1.3 in text@eqsin - T170567 [production]
12:03 <jbond42> upgrade haproxy on dns servers [production]
11:08 <Urbanecm> EU SWAT done [production]
11:05 <Urbanecm> Purge https://en.wikipedia.org/static/images/project-logos/cswiki*.png (T249173) [production]
11:04 <urbanecm@deploy1001> Synchronized static/images/project-logos/: SWAT: 7da408e: Revert "Enable cswiki anniversary logo" (T249173) (duration: 01m 00s) [production]
11:01 <jynus> resizing backup1001:/srv/databases to 40 TB [production]
10:55 <XioNoX> set uRPF log action to syslog infra wide - T244147 [production]
10:15 <XioNoX> update prefix-list LVS-service-ips to add missing prefixes [production]
09:49 <XioNoX> re-order aggregate routes to standardize order [production]