801-850 of 10000 results (58ms)
2019-06-28 §
15:02 <eevans@deploy1001> scap-helm sessionstore upgrade production -f sessionstore-eqiad-values.yaml stable/kask [namespace: sessionstore, clusters: eqiad] [production]
14:48 <ema> pool cp2011 w/ ATS backend T226637 [production]
14:47 <XioNoX> upload kafkatee to buster-wikimedia [production]
14:11 <eevans@deploy1001> scap-helm sessionstore finished [production]
14:11 <eevans@deploy1001> scap-helm sessionstore cluster staging completed [production]
14:11 <eevans@deploy1001> scap-helm sessionstore upgrade staging -f sessionstore-staging-values.yaml stable/kask [namespace: sessionstore, clusters: staging] [production]
14:07 <eevans@deploy1001> scap-helm sessionstore upgrade production -f sessionstore-staging-values.yaml stable/kask [namespace: sessionstore, clusters: staging] [production]
14:06 <ema> depool cp2011 and reimage as upload_ats T226637 [production]
11:36 <elukey> roll restart eventstreams on all scb1* nodes [production]
11:33 <elukey> restart eventstreams on scb1001 [production]
11:18 <fsero> draining kubernetes1006 for applying updates [production]
11:14 <fsero> draining kubernetes1005 for applying updates [production]
11:13 <fsero> draining kubernetes2006 for applying updates [production]
11:09 <fsero> draining kubernetes2005 for applying updates [production]
11:04 <_joe_> uploading php-wmerrors to thirdparty/php72 - T187147 [production]
10:31 <Reedy> running `foreachwiki extensions/TimedMediaHandler/maintenance/requeueTranscodes.php --audio --mime=audio/midi --missing --throttle` on mwmaint1002 in screen T226713 [production]
10:20 <reedy@deploy1001> Synchronized php-1.34.0-wmf.11/extensions/TimedMediaHandler/maintenance/requeueTranscodes.php: Extra filtering option (duration: 00m 51s) [production]
10:09 <ema> pool cp2008 w/ ATS backend T226637 [production]
09:28 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
09:28 <jmm@cumin2001> START - Cookbook sre.hosts.downtime [production]
09:17 <ema> depool cp2008 and reimage as upload_ats T226637 [production]
09:16 <elukey> systemctl reset-failed kafka* units on kafka2002 (role spare, failed units, already masked) [production]
09:16 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
09:15 <jmm@cumin2001> START - Cookbook sre.hosts.downtime [production]
09:10 <moritzm> rebooting releases* hosts for MDS-enabled qemu/kernel [production]
09:10 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
09:10 <jmm@cumin2001> START - Cookbook sre.hosts.downtime [production]
08:43 <elukey> roll restart of eventstreams on all scb2* nodes, service now working (kafka transport failures logged) [production]
08:02 <moritzm> updating openssl packages on mw1265 [production]
07:57 <ema> pool cp2005 w/ ATS backend T226637 [production]
07:11 <_joe_> upgrading php-wikidiff2 on the mw canaries, only on php7 - T223391 [production]
07:05 <ema> depool cp2005 and reimage as upload_ats T226637 [production]
01:22 <Krinkle> Killing arclamp-log on webperf1002, no flame graphs for three days, presumably mwlog/redis connection dropped again. T215740 [production]
2019-06-27 §
23:28 <catrope@deploy1001> Synchronized php-1.34.0-wmf.11/extensions/TimedMediaHandler/: T226748 (duration: 00m 50s) [production]
23:26 <catrope@deploy1001> Synchronized php-1.34.0-wmf.11/extensions/GrowthExperiments/includes/HomepageHooks.php: Fix JS error on Special:Homepage (duration: 00m 50s) [production]
23:25 <brion> roan is fixing deploy of T226748 which failed to include the patch (whoops) [production]
21:58 <cdanis> cdanis@cp1075.eqiad.wmnet ~ % sudo -i varnish-backend-restart [production]
21:44 <brion> deploying fix for TMH jobqueue bug T226748 [production]
20:31 <jforrester@deploy1001> Synchronized php-1.34.0-wmf.11/extensions/MobileFrontend/resources/dist: T221191: Log editor switches to visualeditorfeatureuse (duration: 00m 50s) [production]
20:18 <ladsgroup@deploy1001> Synchronized php-1.34.0-wmf.11/extensions/Wikibase: [[gerrit:519492|Avoid inserting a new addUsage job when the current usage stays untouched (duration: 01m 14s) [production]
19:23 <Urbanecm> run namespaceDupes.php for wikis in P8674 (T173070) [production]
19:23 <jhuneidi@deploy1001> rebuilt and synchronized wikiversions files: all wikis to 1.34.0-wmf.11 refs T220736 [production]
19:16 <ppchelko@deploy1001> Finished deploy [restbase/deploy@ff6f302]: Use new projects and new config layout T220855 (duration: 11m 21s) [production]
19:04 <ppchelko@deploy1001> Started deploy [restbase/deploy@ff6f302]: Use new projects and new config layout T220855 [production]
18:52 <Urbanecm> Morning SWAT done for real [production]
18:51 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: [[:gerrit:519484|Tidy up GroupOverrides]] (T173070) (duration: 00m 56s) [production]
18:49 <urbanecm@deploy1001> Synchronized dblists/commonsuploads.dblist: [[:gerrit:Tidy up GroupOverrides]], part 1 (T173070) (duration: 00m 57s) [production]
18:48 <Urbanecm> foreachwiki namespaceDupes.php --fix done (T173070) [production]
18:46 <Urbanecm> Reopen Morning SWAT [production]
18:33 <legoktm> gerrit set-account --active '"Dzahn"' [production]