6401-6450 of 10000 results (40ms)
2020-04-16 §
11:17 <elukey> stop atskafka on cp3050 to re-create the topic atskafka_test_webrequest_text on Kafka Jumbo - T250347 [production]
11:16 <Urbanecm> EU SWAT done [production]
11:14 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: a105f38: Remove broken groupOverrides from amwikimedia (T249585) (duration: 01m 05s) [production]
11:12 <urbanecm@deploy1001> Synchronized wmf-config/CommonSettings.php: SWAT: 70ee5f6: Remove grants for tboverride and tboverride-account (T241114) (duration: 01m 06s) [production]
11:05 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: 74ad793: Turn off direct account creations at Testwikidata (T250348; take II) (duration: 01m 04s) [production]
11:04 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: 74ad793: Turn off direct account creations at Testwikidata (T250348) (duration: 01m 06s) [production]
11:03 <urbanecm@deploy1001> sync-file aborted: SWAT: 74ad793: Turn off direct account creations at Testwikidata (duration: 00m 00s) [production]
10:54 <hnowlan@deploy1001> helmfile [EQIAD] Ran 'sync' command on namespace 'changeprop' for release 'production' . [production]
10:45 <hnowlan@deploy1001> Finished deploy [changeprop/deploy@354ae2d]: Testing rules moved to k8s (duration: 01m 16s) [production]
10:44 <vgutierrez> upgrading ATS to version 8.0.7-rc0-1wm3 - T249335 [production]
10:44 <hnowlan@deploy1001> Started deploy [changeprop/deploy@354ae2d]: Testing rules moved to k8s [production]
10:44 <vgutierrez> rolling restart of ats-tls to enable TLSv1.3 globally and disable the old TLS session cache - T170567 [production]
10:35 <hnowlan@deploy1001> helmfile [CODFW] Ran 'sync' command on namespace 'changeprop' for release 'production' . [production]
10:35 <hnowlan@deploy1001> helmfile [EQIAD] Ran 'sync' command on namespace 'changeprop' for release 'production' . [production]
10:31 <hnowlan@deploy1001> helmfile [EQIAD] Ran 'sync' command on namespace 'changeprop' for release 'production' . [production]
10:22 <hnowlan@deploy1001> helmfile [STAGING] Ran 'sync' command on namespace 'changeprop' for release 'staging' . [production]
09:33 <elukey> restart atskafka on cp3050 to pick up snappy compression - T250347 [production]
09:32 <ema> cp2027: upgrade varnish to 5.1.3-1wm14 T249810 [production]
09:17 <ema> text@esams: stop vhtcpd, start purged T249325 [production]
09:16 <jynus> starting es backups on backup2002 T79922 [production]
08:33 <kormat> Disconnect pc1008 replication from pc1010 T247787 [production]
08:22 <ema> cp3050: upgrade purged to 0.7 T249583 [production]
08:22 <ema> upload purged 0.7 to buster-wikimedia T249583 [production]
08:21 <Urbanecm> Set email for Geraki@grwikimedia (T245911) [production]
08:18 <kormat@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool pc1008 as pc2 master T247787 (duration: 01m 08s) [production]
08:06 <mutante> mw1396 - restarted php7.2-fpm - was: 503 Service Unavailable - header 'X-Powered-By: PHP/7.' not found on 'http://en.wikipedia.org:80/wiki/Main_Page' [production]
08:04 <mutante> mw1396 - restarted apache [production]
07:50 <vgutierrez> rolling update ats to version 8.0.7-rc0-1wm3 in cp[4026,4032,5006,5012] - T249335 [production]
07:49 <vgutierrez> upload trafficserver 8.0.7-rc0-1wm3 to apt.wm.o (buster) - T249335 [production]
07:15 <volker-e@deploy1001> Finished deploy [design/style-guide@2a7cc4a]: Deploy design/style-guide: (duration: 00m 08s) [production]
07:15 <volker-e@deploy1001> Started deploy [design/style-guide@2a7cc4a]: Deploy design/style-guide: [production]
06:33 <moritzm> installing apache-log4j1.2 security updates on jessie [production]
06:29 <moritzm> installing icu security updates on jessie [production]
06:15 <moritzm> installing git security updates on jessie [production]
05:43 <marostegui@cumin1001> dbctl commit (dc=all): 'Reorganize s8 weights a little bit after the addition of the new host db1114', diff saved to https://phabricator.wikimedia.org/P10995 and previous config saved to /var/cache/conftool/dbconfig/20200416-054353-marostegui.json [production]
05:33 <elukey> restart hadoop-yarn-nodemanager on an-worker108[4,5] - failed after GC OOM events (heavy spark jobs) [production]
2020-04-15 §
22:11 <mholloway-shell@deploy1001> Synchronized php-1.35.0-wmf.28/extensions/MachineVision: Fix: Initialize categories array for initial images (T250321) (duration: 01m 07s) [production]
21:48 <maryum> removing duplicate incdices from production ES clusters that were created when reindexing failed [production]
20:16 <bsitzmann@deploy1001> Finished deploy [mobileapps/deploy@1907571]: Update mobileapps to ff34d0b5 (duration: 04m 57s) [production]
20:11 <bsitzmann@deploy1001> Started deploy [mobileapps/deploy@1907571]: Update mobileapps to ff34d0b5 [production]
19:53 <addshore> pool wdqs1006 caught up [production]
19:44 <addshore> depool wdqs1006 to catch up on lag [production]
19:04 <jforrester@deploy1001> Synchronized php: group1 wikis to 1.35.0-wmf.28 (duration: 01m 05s) [production]
19:03 <jforrester@deploy1001> rebuilt and synchronized wikiversions files: group1 wikis to 1.35.0-wmf.28 [production]
18:44 <krinkle@deploy1001> Synchronized wmf-config/CommonSettings.php: Idc81a885b2f3, T196309 (duration: 01m 07s) [production]
18:12 <tgr@deploy1001> Synchronized wmf-config/InitialiseSettings.php: re-sync (duration: 01m 07s) [production]
18:10 <tgr@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:589029|Fix GrowthExperiments helpdesk URL for frwiktionary (T235964)]] (duration: 01m 06s) [production]
16:08 <volker-e@deploy1001> Finished deploy [design/style-guide@a4d5794]: Deploy design/style-guide: (duration: 00m 11s) [production]
16:08 <volker-e@deploy1001> Started deploy [design/style-guide@a4d5794]: Deploy design/style-guide: [production]
15:46 <ejegg> updated fundraising CiviCRM from 18d7567cd7 to 1224b080c1 [production]