3801-3850 of 10000 results (93ms)
2020-03-05 §
10:48 <hnowlan@deploy1001> Synchronized multiversion/MWScript.php: T244549: enable running MWScript with phpdbg (duration: 01m 04s) [production]
10:48 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime [production]
10:18 <oblivian@deploy1001> Synchronized wmf-config/ProductionServices.php: Switch parsoid calls to use envoy as a proxy (duration: 01m 07s) [production]
10:14 <vgutierrez> Enable keep alive between ats-tls and varnish-fe globally - T244464 [production]
10:12 <marostegui> Stop MySQL on db2109 for reimage - T246604 [production]
10:11 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db2109 for reimage to buster - T246604', diff saved to https://phabricator.wikimedia.org/P10621 and previous config saved to /var/cache/conftool/dbconfig/20200305-101111-marostegui.json [production]
10:11 <addshore> START warm cache for db1111 & db1126 for Q25-30 million T219123 (pass 2 today) [production]
09:53 <hashar> Restarting Zuul, it no more process Gerrit events due to a thread stuck waiting on Gerrit.. T246973 [production]
08:50 <addshore> START warm cache for db1111 & db1126 for Q25-30 million T219123 (pass 1 today) [production]
08:12 <marostegui@cumin1001> dbctl commit (dc=all): 'Fully repool db1103:3312 db1103:3314 after reimage to buster T246604', diff saved to https://phabricator.wikimedia.org/P10619 and previous config saved to /var/cache/conftool/dbconfig/20200305-081227-marostegui.json [production]
07:33 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1103:3312 db1103:3314 after reimage to buster T246604', diff saved to https://phabricator.wikimedia.org/P10618 and previous config saved to /var/cache/conftool/dbconfig/20200305-073319-marostegui.json [production]
07:19 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1103:3312 db1103:3314 after reimage to buster T246604', diff saved to https://phabricator.wikimedia.org/P10617 and previous config saved to /var/cache/conftool/dbconfig/20200305-071915-marostegui.json [production]
06:56 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1103:3312 db1103:3314 after reimage to buster T246604', diff saved to https://phabricator.wikimedia.org/P10616 and previous config saved to /var/cache/conftool/dbconfig/20200305-065603-marostegui.json [production]
06:48 <elukey> restart yarn on analytics1074 (GC overhead, traces of network errors with datanodes) [production]
06:37 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
06:34 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime [production]
06:20 <marostegui> Stop MySQL on db1103:3312 and db1103:3314 for reimage T246604 [production]
06:18 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1103:3312 db1103:3314 for reimage T246604', diff saved to https://phabricator.wikimedia.org/P10615 and previous config saved to /var/cache/conftool/dbconfig/20200305-061811-marostegui.json [production]
04:22 <krinkle@deploy1001> Synchronized dblists/commonsuploads.dblist: Idb69b82f5 (duration: 01m 04s) [production]
03:23 <krinkle@deploy1001> Synchronized php-1.35.0-wmf.22/extensions/WikimediaMaintenance/dumpInterwiki.php: Iec6da824cca (duration: 01m 04s) [production]
03:10 <pt1979@cumin2001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
03:09 <krinkle@deploy1001> Synchronized php-1.35.0-wmf.22/includes/SiteConfiguration.php: I723133e68, I2b90e8e9b0 (duration: 01m 05s) [production]
03:08 <pt1979@cumin2001> START - Cookbook sre.hosts.downtime [production]
03:00 <pt1979@cumin2001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
02:57 <pt1979@cumin2001> START - Cookbook sre.hosts.downtime [production]
02:46 <pt1979@cumin2001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
02:43 <pt1979@cumin2001> START - Cookbook sre.hosts.downtime [production]
02:42 <krinkle@deploy1001> Synchronized multiversion/MWConfigCacheGenerator.php: Ib2aaf6540d85 (duration: 01m 04s) [production]
02:37 <pt1979@cumin2001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
02:35 <pt1979@cumin2001> START - Cookbook sre.hosts.downtime [production]
02:33 <krinkle@deploy1001> Synchronized wmf-config/InitialiseSettings.php: I52bb7024384 (no-op) (duration: 01m 04s) [production]
02:30 <krinkle@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Ia5b12516be59fd (duration: 01m 05s) [production]
02:23 <pt1979@cumin2001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
02:20 <pt1979@cumin2001> START - Cookbook sre.hosts.downtime [production]
02:13 <pt1979@cumin2001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
02:11 <pt1979@cumin2001> START - Cookbook sre.hosts.downtime [production]
01:03 <twentyafterfour> phabricator deployment done [production]
00:57 <twentyafterfour> deploying phabricator-extensions tag release/2020-03-04/1 ( https://phabricator.wikimedia.org/source/phab-extensions/history/wmf%252Fstable/;release/2020-03-04/1 ) [production]
00:45 <tgr@deploy1001> Synchronized php-1.35.0-wmf.22/extensions/GrowthExperiments/modules/homepage/: SWAT: [[gerrit:577002|Adjust topic UX (T244421)]] (duration: 01m 05s) [production]
00:41 <tgr@deploy1001> Synchronized php-1.35.0-wmf.22/extensions/GrowthExperiments/includes/NewcomerTasks/TaskSuggester/SearchStrategy/SearchStrategy.php: SWAT: [[gerrit:577001|Newcomer tasks: Set search sort to random for ORES based topics (T242476)]] (duration: 01m 04s) [production]
00:27 <krinkle@deploy1001> Synchronized wmf-config/interwiki.php: Update interwiki cache (duration: 01m 02s) [production]
00:24 <krinkle@deploy1001> Synchronized dblists/: I4fb3d14ed86 (duration: 01m 04s) [production]
00:14 <krinkle@deploy1001> update-interwiki-cache aborted: Update interwiki cache (duration: 00m 31s) [production]
00:08 <ebernhardson@deploy1001> Synchronized wmf-config/InitialiseSettings.php: cirrus: use 2 shards for commonswiki_content (duration: 01m 04s) [production]
00:06 <ebernhardson@deploy1001> Synchronized wmf-config/InitialiseSettings.php: cirrus: Backend configuration for glent m0 ab test (duration: 01m 04s) [production]
2020-03-04 §
23:30 <krinkle@deploy1001> Synchronized src/: Ic344b48a1f8 - creates StaticSiteConfiguration.php (build-only) (duration: 01m 03s) [production]
23:26 <reedy@deploy1001> Synchronized php-1.35.0-wmf.22/extensions/CirrusSearch/includes/: T245303 (duration: 01m 02s) [production]
23:01 <eileen> process-control config revision is 734a7bfadd [production]
22:59 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
22:56 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]