1301-1350 of 10000 results (78ms)
2019-09-16 §
12:17 <elukey@cumin1001> START - Cookbook sre.hadoop.reboot-workers [production]
12:07 <_joe_> rolling restart ended on eqiad T232613 [production]
11:56 <_joe_> rolling restart of php-fpm in eqiad to pick up the new memcached extension T232613 [production]
11:50 <_joe_> rolling restart of php-fpm in codfw to pick up the new memcached extension T232613 [production]
11:43 <Urbanecm> EU SWAT is done [production]
11:38 <urbanecm@deploy1001> Synchronized wmf-config/throttle.php: SWAT: e37aed2: Remove expired throttle rules (duration: 01m 03s) [production]
11:36 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: 313e3d9: Increase move rate-limit on Commons for all autopatrolled users (T232657) (duration: 01m 05s) [production]
11:33 <jbond42> update peer address of AS28598 [production]
11:30 <effie> Upgrading php-memcached to 3.0.1+2.2.0-1~wmf3 [production]
11:30 <awight@deploy1001> Synchronized php-1.34.0-wmf.22/extensions/FileImporter: SWAT: [[gerrit:537053|Send a User-Agent with remote API requests (T232840)]] (duration: 01m 02s) [production]
11:28 <urbanecm@deploy1001> Synchronized wmf-config/throttle.php: SWAT: 869b56f: Lift IP cap on 2019-10-02 for Senior Citizen Write Wikipedia course - cs.wikipedia (T232831) (duration: 01m 02s) [production]
11:21 <awight@deploy1001> Synchronized wmf-config/CommonSettings-labs.php: SWAT: [[gerrit:536974|Enable File Importer source wiki edits on beta cluster (T228851)]] (duration: 01m 03s) [production]
11:14 <awight@deploy1001> Synchronized wmf-config/CommonSettings.php: SWAT: [[gerrit:536590|Enable source wiki editing for testwiki (T228851)]] (duration: 01m 02s) [production]
11:10 <awight@deploy1001> Synchronized php-1.34.0-wmf.22/extensions/FileImporter: SWAT: [[gerrit:536980|Add debug logging for remote API failures (T228851)]] (duration: 01m 05s) [production]
11:06 <_joe_> uploaded php-memcached_3.0.1+2.2.0-1~wmf3 to component/php72 for stretch T232613 [production]
10:52 <vgutierrez@puppetmaster1001> conftool action : set/pooled=yes; selector: name=ncredir2002.codfw.wmnet [production]
10:51 <vgutierrez@puppetmaster1001> conftool action : set/pooled=no; selector: name=ncredir2002.codfw.wmnet [production]
10:50 <jdrewniak@deploy1001> Synchronized portals: Wikimedia Portals Update: [[gerrit:537060| Bumping portals to master (T128546)]] (duration: 01m 03s) [production]
10:49 <jdrewniak@deploy1001> Synchronized portals/wikipedia.org/assets: Wikimedia Portals Update: [[gerrit:537060| Bumping portals to master (T128546)]] (duration: 01m 04s) [production]
10:45 <vgutierrez> Enabling OCSP prefetched responses for the non-canonical redirect service - T232988 [production]
10:29 <_joe_> installing a patched php-memcached on mw1347 T232613 [production]
10:16 <vgutierrez> upgrade acme-chief production servers to acme-chief 0.21 - T219765 [production]
10:16 <moritzm> upload libtrapperkeeper-webserver-jetty9-clojure 1.7.0-2+wmf1 to buster-wikimedia [production]
10:05 <vgutierrez> restarting acmechief servers to get latest kernel upgrades [production]
09:26 <elukey@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
09:24 <elukey@cumin1001> START - Cookbook sre.hosts.downtime [production]
08:57 <vgutierrez> replacing nginx with ATS in cp3034 (upload cluster) - T231433 [production]
08:56 <mobrovac@deploy1001> Synchronized wmf-config/CommonSettings.php: Beta: enable the Parsoid extension - T231569 (duration: 01m 01s) [production]
08:50 <marostegui> Apply grants for dbproxy1021 on db1133 (m5 master) with replication - T202367 [production]
08:16 <elukey@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
08:14 <elukey@cumin1001> START - Cookbook sre.hosts.downtime [production]
08:11 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
08:09 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime [production]
07:38 <moritzm> installing faad2 security updates [production]
07:15 <moritzm> repooling restbase2009 [production]
06:48 <marostegui> Stop MySQL on db1114 to upgrade it to 10.3 [production]
06:03 <marostegui> Stop MySQL on db2054 for decommissioning T232969 [production]
06:01 <marostegui> Remove db2054 from tendril and zarcillo T232969 [production]
05:59 <marostegui@deploy1001> Synchronized wmf-config/db-codfw.php: Remove db2054 from config T232969 (duration: 01m 03s) [production]
05:58 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Remove db2054 from config T232969 (duration: 01m 05s) [production]
2019-09-15 §
16:51 <Krinkle> Fixed a dozen abuse filtes, listed at https://phabricator.wikimedia.org/T156096#5494060. The trailing pipe character was removed from filters that had it which is no longer supported in a future version of AbuseFilter. [production]
14:35 <_joe_> test: setting opcache.interned_strings_buffer to 0 on mw1348 for T232613 [production]
2019-09-14 §
23:42 <onimisionipe> force shard allocation (dewiki_content_1566659363[4]) on eqiad cluster [production]
04:39 <effie> Depool and reload mw1286 [production]
01:14 <ejegg> updated fundraising python tools from 1e405864d7 to e1b81688c6 [production]
00:29 <ejegg> updated payments-wiki from 1f556670cf to fc82318180 [production]
2019-09-13 §
23:44 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
23:42 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime [production]
23:06 <gehel> re-enable puppet on maps - T232817 [production]
20:23 <chaomodus> restarting netbox1001.wikimedia.org [production]