6351-6400 of 10000 results (70ms)
2019-03-25 §
13:41 <ema> cp1076: depool varnish-fe and point it to cp-ats T213263 [production]
13:28 <mutante> planet - manually updating en version since new monitoring check warned it wasn't current (T203208) [production]
13:17 <mutante> mwmaint1002 - manually running tor_exit_node cron command and test with PHP 7.2 [production]
12:48 <mutante> reloading icinga config [production]
12:15 <Lucas_WMDE> EU SWAT finished [production]
12:08 <oblivian@deploy1001> Synchronized wmf-config/CommonSettings.php: Move 0.1% of anonymous users to php7 T212828 (duration: 00m 49s) [production]
12:07 <moritzm> installing openssl1.0 security updates on stretch [production]
12:00 <lucaswerkmeister-wmde@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:498814|Revert "Remove $wgAbuseFilterRuntimeProfile" (T191039)]] (duration: 00m 51s) [production]
11:48 <lucaswerkmeister-wmde@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:486470|Remove $wgAbuseFilterRuntimeProfile (T191039)]] (duration: 00m 49s) [production]
11:46 <ema> cp-ats-codfw: upgrade trafficserver to 8.0.3-1wm1 [production]
11:35 <lucaswerkmeister-wmde@deploy1001> Synchronized php-1.33.0-wmf.22/extensions/Wikibase/repo: SWAT: [[gerrit:498354| Revert "OutputPageBeforeHTML: do nothing for non entity pages" (T218907)]] (duration: 01m 06s) [production]
11:26 <filippo@puppetmaster1001> conftool action : set/pooled=no; selector: name=prometheus2003.codfw.wmnet [production]
11:23 <filippo@puppetmaster1001> conftool action : set/pooled=yes; selector: name=prometheus2004.codfw.wmnet [production]
11:23 <godog> switch codfw prometheus from prometheus2003 to prometheus2004 [production]
11:19 <ema> cp-ats-eqiad: upgrade trafficserver to 8.0.3-1wm1 [production]
11:18 <oblivian@deploy1001> Synchronized wmf-config/ProductionServices.php: switching to use of the local proxy for search in php7 (duration: 00m 50s) [production]
11:16 <oblivian@deploy1001> Synchronized wmf-config/LabsServices.php: switching to use of the local proxy for search in php7 (duration: 00m 50s) [production]
11:09 <ema> trafficserver 8.0.3-1wm1 uploaded to stretch-wikimedia [production]
10:56 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Fully repool db1083 (duration: 00m 48s) [production]
10:40 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: More traffic to db1083 (duration: 00m 49s) [production]
10:40 <gehel> disable deprecation warnings on elasticsearch eqiad - T218994 [production]
10:38 <jdrewniak@deploy1001> Synchronized portals: Wikimedia Portals Update: [[gerrit:498800| Bumping portals to master (T128546)]] (duration: 00m 49s) [production]
10:37 <jdrewniak@deploy1001> Synchronized portals/wikipedia.org/assets: Wikimedia Portals Update: [[gerrit:498800| Bumping portals to master (T128546)]] (duration: 00m 49s) [production]
10:27 <moritzm> installing Java security updates on Hadoop/Druid test cluster [production]
10:26 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: More traffic to db1083 (duration: 00m 49s) [production]
10:12 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: More traffic to db1083 (duration: 00m 49s) [production]
10:07 <moritzm> installing ntfs-3g security updates [production]
10:02 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Slowly repool db1083 (duration: 00m 49s) [production]
09:42 <moritzm> uploaded openjdk 8u212-b01-1~deb8u1 to apt.wikimedia.org/jessie-wikimedia/main [production]
09:34 <marostegui> Upgrade db2062 [production]
09:24 <hashar> contint1001: manually compressing Zuul log files sudo -u zuul gzip --best /var/log/zuul/*.log.????-??-?? [production]
09:22 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1083+ (duration: 00m 49s) [production]
09:18 <marostegui> Upgrade db2055 [production]
09:16 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool db1105:3311 (duration: 00m 49s) [production]
09:10 <mutante> contint1001 - restarting zuul [production]
08:21 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1105:3311 (duration: 00m 49s) [production]
08:08 <vgutierrez> reenabling puppet in openldap servers [production]
08:05 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Fully repool db1118 (duration: 00m 49s) [production]
07:58 <vgutierrez> disable puppet and downtime host in icinga for labtestservices2001 - T218022 [production]
07:40 <vgutierrez> disable puppet in production openldap servers before merging https://gerrit.wikimedia.org/r/498776 [production]
07:24 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: More traffic to db1118 (duration: 00m 49s) [production]
06:59 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Slowly repool db1118 after mysql upgrade (duration: 00m 50s) [production]
06:45 <marostegui> Stop MySQL on db1118 for upgrade [production]
06:44 <marostegui> Deploy schema change on s1 codfw master, this will generate lag on codfw [production]
06:10 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1118 for schema change and upgrade (duration: 00m 54s) [production]
04:31 <chaomodus> restarted pdfrender on scb1003 to try to help flapping [production]
2019-03-24 §
15:00 <jijiki> Restart pdfrender on scb1002 and scb1004 [production]
2019-03-23 §
13:02 <mholloway-shell@deploy1001> Synchronized wmf-config/InitialiseSettings-labs.php: Fix WikimediaEditorTasks Beta Cluster DB config, take 2 (duration: 00m 50s) [production]
12:36 <mholloway-shell@deploy1001> Synchronized wmf-config/InitialiseSettings-labs.php: Fix WikimediaEditorTasks Beta Cluster DB config (duration: 00m 52s) [production]
2019-03-22 §
22:13 <bd808> Restarted uwsgi-striker on labweb1002 [production]