3551-3600 of 10000 results (80ms)
2019-11-04 §
15:43 <mholloway-shell@deploy1001> Synchronized wmf-config/InitialiseSettings-labs.php: Enable revert counts on beta (T234955) (duration: 00m 53s) [production]
15:36 <jynus> running failing check_private_data report on labsdb1009 T235743 [production]
15:33 <ladsgroup@deploy1001> Synchronized php-1.35.0-wmf.4/extensions/Wikibase: Wikibase deadlock reduction, [[gerrit:548279|Stop locking and use DISTINCT when finding used terms to delete]] (T236466) (duration: 00m 59s) [production]
15:01 <joal@deploy1001> Started restart [analytics/aqs/deploy@59a97fa]: (no justification provided) [production]
14:36 <ema@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
14:36 <ema@cumin1001> START - Cookbook sre.hosts.downtime [production]
13:53 <ema> upload trafficserver 8.0.5-1wm10 to stretch-wikimedia [production]
13:49 <ema@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
13:47 <ema@cumin1001> START - Cookbook sre.hosts.downtime [production]
13:38 <elukey> update bacula terms on analytics-in{4,6} filters on cr{1,2}-eqiad - T237016 [production]
13:28 <jbond42> update production puppetmasters to use new puppetdb servers [production]
13:20 <Amir1> Creating Mon Wikipedia is done T235739 [production]
13:19 <ladsgroup@deploy1001> Synchronized wmf-config/interwiki.php: Update interwiki cache (duration: 02m 39s) [production]
13:16 <ladsgroup@deploy1001> Synchronized langlist: T235739 (duration: 00m 52s) [production]
13:15 <ladsgroup@deploy1001> Synchronized static/images/project-logos/: T235739 (duration: 00m 53s) [production]
13:14 <ladsgroup@deploy1001> Synchronized wmf-config/InitialiseSettings.php: T235739 (duration: 00m 53s) [production]
13:13 <ladsgroup@deploy1001> Synchronized multiversion/MWMultiVersion.php: T235739 (duration: 00m 52s) [production]
13:12 <ladsgroup@deploy1001> rebuilt and synchronized wikiversions files: T235739 [production]
13:07 <ladsgroup@deploy1001> Synchronized dblists: (no justification provided) (duration: 00m 53s) [production]
13:06 <ema> depool cp5012 and reimage as text_ats T227432 [production]
12:21 <Urbanecm> EU SWAT done [production]
12:14 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: 7c1c64c: Add localized Minerva wordmark for Sindhi Wikipedia (T200870; 2/2) (duration: 00m 52s) [production]
12:12 <Urbanecm> Purge https://en.wikipedia.org/static/images/project-logos/zh_classicalwiki* (T236905) [production]
12:11 <urbanecm@deploy1001> Synchronized static/images/mobile/copyright/: SWAT: 7c1c64c: Add localized Minerva wordmark for Sindhi Wikipedia (T200870; 1/2) (duration: 00m 53s) [production]
12:08 <urbanecm@deploy1001> Synchronized static/images/project-logos/: SWAT: a6d64b1: Update logo for zh-classical Wikipedia (T236905) (duration: 00m 53s) [production]
12:05 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: c92a13c: Enable partial blocks on kowiki (T236752) (duration: 00m 54s) [production]
12:00 <moritzm> upgrading mw1261 to PHP 7.2.24 (T237239) [production]
11:39 <jdrewniak@deploy1001> Synchronized portals: Wikimedia Portals Update: [[gerrit:548231| Bumping portals to master (T128546)]] (duration: 00m 52s) [production]
11:38 <jdrewniak@deploy1001> Synchronized portals/wikipedia.org/assets: Wikimedia Portals Update: [[gerrit:548231| Bumping portals to master (T128546)]] (duration: 01m 03s) [production]
11:08 <moritzm> uploaded PHP 7.2.24 to apt.wikimedia.org stretch-wikimedia/component/php72 (T237239) [production]
04:53 <vgutierrez> Switch from nginx to ats-tls on cp5009 - T231627 [production]
04:39 <vgutierrez> Switch from nginx to ats-tls on cp5008 - T231627 [production]
2019-11-03 §
03:54 <andrew@deploy1001> Finished deploy [horizon/deploy@0c024d4]: one more prefix fix (duration: 03m 35s) [production]
03:50 <andrew@deploy1001> Started deploy [horizon/deploy@0c024d4]: one more prefix fix [production]
03:10 <andrew@deploy1001> Finished deploy [horizon/deploy@9972ed2]: deploying fix for puppet prefix creation (second try) (duration: 00m 25s) [production]
03:10 <andrew@deploy1001> Started deploy [horizon/deploy@9972ed2]: deploying fix for puppet prefix creation (second try) [production]
03:09 <andrew@deploy1001> Finished deploy [horizon/deploy@9972ed2]: deploying fix for puppet prefix creation (duration: 06m 01s) [production]
03:03 <andrew@deploy1001> Started deploy [horizon/deploy@9972ed2]: deploying fix for puppet prefix creation [production]
2019-11-02 §
00:58 <mutante> gerrit-replica - created missing /var/lib/gerrit2/review_site/tmp and restarted service - service back up on buster (T176774) [production]
00:34 <mutante> gerrit-replica - fixing permissions of files in /srv/gerrit and restarting [production]
00:27 <mutante> gerrit2001 - copy mysql-connector-java.jar into /usr/share/java/ and link it into /var/lib/gerrit2/review_site/lib (T176774) [production]
00:05 <mutante> rsyncing gerrit plugin dir from gerrit1001 to gerrit2001 (T176774) [production]
2019-11-01 §
23:45 <mutante> rsyncing gerrit git data from gerrit1001 to gerrit2001 (using --delete too!) T176774 [production]
22:00 <mutante> gerrit - repo sync between gerrit and gerrit-replica in progress .. if you can't clone from replica you can use main gerrit and replica will come back [production]
21:20 <jforrester@deploy1001> Synchronized php-1.35.0-wmf.4/extensions/UploadWizard/resources/mw.UploadWizardUploadInterface.js: T237126 Fixing DOM in upload interface of UploadWizard (duration: 00m 56s) [production]
21:06 <mutante> scp /usr/share/java/mysql-connector-java.jar from gerrit1001 to gerrit2001 (T176774) [production]
20:46 <cdanis> add to bot_blocked_nets the IPs of several EC2 instances sending expensive requests to ORES T237134 [production]
19:54 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
19:52 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime [production]
19:36 <mutante> gerrit2001 - reinstalling with buster [production]