6251-6300 of 10000 results (63ms)
2018-10-01 §
17:06 <jynus@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool db1093, db1064 (duration: 00m 57s) [production]
17:02 <jynus> stopping some mariadb instances on dbstore1001 and starting compression T201392 [production]
16:26 <ppchelko@deploy1001> Started restart [cpjobqueue/deploy@58f9ed3]: Fix KafkaConsumer not connected error [production]
15:16 <jynus> stopping db1064 to clone it to dbstore1001 [production]
15:00 <akosiaris> upgrade etherpad to 1.7.0-2 [production]
14:14 <anomie@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Setting MCR migration stage to write-both/read-new on mediawikiwiki (T198308) (duration: 00m 56s) [production]
13:51 <banyek> Downtimed the slave lag monitoring on dbstore1002 while the tables getting converted (T205544) [production]
12:38 <akosiaris> upload hfst_3.13.0~r3461-1+wmf2 to apt.wikimedia.org/jessie-wikimedia/main. T199962 [production]
12:26 <banyek> converting enwiki.categorylinks to TokuDB on host dbstrore1002 (T205544) [production]
12:19 <banyek> stopping replication on s2@dbstore20002: the tables being compressed (T204930) [production]
12:19 <banyek> stopping replication on s2@dbstore20002: the tables being compressed [production]
12:15 <banyek> enabling puppet on labsdb1009, labsdb1010, labsdb1011 (T183983) [production]
12:13 <zeljkof> EU SWAT finished [production]
12:12 <zfilipin@deploy1001> Synchronized php-1.32.0-wmf.23/extensions/ContentTranslation/: SWAT: [[gerrit:463446|Fix error in CXTransclusionNode#afterRender method (T205521)]] (duration: 00m 59s) [production]
11:56 <jynus> stopping db1093 to clone it to dbstore1001 [production]
11:52 <arturo> install prometheus-openstack-exporte 0.0.8-3 in reprepro T203177 [production]
11:41 <zfilipin@deploy1001> Synchronized wmf-config: SWAT: [[gerrit:460492|Remove unused default source language config for CX]] (duration: 00m 57s) [production]
11:16 <jynus@deploy1001> Synchronized wmf-config/db-codfw.php: Repool db2058 (duration: 00m 55s) [production]
11:09 <_joe_> killed bash runner.sh by user ladsgroup on mwmaint2001 [production]
10:58 <jynus@deploy1001> Synchronized wmf-config/db-codfw.php: Depool db2058 (duration: 00m 57s) [production]
10:52 <jynus@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1093, db1064 (duration: 00m 57s) [production]
10:42 <jdrewniak@deploy1001> Synchronized portals: Wikimedia Portals Update: [[gerrit:463723|Bumping portals to master (T128546)]] (duration: 00m 56s) [production]
10:41 <jdrewniak@deploy1001> Synchronized portals/wikipedia.org/assets: Wikimedia Portals Update: [[gerrit:463723|Bumping portals to master (T128546)]] (duration: 00m 59s) [production]
10:21 <godog> repair /dev/sdf1 /dev/sde1 on ms-be1041 - T199198 [production]
10:15 <Amir1> ladsgroup@mwmaint2001:~$ mwscript extensions/CentralAuth/maintenance/deleteLocalPasswords.php --prefix on all CentralAuth wikis (T201009) [production]
10:10 <Amir1> mwscript extensions/CentralAuth/maintenance/deleteLocalPasswords.php --wiki=fawiki --delete (T201009) [production]
09:33 <godog> test formatting sdh and sdi on ms-be2040 with crc=0 - T199198 [production]
09:15 <volans> Set Racktables in read-only mode - T199083 [production]
08:56 <_joe_> rolling restart of parsoid in codfw; afterwards, parsoid will connect to the MediaWiki API via HTTPS [production]
08:54 <_joe_> rolling restart of parsoid in eqiad [production]
07:54 <banyek> disabling puppet on labsdb1009, labsdb1010, labsdb1011 (T183983) [production]
07:54 <banyek> disabling puppet on labsdb1009, labsdb1010, labsdb1011 [production]
07:00 <mholloway-shell@deploy1001> Finished deploy [kartotherian/deploy@ab6cb74] (maps1004): Update kartotherian to latest (T205462) (duration: 00m 16s) [production]
06:59 <mholloway-shell@deploy1001> Started deploy [kartotherian/deploy@ab6cb74] (maps1004): Update kartotherian to latest (T205462) [production]
06:39 <mholloway-shell@deploy1001> Finished deploy [tilerator/deploy@22f90ee] (maps1004): Update tilerator to latest (T205462) (duration: 00m 19s) [production]
06:39 <mholloway-shell@deploy1001> Started deploy [tilerator/deploy@22f90ee] (maps1004): Update tilerator to latest (T205462) [production]
05:35 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool db1103:3312 (duration: 00m 56s) [production]
05:19 <marostegui> Stop replication on dbstore1002 and db1103:3312 in sync [production]
05:19 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1103:3312 (duration: 01m 01s) [production]
05:19 <mholloway-shell@deploy1001> Finished deploy [mobileapps/deploy@07cbfb4]: Update mobileapps to a1fa41b (duration: 03m 18s) [production]
05:15 <mholloway-shell@deploy1001> Started deploy [mobileapps/deploy@07cbfb4]: Update mobileapps to a1fa41b [production]
05:07 <marostegui> Deploy schema change on s1 codfw msater - T203709 [production]
03:21 <onimisionipe> restarting inplace reindexing of enwiki and viwiki at codfw - T204362 [production]
2018-09-30 §
12:41 <ariel@deploy1001> Finished deploy [dumps/dumps@26aaee6]: make location of MWScript.php configurable (duration: 00m 03s) [production]
12:41 <ariel@deploy1001> Started deploy [dumps/dumps@26aaee6]: make location of MWScript.php configurable [production]
2018-09-29 §
13:01 <gtirloni> tools-mail cleaned frozen messages in exim queue [production]
01:13 <krinkle@deploy1001> Synchronized php-1.32.0-wmf.23/extensions/ORES/includes/ORESService.php: T205651 - I1beaeab732a31d (duration: 00m 59s) [production]
2018-09-28 §
20:57 <mutante> analytics1003 - unmounted and remounted /mnt/hdfs after Icinga alerts that it was not accessible - commands from https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Hadoop/Administration#Fixing_HDFS_mount_at_/mnt/hdfs - like it happened before on stat1004 and others (T182342) [production]
19:18 <mutante> phab1001 (Phabricator), scheduled downtime, reboot for maintenance [production]
19:03 <mutante> phab2001 - scheduled downtime, rebooting for kernel [production]