651-700 of 10000 results (52ms)
2018-09-27 §
07:08 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Repool db1089 (duration: 00m 55s) [production]
07:05 <jynus> disablingh puppet on all databases [production]
07:00 <marostegui> Stop replication in sync on db1089 and dbstore1002:s1 [production]
06:27 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Depool db1089 (duration: 00m 56s) [production]
06:17 <marostegui> Deploy schema change on dbstore2002:3311 [production]
05:41 <marostegui> Deploy schema change on s1 eqiad master, lag will be generated - T203709 [production]
05:40 <marostegui@deploy1001> Synchronized wmf-config/db-codfw.php: Repool db2072 (duration: 00m 57s) [production]
05:26 <marostegui> Drop wikiuser on dbstore1002 [production]
05:21 <marostegui> Deploy schema change on db2072 [production]
05:21 <marostegui@deploy1001> Synchronized wmf-config/db-codfw.php: Depool db2072 (duration: 01m 00s) [production]
03:04 <l10nupdate@deploy1001> ResourceLoader cache refresh completed at Thu Sep 27 03:04:40 UTC 2018 (duration 10m 51s) [production]
02:53 <l10nupdate@deploy1001> scap sync-l10n completed (1.32.0-wmf.23) (duration: 07m 16s) [production]
02:41 <onimisionipe> starting inplace reindexing of viwiki and commonswiki - T204362 [production]
02:35 <l10nupdate@deploy1001> scap sync-l10n completed (1.32.0-wmf.22) (duration: 13m 25s) [production]
01:14 <mutante> repair /dev/sdn1 on ms-be0241 - T199198 [production]
01:14 <mutante> repair /dev/sde1 on ms-be0240 - T199198 [production]
00:22 <krinkle@deploy1001> Synchronized php-1.32.0-wmf.23/extensions/FeaturedFeeds: T205573 (duration: 00m 59s) [production]
2018-09-26 §
23:23 <thcipriani@deploy1001> Synchronized php-1.32.0-wmf.23/extensions/WikimediaEvents/modules/all/ext.wikimediaEvents.readingDepth.js: SWAT: [[gerrit:463139|Ensure Minerva has initialised before loading and executing ReadingDepth]] T204144 (duration: 00m 57s) [production]
23:14 <eileen> process control -renable omnirepair - process-control config revision is b8549d3344 [production]
22:48 <eileen> civicrm revision changed from 018ff6325f to 43ca105873, config revision is 3b785f0af4 [production]
22:28 <XioNoX> enable BGP to transits on cr3-ulsfo [production]
21:20 <thcipriani> restarting releases jenkins for updates [production]
21:03 <gtirloni> tools-mail deleted frozen messages from exim queue [production]
19:54 <mutante> icinga1001 - temp. disabling puppet, remove --purge all icinga packages, rm -rf /etc/nagios and /etc/icinga, let puppet recreate everything now that it should not mess with user/group on stretch (T202782) [production]
19:42 <krinkle@deploy1001> Synchronized php-1.32.0-wmf.23/extensions/Gadgets/includes/SpecialGadgets.php: (no justification provided) (duration: 00m 57s) [production]
19:25 <shdubsh> manually installing python3-jinja2 on puppetmaster1001 to test naggen2 python3 upgrade [production]
18:01 <joal@deploy1001> Finished deploy [analytics/aqs/deploy@39b909e]: Deploy Wikistats2 top-metrics updates (duration: 11m 58s) [production]
17:49 <joal@deploy1001> Started deploy [analytics/aqs/deploy@39b909e]: Deploy Wikistats2 top-metrics updates [production]
17:22 <jforrester@deploy1001> Synchronized php-1.32.0-wmf.23/extensions/3D/src/ThreeDThumbnailImage.php: Hot-deploy I5bb4b699a fix for T205554 train-blocker (duration: 00m 56s) [production]
16:47 <jforrester@deploy1001> Synchronized php-1.32.0-wmf.23/extensions/ORES/includes/SpecialORESModels.php: SWAT T205228 fix Iddb9c1f9e (duration: 00m 56s) [production]
16:36 <XioNoX> shutting down cr1/2-ulsfo for DC move [production]
16:36 <robh> shutting down all ulsfo servers for relocation [production]
16:35 <jforrester@deploy1001> Synchronized php-1.32.0-wmf.22/extensions/CentralAuth/includes/CentralAuthHooks.php: SWAT deploy I65ae2f05e (duration: 00m 56s) [production]
16:34 <jforrester@deploy1001> Synchronized php-1.32.0-wmf.23/extensions/CentralAuth/includes/CentralAuthHooks.php: SWAT deploy I65ae2f05e (duration: 00m 57s) [production]
16:18 <godog> test formatting sdd on ms-be2040 with crc=0 - T199198 [production]
16:10 <zfilipin@deploy1001> Synchronized php: group1 wikis to 1.32.0-wmf.23 (duration: 00m 54s) [production]
16:09 <zfilipin@deploy1001> rebuilt and synchronized wikiversions files: group1 wikis to 1.32.0-wmf.23 [production]
16:07 <bblack> downtimed ipsec alerts on cp[12]xxx for ulsfo outage [production]
15:55 <addshore@deploy1001> Synchronized php-1.32.0-wmf.22/extensions/WikibaseLexeme: [[gerrit:463060|form statement groups: ids without lexeme part]] T196226 (duration: 01m 01s) [production]
15:54 <XioNoX> ulsfo X-connect hot cut scheduled for in 5min [production]
15:53 <addshore@deploy1001> Synchronized php-1.32.0-wmf.23/extensions/WikibaseLexeme: [[gerrit:463050|form statement groups: ids without lexeme part]] T196226 (duration: 01m 06s) [production]
15:29 <pmiazga@deploy1001> Synchronized php-1.32.0-wmf.23/skins/MinervaNeue/includes/skins/SkinMinerva.php: SWAT: [[gerrit:463087|Create $returntoquery variable properly(T205449)]] (duration: 00m 56s) [production]
15:02 <gehel> rolling restart of wdqs-test and wdqs-internal for JVM + kernel upgrade [production]
14:48 <reedy@deploy1001> Synchronized php-1.32.0-wmf.23/includes/specials/SpecialMostlinkedcategories.php: T205469 (duration: 00m 55s) [production]
14:46 <reedy@deploy1001> Synchronized php-1.32.0-wmf.23/extensions/CirrusSearch/: T205473 (duration: 01m 06s) [production]
13:45 <marostegui> Stop replication on dbstore1002:s4 for maintenance [production]
13:38 <elukey> reboot an-master1001 to clear out an issue with systemd@fsck (Hadoop master, failover to an-master1002 included) [production]
13:34 <marostegui> Stop replication on s4 eqiad master for maintenance, lag will be generated [production]
13:32 <marostegui@deploy1001> Synchronized wmf-config/db-codfw.php: Repool db2071 (duration: 00m 55s) [production]
13:12 <marostegui> Deploy schema change on db2071 - T203709 [production]