| 2018-09-26
      
      § | 
    
  | 23:23 | <thcipriani@deploy1001> | Synchronized php-1.32.0-wmf.23/extensions/WikimediaEvents/modules/all/ext.wikimediaEvents.readingDepth.js: SWAT: [[gerrit:463139|Ensure Minerva has initialised before loading and executing ReadingDepth]] T204144 (duration: 00m 57s) | [production] | 
            
  | 23:14 | <eileen> | process control -renable omnirepair -  process-control config revision is b8549d3344 | [production] | 
            
  | 22:48 | <eileen> | civicrm revision changed from 018ff6325f to 43ca105873, config revision is 3b785f0af4 | [production] | 
            
  | 22:28 | <XioNoX> | enable BGP to transits on cr3-ulsfo | [production] | 
            
  | 21:20 | <thcipriani> | restarting releases jenkins for updates | [production] | 
            
  | 21:03 | <gtirloni> | tools-mail deleted frozen messages from exim queue | [production] | 
            
  | 19:54 | <mutante> | icinga1001 - temp. disabling puppet, remove --purge all icinga packages, rm -rf /etc/nagios and /etc/icinga, let puppet recreate everything now that it should not mess with user/group on stretch (T202782) | [production] | 
            
  | 19:42 | <krinkle@deploy1001> | Synchronized php-1.32.0-wmf.23/extensions/Gadgets/includes/SpecialGadgets.php: (no justification provided) (duration: 00m 57s) | [production] | 
            
  | 19:25 | <shdubsh> | manually installing python3-jinja2 on puppetmaster1001 to test naggen2 python3 upgrade | [production] | 
            
  | 18:01 | <joal@deploy1001> | Finished deploy [analytics/aqs/deploy@39b909e]: Deploy Wikistats2 top-metrics updates (duration: 11m 58s) | [production] | 
            
  | 17:49 | <joal@deploy1001> | Started deploy [analytics/aqs/deploy@39b909e]: Deploy Wikistats2 top-metrics updates | [production] | 
            
  | 17:22 | <jforrester@deploy1001> | Synchronized php-1.32.0-wmf.23/extensions/3D/src/ThreeDThumbnailImage.php: Hot-deploy I5bb4b699a fix for T205554 train-blocker (duration: 00m 56s) | [production] | 
            
  | 16:47 | <jforrester@deploy1001> | Synchronized php-1.32.0-wmf.23/extensions/ORES/includes/SpecialORESModels.php: SWAT T205228 fix Iddb9c1f9e (duration: 00m 56s) | [production] | 
            
  | 16:36 | <XioNoX> | shutting down cr1/2-ulsfo for DC move | [production] | 
            
  | 16:36 | <robh> | shutting down all ulsfo servers for relocation | [production] | 
            
  | 16:35 | <jforrester@deploy1001> | Synchronized php-1.32.0-wmf.22/extensions/CentralAuth/includes/CentralAuthHooks.php: SWAT deploy I65ae2f05e (duration: 00m 56s) | [production] | 
            
  | 16:34 | <jforrester@deploy1001> | Synchronized php-1.32.0-wmf.23/extensions/CentralAuth/includes/CentralAuthHooks.php: SWAT deploy I65ae2f05e (duration: 00m 57s) | [production] | 
            
  | 16:18 | <godog> | test formatting sdd on ms-be2040 with crc=0 - T199198 | [production] | 
            
  | 16:10 | <zfilipin@deploy1001> | Synchronized php: group1 wikis to 1.32.0-wmf.23 (duration: 00m 54s) | [production] | 
            
  | 16:09 | <zfilipin@deploy1001> | rebuilt and synchronized wikiversions files: group1 wikis to 1.32.0-wmf.23 | [production] | 
            
  | 16:07 | <bblack> | downtimed ipsec alerts on cp[12]xxx for ulsfo outage | [production] | 
            
  | 15:55 | <addshore@deploy1001> | Synchronized php-1.32.0-wmf.22/extensions/WikibaseLexeme: [[gerrit:463060|form statement groups: ids without lexeme part]] T196226 (duration: 01m 01s) | [production] | 
            
  | 15:54 | <XioNoX> | ulsfo X-connect hot cut scheduled for in 5min | [production] | 
            
  | 15:53 | <addshore@deploy1001> | Synchronized php-1.32.0-wmf.23/extensions/WikibaseLexeme: [[gerrit:463050|form statement groups: ids without lexeme part]] T196226 (duration: 01m 06s) | [production] | 
            
  | 15:29 | <pmiazga@deploy1001> | Synchronized php-1.32.0-wmf.23/skins/MinervaNeue/includes/skins/SkinMinerva.php: SWAT: [[gerrit:463087|Create $returntoquery variable properly(T205449)]] (duration: 00m 56s) | [production] | 
            
  | 15:02 | <gehel> | rolling restart of wdqs-test and wdqs-internal for JVM + kernel upgrade | [production] | 
            
  | 14:48 | <reedy@deploy1001> | Synchronized php-1.32.0-wmf.23/includes/specials/SpecialMostlinkedcategories.php: T205469 (duration: 00m 55s) | [production] | 
            
  | 14:46 | <reedy@deploy1001> | Synchronized php-1.32.0-wmf.23/extensions/CirrusSearch/: T205473 (duration: 01m 06s) | [production] | 
            
  | 13:45 | <marostegui> | Stop replication on dbstore1002:s4 for maintenance | [production] | 
            
  | 13:38 | <elukey> | reboot an-master1001 to clear out an issue with systemd@fsck (Hadoop master, failover to an-master1002 included) | [production] | 
            
  | 13:34 | <marostegui> | Stop replication on s4 eqiad master for maintenance, lag will be generated | [production] | 
            
  | 13:32 | <marostegui@deploy1001> | Synchronized wmf-config/db-codfw.php: Repool db2071 (duration: 00m 55s) | [production] | 
            
  | 13:12 | <marostegui> | Deploy schema change on db2071 - T203709 | [production] | 
            
  | 13:12 | <marostegui@deploy1001> | Synchronized wmf-config/db-codfw.php: Depool db2071 (duration: 00m 56s) | [production] | 
            
  | 13:03 | <moritzm> | installing tomcat8 security updates | [production] | 
            
  | 12:33 | <marostegui> | Deploy schema change on s4 eqiad, will generate lag - T203709 | [production] | 
            
  | 11:43 | <zeljkof> | EU SWAT finished | [production] | 
            
  | 11:42 | <zfilipin@deploy1001> | Synchronized wmf-config/throttle.php: SWAT: [[gerrit:462928|Lift account creation IP cap for 2018-09-26 (T205529)]] (duration: 00m 56s) | [production] | 
            
  | 11:30 | <zfilipin@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:462191|Enable VisualEditor in Project namespace on srwiki (T205206)]] (duration: 00m 57s) | [production] | 
            
  | 11:11 | <volans> | disabled puppet on einsteinium to test ircecho failure, I'll re-enable it in max ~30m | [production] | 
            
  | 10:30 | <volans> | started ircecho on einsteinium | [production] | 
            
  | 10:27 | <volans> | stop ircecho on einsteinium to register the nickname | [production] | 
            
  | 10:26 | <moritzm> | rebooting mw1270-mw1290 for kernel security updates | [production] | 
            
  | 10:13 | <_joe_> | reenabling puppet on the MediaWiki hosts | [production] | 
            
  | 10:08 | <moritzm> | rebooting conf1006 for kernel security update | [production] | 
            
  | 09:51 | <moritzm> | rebooting conf1005 for kernel security update | [production] |