| 2018-09-20
      
      § | 
    
  | 09:26 | <marostegui> | Change passwords for wikiuser on dbstore1002 - T200801 | [production] | 
            
  | 09:24 | <moritzm> | rebooting mc1024 for kernel security update | [production] | 
            
  | 08:57 | <moritzm> | rebooting mc1023 for kernel security update | [production] | 
            
  | 08:44 | <moritzm> | rebooting mc1022 for kernel security update | [production] | 
            
  | 08:06 | <banyek> | replication stopped and tables being compressed for s2 on dbstore2002 | [production] | 
            
  | 08:01 | <marostegui@deploy1001> | Synchronized wmf-config/db-codfw.php: Repool db2066 (duration: 00m 57s) | [production] | 
            
  | 07:53 | <marostegui> | Deploy schema change on db2066 | [production] | 
            
  | 07:53 | <marostegui@deploy1001> | Synchronized wmf-config/db-codfw.php: Depool db2066 (duration: 00m 57s) | [production] | 
            
  | 07:46 | <moritzm> | upgrading intel-microcode on trusty systems to 3.20180807a | [production] | 
            
  | 07:31 | <godog> | repair sdl on ms-be2042 - T199198 | [production] | 
            
  | 07:25 | <marostegui@deploy1001> | Synchronized wmf-config/db-eqiad.php: Repool pc1004 (duration: 00m 57s) | [production] | 
            
  | 07:15 | <marostegui> | Stop MySQL on pc1004 for kernel upgrade | [production] | 
            
  | 07:14 | <marostegui@deploy1001> | Synchronized wmf-config/db-eqiad.php: Depool pc1004 (duration: 00m 57s) | [production] | 
            
  | 06:50 | <eileen> | process-control config revision is 3069f85578 | [production] | 
            
  | 06:42 | <marostegui@deploy1001> | Synchronized wmf-config/db-codfw.php: Repool db2070 (duration: 00m 58s) | [production] | 
            
  | 06:25 | <mobrovac@deploy1001> | Started restart [cpjobqueue/deploy@58f9ed3]: Reset the Kafka connections | [production] | 
            
  | 06:20 | <mobrovac@deploy1001> | Synchronized rpc/RunSingleJob.php: Have RunSingleJob send the X-Readonly header - T204154 (duration: 00m 58s) | [production] | 
            
  | 06:09 | <marostegui> | Deploy schema change on db2070 - T203709 | [production] | 
            
  | 06:07 | <marostegui@deploy1001> | Synchronized wmf-config/db-codfw.php: Depool db2070 (duration: 00m 58s) | [production] | 
            
  | 05:54 | <marostegui> | Deploy schema change on s3:mediawikiwiki for echo tables codfw - T51593 | [production] | 
            
  | 05:49 | <marostegui@deploy1001> | Synchronized wmf-config/db-codfw.php: Resume reads on db2034 (duration: 00m 57s) | [production] | 
            
  | 05:34 | <marostegui> | Deploy schema change on db2034 (x1 master) | [production] | 
            
  | 05:33 | <marostegui@deploy1001> | Synchronized wmf-config/db-codfw.php: Stop reads on db2034 (duration: 00m 57s) | [production] | 
            
  | 05:28 | <marostegui@deploy1001> | Synchronized wmf-config/db-codfw.php: Repool db2069 after alter table (duration: 00m 57s) | [production] | 
            
  | 05:12 | <marostegui> | Deploy schema change on db2069 | [production] | 
            
  | 05:11 | <marostegui@deploy1001> | Synchronized wmf-config/db-codfw.php: Depool db2069 for alter table (duration: 01m 00s) | [production] | 
            
  | 03:06 | <l10nupdate@deploy1001> | ResourceLoader cache refresh completed at Thu Sep 20 03:06:52 UTC 2018 (duration 10m 47s) | [production] | 
            
  | 02:56 | <l10nupdate@deploy1001> | scap sync-l10n completed (1.32.0-wmf.22) (duration: 13m 54s) | [production] | 
            
  | 02:23 | <l10nupdate@deploy1001> | scap sync-l10n completed (1.32.0-wmf.20) (duration: 09m 26s) | [production] | 
            
  | 01:27 | <krinkle@deploy1001> | Synchronized php-1.32.0-wmf.22/resources/src/startup/: I6c77b25856 (duration: 00m 55s) | [production] | 
            
  | 01:14 | <krinkle@deploy1001> | Synchronized php-1.32.0-wmf.22/extensions/GlobalPreferences/includes/GlobalPreferencesFactory.php: I52434a523f60e, T204864 (duration: 00m 58s) | [production] | 
            
  | 00:52 | <krinkle@deploy1001> | Synchronized php-1.32.0-wmf.22/extensions/3D/modules/: I9e718957a497, T204621 (duration: 00m 58s) | [production] | 
            
  | 00:49 | <mutante> | temp disabled puppet on mw2*, deployed gerrit 461393, confirmed noop, re-enabled puppet on mw2* | [production] | 
            
  
    | 2018-09-19
      
      § | 
    
  | 23:21 | <catrope@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: Remove obsolete $wgPopupsBetaFeature, part 2 (T203589) (duration: 00m 56s) | [production] | 
            
  | 23:21 | <catrope@deploy1001> | Synchronized wmf-config/CommonSettings.php: Remove obsolete $wgPopupsBetaFeature, part 1 (T203589) (duration: 00m 56s) | [production] | 
            
  | 23:17 | <catrope@deploy1001> | Synchronized php-1.32.0-wmf.22/extensions/PageTriage/: PageTriage fixes for T203184 (duration: 00m 58s) | [production] | 
            
  | 23:14 | <catrope@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: Enable Page issues A/B test on lvwiki (T204609) (duration: 00m 58s) | [production] | 
            
  | 22:00 | <XioNoX> | merging icinga check_bfd | [production] | 
            
  | 21:24 | <jforrester@deploy1001> | Synchronized php-1.32.0-wmf.22/extensions/VisualEditor/includes/SpecialCollabPad.php: T204873 Fix special page override hot deploy (duration: 00m 57s) | [production] | 
            
  | 20:26 | <ladsgroup@deploy1001> | Finished deploy [ores/deploy@76fe25a]: ORES doesn't block hammering IPs (T204862) (duration: 21m 13s) | [production] | 
            
  | 20:24 | <XioNoX> | repool ulsfo | [production] | 
            
  | 20:21 | <ejegg> | updated fundraising CiviCRM from 2d9360cfac to 7f58670e88 | [production] | 
            
  | 20:11 | <bsitzmann@deploy1001> | Finished deploy [mobileapps/deploy@55d0b34]: Update mobileapps to a224e99 Ia80abe02490 (duration: 03m 29s) | [production] | 
            
  | 20:07 | <bsitzmann@deploy1001> | Started deploy [mobileapps/deploy@55d0b34]: Update mobileapps to a224e99 Ia80abe02490 | [production] | 
            
  | 20:05 | <ladsgroup@deploy1001> | Started deploy [ores/deploy@76fe25a]: ORES doesn't block hammering IPs (T204862) | [production] | 
            
  | 19:40 | <hashar> | web request 60 second timeout when deploying is filled as https://phabricator.wikimedia.org/T204871 | [production] | 
            
  | 19:30 | <hashar> | while promoting group1 to 1.32.0-wmf.22 lot of web requests timed out at 60 seconds. Roughly from 19:24 to 19:28. But that is no more occurring | T191068 | [production] | 
            
  | 19:25 | <hashar@deploy1001> | Synchronized php: group1 wikis to 1.32.0-wmf.22 (duration: 00m 55s) | [production] | 
            
  | 19:24 | <hashar@deploy1001> | rebuilt and synchronized wikiversions files: group1 wikis to 1.32.0-wmf.22 | [production] | 
            
  | 19:16 | <hashar@deploy1001> | Synchronized php-1.32.0-wmf.22/extensions/Translate/tag/PageTranslationHooks.php: Avoid warnings and errors caused by x-pagetranslation-tag - T204797 (duration: 00m 58s) | [production] |