| 
      
        2018-09-04
      
      ยง
     | 
  
    
  | 15:08 | 
  <marostegui> | 
  Deploy schema change on db1078 | 
  [production] | 
            
  | 15:07 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1078 (duration: 00m 56s) | 
  [production] | 
            
  | 15:02 | 
  <marostegui> | 
  Deploy schema change on db1095:3313 | 
  [production] | 
            
  | 15:02 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1123 (duration: 00m 57s) | 
  [production] | 
            
  | 14:45 | 
  <addshore@deploy1001> | 
  Synchronized php-1.32.0-wmf.20/extensions/Wikibase: [[gerrit:457912|Track new ItemId formatter usages]] (duration: 01m 17s) | 
  [production] | 
            
  | 14:14 | 
  <bstorm_> | 
  Removing subtree_check from project nfs on labstore1004/5 | 
  [production] | 
            
  | 14:01 | 
  <jynus> | 
  stopping replication and running partitioning on logging on db1088:3311 T189107 | 
  [production] | 
            
  | 14:00 | 
  <hashar@deploy1001> | 
  rebuilt and synchronized wikiversions files: group0 to 1.32.0-wmf.20 | 
  [production] | 
            
  | 13:45 | 
  <hashar@deploy1001> | 
  Finished scap: testwiki to php-1.32.0-wmf.20 and rebuild l10n cache - T191066 (duration: 62m 55s) | 
  [production] | 
            
  | 13:07 | 
  <hashar> | 
  1.32.0-wmf.20 is still syncing for testwiki due to l10ncache generation | 
  [production] | 
            
  | 13:06 | 
  <elukey> | 
  restart memcached on mc1035 with -v option - T203429 | 
  [production] | 
            
  | 12:57 | 
  <moritzm> | 
  reimaging backup2001 | 
  [production] | 
            
  | 12:43 | 
  <hashar@deploy1001> | 
  Started scap: testwiki to php-1.32.0-wmf.20 and rebuild l10n cache - T191066 | 
  [production] | 
            
  | 12:42 | 
  <hashar@deploy1001> | 
  Pruned MediaWiki: 1.32.0-wmf.18 [keeping static files] (duration: 06m 58s) | 
  [production] | 
            
  | 12:35 | 
  <hashar> | 
  scap clean 1.32.0-wmf.18 | T191066 | 
  [production] | 
            
  | 12:32 | 
  <hashar> | 
  Applied security patches for 1.32.0-wmf.20 | T191066 | 
  [production] | 
            
  | 12:24 | 
  <hashar> | 
  scap prep 1.32.0-wmf.20 # T191066 | 
  [production] | 
            
  | 12:07 | 
  <marostegui> | 
  Deploy schema change on db1123 | 
  [production] | 
            
  | 12:07 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1123 (duration: 00m 48s) | 
  [production] | 
            
  | 11:58 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1090:3317 (duration: 00m 48s) | 
  [production] | 
            
  | 11:50 | 
  <hashar> | 
  Cutting branch 1.32.0-wmf.20 | T191066 | 
  [production] | 
            
  | 11:46 | 
  <hashar> | 
  Preparing train deploy 1.32.0-wmf.20 | T191066 | 
  [production] | 
            
  | 11:35 | 
  <hashar> | 
  European SWAT completed | 
  [production] | 
            
  | 11:34 | 
  <hashar@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: Wikidata: Use new item ID formatter for Q1-Q100000 - T201835 (duration: 00m 49s) | 
  [production] | 
            
  | 11:19 | 
  <hashar@deploy1001> | 
  Synchronized php-1.32.0-wmf.19/extensions/DismissableSiteNotice: Revert "Use session storage instead of cookies for site notices" - T199274 (duration: 00m 50s) | 
  [production] | 
            
  | 11:11 | 
  <jynus> | 
  stopping replication and running partitioning on logging on db1085:3311 T189107 | 
  [production] | 
            
  | 11:10 | 
  <hashar@deploy1001> | 
  Synchronized static/images/project-logos: Update logos for the Russian Wikisource - T203343 (duration: 00m 49s) | 
  [production] | 
            
  | 11:06 | 
  <hashar@deploy1001> | 
  Synchronized wmf-config/throttle.php: Two throttle rules for SMEX editathon - T203392 (duration: 00m 51s) | 
  [production] | 
            
  | 11:05 | 
  <hashar@deploy1001> | 
  sync-file aborted: (no justification provided) (duration: 00m 01s) | 
  [production] | 
            
  | 10:29 | 
  <marostegui> | 
  Deploy schema change on dbstore1002:s3 | 
  [production] | 
            
  | 10:23 | 
  <marostegui> | 
  Deploy schema change on s3 codfw masters (this will generate lag on s3 codfw) | 
  [production] | 
            
  | 10:17 | 
  <elukey> | 
  restart ircecho on einstenium to force it re-join #wikimedia-analytics | 
  [production] | 
            
  | 09:57 | 
  <_joe_> | 
  restarted pdfrender on scb1001 | 
  [production] | 
            
  | 09:40 | 
  <jynus> | 
  deploying latest event schedulers to all core db hosts | 
  [production] | 
            
  | 09:32 | 
  <elukey> | 
  restart ircecho as attempt to see if icinga-vm re-joins the analytics chan | 
  [production] | 
            
  | 09:24 | 
  <jynus> | 
  stop, upgrade and running analyze on db1114 | 
  [production] | 
            
  | 09:23 | 
  <marostegui> | 
  Deploy schema change on db1090:3317 | 
  [production] | 
            
  | 09:23 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1090:3317 (duration: 00m 49s) | 
  [production] | 
            
  | 09:12 | 
  <akosiaris> | 
  depool pdfrender in eqiad, hopefully codfw will be better equipped to handle the load | 
  [production] | 
            
  | 09:10 | 
  <volans> | 
  restarted pdfrender on scb1003 | 
  [production] | 
            
  | 09:07 | 
  <volans> | 
  restarted pdfrender on scb1002 | 
  [production] | 
            
  | 09:06 | 
  <volans> | 
  restarted pdfrender on scb1004 | 
  [production] | 
            
  | 08:57 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1094, depool db1114 (duration: 00m 50s) | 
  [production] | 
            
  | 08:28 | 
  <gehel> | 
  starting rolling restart of elasticsearch / cirrus / eqiad for various updates and data directory migration - T198351 | 
  [production] | 
            
  | 08:04 | 
  <moritzm> | 
  upgrading mwmaint* to wikidiff 1.7.3 | 
  [production] | 
            
  | 07:52 | 
  <gehel> | 
  rolling restart of elasticsearch / cirrus / codfw for various updates and data directory migration completed - T198351 | 
  [production] | 
            
  | 07:44 | 
  <moritzm> | 
  rearmed keyholder on deploy2001 | 
  [production] | 
            
  | 07:10 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1094 (duration: 00m 52s) | 
  [production] | 
            
  | 07:05 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1079 (duration: 00m 54s) | 
  [production] | 
            
  | 07:00 | 
  <moritzm> | 
  upgrading remaining appservers/API servers to wikidiff 1.7.3 | 
  [production] |