| 2017-05-24
      
      § | 
    
  | 08:48 | <joal@tin> | Started deploy [analytics/refinery@9377d9c]: Deploying to fix yesterday's deploy bugs | [production] | 
            
  | 07:28 | <marostegui@tin> | Synchronized wmf-config/db-eqiad.php: Repool db1094 - T164530 (duration: 00m 41s) | [production] | 
            
  | 07:17 | <marostegui@tin> | Synchronized wmf-config/db-eqiad.php: Repool db1086, depool db1094 - T164530 (duration: 00m 41s) | [production] | 
            
  | 07:03 | <marostegui@tin> | Synchronized wmf-config/db-eqiad.php: Repool db1079, depool db1086 - T164530 (duration: 00m 42s) | [production] | 
            
  | 06:56 | <marostegui@tin> | Synchronized wmf-config/db-eqiad.php: Depool db1079 - T164530 (duration: 00m 54s) | [production] | 
            
  | 06:34 | <marostegui> | Deploy alter table on s2.fawiki directly on codfw master (db2029) after running the clean up duplicates script - https://phabricator.wikimedia.org/T164530 | [production] | 
            
  | 06:04 | <marostegui> | Run pt-table-checksum on s7.frwiktionary - https://phabricator.wikimedia.org/T163190 | [production] | 
            
  | 06:02 | <marostegui> | Deploy alter table on s2  db1047 - https://phabricator.wikimedia.org/T162611 | [production] | 
            
  | 03:04 | <l10nupdate@tin> | ResourceLoader cache refresh completed at Wed May 24 03:04:03 UTC 2017 (duration 6m 45s) | [production] | 
            
  | 02:57 | <l10nupdate@tin> | scap sync-l10n completed (1.30.0-wmf.2) (duration: 13m 38s) | [production] | 
            
  | 02:24 | <l10nupdate@tin> | scap sync-l10n completed (1.30.0-wmf.1) (duration: 07m 30s) | [production] | 
            
  
    | 2017-05-23
      
      § | 
    
  | 21:00 | <mepps> | deployed payment wiki 0c06f8e6c998dccaa9857476dd71c16103e9b9e2 | [production] | 
            
  | 20:24 | <bblack> | enable BBR for all caches - T147569 | [production] | 
            
  | 20:20 | <bblack> | enable BBR for all caches @ codfw - T147569 | [production] | 
            
  | 20:10 | <bblack> | enable BBR for all caches @ ulsfo - T147569 | [production] | 
            
  | 20:06 | <bblack> | disabling puppet on all caches for BBR deploy control | [production] | 
            
  | 19:52 | <thcipriani@tin> | rebuilt wikiversions.php and synchronized wikiversions files: group0 to 1.30.0-wmf.2 | [production] | 
            
  | 19:34 | <thcipriani@tin> | Finished scap: testwiki to php-1.30.0-wmf.2 and rebuild l10n cache (duration: 27m 52s) | [production] | 
            
  | 19:07 | <bblack> | resetting cp1074 queues again: "fq flow_limit 200 buckets 10240" | [production] | 
            
  | 19:06 | <thcipriani@tin> | Started scap: testwiki to php-1.30.0-wmf.2 and rebuild l10n cache | [production] | 
            
  | 18:43 | <bblack> | resetting cp1074 queues again: "fq flow_limit 200 buckets 4096" | [production] | 
            
  | 17:40 | <bblack> | fq on cp1074 reset to flow_limit 200 (resets counters) | [production] | 
            
  | 17:24 | <ladsgroup@tin> | Finished deploy [ores/deploy@4874809]: Trying again with deploying ores (duration: 21m 30s) | [production] | 
            
  | 17:09 | <thcipriani> | starting branch cut for 1.30.0-wmf.2 T163512 | [production] | 
            
  | 17:03 | <ladsgroup@tin> | Started deploy [ores/deploy@4874809]: Trying again with deploying ores | [production] | 
            
  | 16:50 | <volans> | upgrading facter on mw[2250-2259] as a test batch | [production] | 
            
  | 16:49 | <bblack> | BBR: enabling bbr on cp1074 - T147569 | [production] | 
            
  | 16:43 | <bblack> | BBR: enabling mq+fq on cp1074 - T147569 | [production] | 
            
  | 16:26 | <bblack> | puppet re-enables on caches | [production] | 
            
  | 16:24 | <demon@tin> | Synchronized README: testing (duration: 00m 38s) | [production] | 
            
  | 16:17 | <bblack> | disabled puppet on all cp* for RPS-related deployments (just in case!) | [production] | 
            
  | 16:16 | <bblack> | disabled puppet on all lvs* for RPS-related deployments | [production] | 
            
  | 16:15 | <ema> | cp1074: enable prometheus node_exporter qdisc collector T147569 | [production] | 
            
  | 15:50 | <marostegui> | Stop replication on dbstore1002 s7 thread for maintenance - T163190 | [production] | 
            
  | 15:23 | <volans> | re-enabled raid_handler and puppet on tegmen | [production] | 
            
  | 15:02 | <otto@tin> | Finished deploy [eventlogging/analytics@UNKNOWN]: (no justification provided) (duration: 00m 02s) | [production] | 
            
  | 15:01 | <otto@tin> | Started deploy [eventlogging/analytics@UNKNOWN]: (no justification provided) | [production] | 
            
  | 14:56 | <otto@tin> | Finished deploy [eventlogging/analytics@25f8096]: (no justification provided) (duration: 00m 04s) | [production] | 
            
  | 14:56 | <otto@tin> | Started deploy [eventlogging/analytics@25f8096]: (no justification provided) | [production] | 
            
  | 14:42 | <volans> | temporarily disabled raid_handler and puppet on tegmen | [production] | 
            
  | 14:25 | <jynus> | deploying new check_raid monitoring write policy for megacli T166108 | [production] | 
            
  | 14:21 | <Dereckson> | EU SWAT done | [production] | 
            
  | 14:21 | <dereckson@tin> | Synchronized wmf-config/InitialiseSettings.php: Enable NewUserMessage on dty.wikipedia (T166121) (duration: 00m 38s) | [production] | 
            
  | 14:09 | <XioNoX> | re-enabling BGP session to Init7 - T165288 | [production] | 
            
  | 14:03 | <moritzm> | installing nutcracker update in codfw (T163795) | [production] | 
            
  | 13:37 | <marostegui> | Run CleanDuplicateScores script to clean up possible duplicates on fawiki before starting to create the UNIQUE keys - https://phabricator.wikimedia.org/T164530 | [production] | 
            
  | 13:23 | <dereckson@tin> | Synchronized wmf-config/InitialiseSettings.php: Add *.esa.int to CopyUploadsDomains (T164643) (duration: 00m 39s) | [production] | 
            
  | 12:47 | <elukey@tin> | Finished deploy [analytics/refinery@679aeea]: Updated stat1002 with the last refinery deployment (duration: 00m 42s) | [production] | 
            
  | 12:46 | <elukey@tin> | Started deploy [analytics/refinery@679aeea]: Updated stat1002 with the last refinery deployment | [production] | 
            
  | 12:46 | <elukey@tin> | Finished deploy [analytics/refinery@679aeea]: (no justification provided) (duration: 00m 01s) | [production] |