| 2017-07-27
      
      § | 
    
  | 20:22 | <moritzm> | installing apache security updates on cobalt | [production] | 
            
  | 20:13 | <ppchelko@tin> | Finished deploy [restbase/deploy@cfb9c46]: Correctly escape the quotes in PDF names (duration: 07m 48s) | [production] | 
            
  | 20:05 | <ppchelko@tin> | Started deploy [restbase/deploy@cfb9c46]: Correctly escape the quotes in PDF names | [production] | 
            
  | 19:50 | <thcipriani@tin> | rebuilt wikiversions.php and synchronized wikiversions files: all wikis to 1.30.0-wmf.11 | [production] | 
            
  | 19:43 | <mutante> | switching https://releases.wikimedia.org backend from bromine to releases1001 - all files have been rsynced (T164030) | [production] | 
            
  | 19:37 | <robh> | cp4021 shutting down for relocation in rack, will put in maint mode for next 2 hours | [production] | 
            
  | 19:21 | <catrope@tin> | Synchronized wmf-config: T171556 (duration: 00m 47s) | [production] | 
            
  | 19:20 | <catrope@tin> | Synchronized dblists: T171556 (duration: 00m 46s) | [production] | 
            
  | 19:11 | <catrope@tin> | Synchronized wmf-config/InitialiseSettings.php: Enable namespace and tag filters in RCFilters on group0 and group1 (duration: 00m 46s) | [production] | 
            
  | 19:07 | <catrope@tin> | Synchronized wmf-config/InitialiseSettings.php: T171751 (duration: 00m 46s) | [production] | 
            
  | 19:06 | <catrope@tin> | Synchronized php-1.30.0-wmf.11/autoload.php: (no justification provided) (duration: 00m 45s) | [production] | 
            
  | 19:05 | <catrope@tin> | Synchronized php-1.30.0-wmf.11/includes/: T168501, take two (duration: 01m 27s) | [production] | 
            
  | 19:05 | <mutante> | added new misc::cache director "releases" for releases* servers, releases moving away from bromine (T164030) | [production] | 
            
  | 18:44 | <catrope@tin> | Synchronized php-1.30.0-wmf.11/includes/: Temporarily revert patch for  T168501 while I fix it (duration: 01m 27s) | [production] | 
            
  | 18:32 | <mutante> | labpuppetmaster1001 - restarted ferm twice, DNS lookup for AAAA worked, error gone on second time. then did same on labpuppetmaster1002 (T171880) | [production] | 
            
  | 18:29 | <catrope@tin> | Synchronized php-1.30.0-wmf.11/includes/: T168501 and T163380 (duration: 01m 31s) | [production] | 
            
  | 18:22 | <catrope@tin> | Synchronized wmf-config/InitialiseSettings.php: Enable WikidataPageBanner on euwiki (T171763) (duration: 00m 46s) | [production] | 
            
  | 18:15 | <catrope@tin> | Synchronized wmf-config/InitialiseSettings.php: Limit FeaturedFeed on dewiki to 7 days (T159664) (duration: 00m 47s) | [production] | 
            
  | 18:05 | <herron> | installed libmail-spf-perl on fermium to address spamassassin "module not installed: Mail::SPF ('require' failed)" error | [production] | 
            
  | 17:05 | <ejegg> | restarted recurring Ingenico charge job | [production] | 
            
  | 16:01 | <ejegg> | updated CiviCRM from e83c012581305012145eae45495e7e8ea6f4e249 to ceff739dcfeb5e5bcf40d880146cfb44eaf462ea | [production] | 
            
  | 15:44 | <jynus> | stopping mysql, upgrading and restarting labsdb1010 | [production] | 
            
  | 15:03 | <moritzm> | stopping jobrunner/jobchron on mw1260 to investigate a few failing ffmpeg2theora invocations (T145742) | [production] | 
            
  | 13:31 | <ema> | lvs1009, lvs1010: upgrade to pybal 1.13.10 (one-packet-scheduling) T104442 | [production] | 
            
  | 13:19 | <moritzm> | installing apache updates on mendelevium and terbium | [production] | 
            
  | 13:08 | <hashar@tin> | Synchronized wmf-config/InitialiseSettings.php: Enable mapframe for cswiki - T171588 (duration: 00m 47s) | [production] | 
            
  | 10:32 | <moritzm> | reimaging mw2246 to jessie (T145742) | [production] | 
            
  | 10:31 | <jynus> | shutting down and rebooting db2016 | [production] | 
            
  | 10:14 | <_joe_> | restarting puppetdb on nihal, T170740 | [production] | 
            
  | 10:06 | <jynus@tin> | Synchronized wmf-config/db-codfw.php: Promote db2048 as the new codfw-s1 master (duration: 00m 46s) | [production] | 
            
  | 10:05 | <jynus> | starting actual master failover s1-codfw db2016->db2048 | [production] | 
            
  | 09:48 | <ema> | pybal 1.13.10 (one-packet-scheduling) built and uploaded to apt.w.o T104442 | [production] | 
            
  | 09:23 | <jynus> | starting s1-codfw database topology changes | [production] | 
            
  | 09:02 | <godog> | copy python-conftool to stretch-wikimedia for scap dep | [production] | 
            
  | 08:54 | <jynus> | stopping mysql and restarting db2048 | [production] | 
            
  | 08:48 | <jynus@tin> | Synchronized wmf-config/db-codfw.php: Depool db2048 (duration: 00m 46s) | [production] | 
            
  | 08:28 | <jynus> | disable puppet on db2016 and db2048 to prepare for switchover | [production] | 
            
  | 08:21 | <godog> | upload scap 3.6.0-1 - T127762 | [production] | 
            
  | 08:12 | <moritzm> | installing apache security updates on graphite* | [production] | 
            
  | 07:37 | <moritzm> | upgrading apache on planet1001 | [production] | 
            
  | 07:02 | <moritzm> | installing spice secuerity updates on trusty hosts (jessie already fixed) | [production] | 
            
  | 03:29 | <ejegg> | disabled recurring Ingenico charges | [production] | 
            
  | 03:12 | <l10nupdate@tin> | ResourceLoader cache refresh completed at Thu Jul 27 03:12:30 UTC 2017 (duration 7m 16s) | [production] | 
            
  | 03:05 | <l10nupdate@tin> | scap sync-l10n completed (1.30.0-wmf.11) (duration: 06m 00s) | [production] | 
            
  | 02:39 | <l10nupdate@tin> | scap sync-l10n completed (1.30.0-wmf.10) (duration: 14m 21s) | [production] |