| 
      
        2018-11-02
      
      §
     | 
  
    
  | 17:04 | 
  <thcipriani> | 
  rollback group2 wikis to 1.33.0-wmf.1 on mwdebug100{1,2} | 
  [production] | 
            
  | 16:54 | 
  <thcipriani> | 
  deploying 1.33.0-wmf.2 to group2 wikis on mwdebug1002 | 
  [production] | 
            
  | 16:43 | 
  <_joe_> | 
  live-hacking removal of time limit on mwdebug1001 | 
  [production] | 
            
  | 16:32 | 
  <thcipriani> | 
  deploying 1.33.0-wmf.2 to group2 wikis on mwdebug1001 | 
  [production] | 
            
  | 15:12 | 
  <jynus> | 
  restarting replication @ db2074 after db2094:s3 table fix T208565 | 
  [production] | 
            
  | 15:00 | 
  <jynus> | 
  stopping replication on db2074 to fix db2094:s3 T208565 | 
  [production] | 
            
  | 14:01 | 
  <vgutierrez> | 
  reimaging eeden.wikimedia.org as jessie test system - T208583 | 
  [production] | 
            
  | 11:43 | 
  <jynus> | 
  ignoring cawikimedia.archive replication on db2094:s3 until a reimport happens T208565 | 
  [production] | 
            
  | 11:29 | 
  <jijiki> | 
  Rebooting mw2244 (spare system) for maintenance | 
  [production] | 
            
  | 10:52 | 
  <ema> | 
  restart varnish-be on cp3032 T208574 | 
  [production] | 
            
  | 08:19 | 
  <jynus> | 
  performing alter table on dbstore2002 s3 and reducing consistency to improve recovery time T208462 T204006 | 
  [production] | 
            
  | 08:01 | 
  <jynus> | 
  reducing consitency on db2050 to improve recovery time T208462 | 
  [production] | 
            
  | 07:59 | 
  <jynus> | 
  performing alter table on db2050 T208462 T204006 | 
  [production] | 
            
  | 07:38 | 
  <godog> | 
  reformat ms-be1043 xfs filesystems - T199198 | 
  [production] | 
            
  | 07:38 | 
  <jynus> | 
  reducing consistency temporarily (flush, binlog sync) at db2040 to prevent lagging | 
  [production] | 
            
  | 07:26 | 
  <jynus> | 
  reducing consistency temporarily (flush, binlog sync) at db2035 to prevent lagging | 
  [production] | 
            
  
    | 
      
        2018-11-01
      
      §
     | 
  
    
  | 23:01 | 
  <shdubsh> | 
  restart hhvm on mw1261 | 
  [production] | 
            
  | 22:29 | 
  <ejegg> | 
  restarted fundraising queue consumer jobs | 
  [production] | 
            
  | 22:21 | 
  <ejegg> | 
  updated fundraising CiviCRM from 65130ef3dd to 042eeaeca9 | 
  [production] | 
            
  | 22:18 | 
  <ejegg> | 
  turned off fundraising queue jobs for civi update | 
  [production] | 
            
  | 22:12 | 
  <_joe_> | 
  rolling restart of hhvm on appservers and api in eqiad | 
  [production] | 
            
  | 22:09 | 
  <shdubsh> | 
  cumin -b 2 -s 30 "O:mediawiki::appserver and *.eqiad.wmnet" "restart-hhvm" | 
  [production] | 
            
  | 22:05 | 
  <_joe_> | 
  restarting hhvm on mw1238,1240 | 
  [production] | 
            
  | 22:02 | 
  <_joe_> | 
  restart hhvm on mw1244 | 
  [production] | 
            
  | 21:52 | 
  <shdubsh> | 
  restart hhvm on mw1247 | 
  [production] | 
            
  | 21:49 | 
  <_joe_> | 
  depooling mw1238 for debugging | 
  [production] | 
            
  | 21:09 | 
  <thcipriani@deploy1001> | 
  rebuilt and synchronized wikiversions files: group2 back to 1.33.0-wmf.1 | 
  [production] | 
            
  | 20:55 | 
  <hoo> | 
  Restarted hhvm on mwdebug2002 | 
  [production] | 
            
  | 20:42 | 
  <thcipriani@deploy1001> | 
  rebuilt and synchronized wikiversions files: all wikis to 1.33.0-wmf.2 | 
  [production] | 
            
  | 19:40 | 
  <hoo> | 
  Ran "UPDATE wb_changes_dispatch SET chd_seen = '775203911' WHERE chd_site LIKE '%wikt%' AND chd_seen < '775180000';" on wikidata master (dispatching for wiktionaries) | 
  [production] | 
            
  | 19:00 | 
  <hoo@deploy1001> | 
  Synchronized php-1.33.0-wmf.1/includes/export/WikiExporter.php: Fix for missing end tag </page> on some exports (T207974) (duration: 01m 01s) | 
  [production] | 
            
  | 18:38 | 
  <hoo@deploy1001> | 
  Synchronized php-1.33.0-wmf.2/includes/export/WikiExporter.php:  Fix for missing end tag </page> on some exports (T207974) (duration: 00m 55s) | 
  [production] | 
            
  | 18:25 | 
  <jijiki> | 
  Enabling puppet on mw servers (T206923) | 
  [production] | 
            
  | 18:19 | 
  <hoo@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: Remove now redundant Wikidata config for wiktionary (T208317) (duration: 00m 54s) | 
  [production] | 
            
  | 18:12 | 
  <hoo@deploy1001> | 
  Synchronized dblists/wikidataclient.dblist: Add all wiktionaries to wikidataclient.dblist, sort list (T208317) (duration: 00m 57s) | 
  [production] | 
            
  | 18:02 | 
  <gehel> | 
  restart nginx on relforge100* | 
  [production] | 
            
  | 17:57 | 
  <jijiki> | 
  Disabling puppet on mw servers (T206923) | 
  [production] | 
            
  | 16:07 | 
  <anomie@mwmaint1002> | 
  Running migrateComments.php on section 4 wikis for T166733 | 
  [production] | 
            
  | 13:46 | 
  <anomie@mwmaint1002> | 
  Running migrateComments.php on remaining section 3 wikis for T166733 | 
  [production] | 
            
  | 13:37 | 
  <anomie@mwmaint1002> | 
  Running migrateComments.php on section 7 wikis for T166733 | 
  [production] | 
            
  | 13:37 | 
  <anomie@mwmaint1002> | 
  Running migrateComments.php on wikitech for T166733 | 
  [production] | 
            
  | 13:37 | 
  <anomie@mwmaint1002> | 
  Running migrateImageCommentTemp.php on wikitech for T188132 | 
  [production] | 
            
  | 13:37 | 
  <anomie@mwmaint1002> | 
  Running migrateComments.php on section 6 wikis for T166733 | 
  [production] | 
            
  | 13:37 | 
  <anomie@mwmaint1002> | 
  Running migrateComments.php on section 8 wikis for T166733 | 
  [production] | 
            
  | 13:37 | 
  <anomie@mwmaint1002> | 
  Running migrateImageCommentTemp.php on section 8 wikis for T188132 | 
  [production] | 
            
  | 13:37 | 
  <anomie@mwmaint1002> | 
  Running migrateImageCommentTemp.php on section 7 wikis for T188132 | 
  [production] | 
            
  | 13:37 | 
  <anomie@mwmaint1002> | 
  Running migrateImageCommentTemp.php on section 6 wikis for T188132 | 
  [production] | 
            
  | 13:36 | 
  <anomie@mwmaint1002> | 
  Running migrateComments.php on section 5 wikis for T166733 | 
  [production] | 
            
  | 13:36 | 
  <anomie@mwmaint1002> | 
  Running migrateComments.php on section 1 wikis for T166733 | 
  [production] | 
            
  | 13:36 | 
  <anomie@mwmaint1002> | 
  Running migrateComments.php on section 2 wikis for T166733 | 
  [production] |