| 
      
        2019-01-23
      
      ยง
     | 
  
    
  | 20:21 | 
  <twentyafterfour> | 
  rolling back because error rate increased significantly after promoting | 
  [production] | 
            
  | 20:09 | 
  <twentyafterfour> | 
  twentyafterfour@deploy1001 rebuilt and synchronized wikiversions files: group0 wikis to 1.33.0-wmf.14  refs T206668 | 
  [production] | 
            
  | 20:07 | 
  <twentyafterfour@deploy1001> | 
  rebuilt and synchronized wikiversions files: group0 wikis to 1.33.0-wmf.14  refs T206668 | 
  [production] | 
            
  | 19:33 | 
  <moritzm> | 
  rebooting dubnium to pick up SSBD-enabled qemu | 
  [production] | 
            
  | 19:03 | 
  <moritzm> | 
  rebooting puppetdb2001 to pick up SSBD-enabled qemu | 
  [production] | 
            
  | 18:46 | 
  <jforrester@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: Config: Disable showing 'depicts' statements on Commons for now via I66d97031 (duration: 00m 52s) | 
  [production] | 
            
  | 18:44 | 
  <jforrester@deploy1001> | 
  Synchronized php-1.33.0-wmf.14/extensions/WikimediaEvents/includes/WikimediaEventsHooks.php: Hot-deploy Ief9c9155c to avoid auto-opting new accounts into PHP7 (duration: 00m 53s) | 
  [production] | 
            
  | 18:35 | 
  <anomie@deploy1001> | 
  Synchronized php-1.33.0-wmf.13/includes/page/WikiPage.php: Add even more temporary logging for T210739 (duration: 00m 54s) | 
  [production] | 
            
  | 18:26 | 
  <moritzm> | 
  rebooting mendelevium/ticket.wikimedia.org to pick up SSBD-enabled qemu | 
  [production] | 
            
  | 18:10 | 
  <sbisson@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:485212|Reapply Enable the Welcome survey on viwiki]] (duration: 00m 53s) | 
  [production] | 
            
  | 18:09 | 
  <sbisson@deploy1001> | 
  Synchronized php-1.33.0-wmf.13/extensions/GrowthExperiments/: SWAT: [[gerrit:485209|Help panel: ResourceLoaderHelpPanelModule handle help panel disabled]] (duration: 00m 54s) | 
  [production] | 
            
  | 18:03 | 
  <jbond@cumin1001> | 
  conftool action : set/pooled=yes; selector: name=dns5002.wikimedia.org | 
  [production] | 
            
  | 17:57 | 
  <jbond@cumin1001> | 
  conftool action : set/pooled=no; selector: name=dns5002.wikimedia.org | 
  [production] | 
            
  | 17:24 | 
  <dcausse@deploy1001> | 
  Finished deploy [search/mjolnir/deploy@a141ad3]: fix retry_on_conflict (duration: 04m 21s) | 
  [production] | 
            
  | 17:20 | 
  <dcausse@deploy1001> | 
  Started deploy [search/mjolnir/deploy@a141ad3]: fix retry_on_conflict | 
  [production] | 
            
  | 16:57 | 
  <jbond@cumin1001> | 
  conftool action : set/pooled=yes; selector: name=dns5001.wikimedia.org | 
  [production] | 
            
  | 16:53 | 
  <jbond@cumin1001> | 
  conftool action : set/pooled=no; selector: name=dns5001.wikimedia.org | 
  [production] | 
            
  | 16:50 | 
  <jbond@cumin1001> | 
  conftool action : set/pooled=yes; selector: name=dns4002.wikimedia.org | 
  [production] | 
            
  | 16:44 | 
  <jbond@cumin1001> | 
  conftool action : set/pooled=no; selector: name=dns4002.wikimedia.org | 
  [production] | 
            
  | 16:43 | 
  <jbond@cumin1001> | 
  conftool action : set/pooled=yes; selector: name=dns4001.wikimedia.org | 
  [production] | 
            
  | 16:36 | 
  <jbond@cumin1001> | 
  conftool action : set/pooled=no; selector: name=dns4001.wikimedia.org | 
  [production] | 
            
  | 16:31 | 
  <jbond@cumin1001> | 
  conftool action : set/pooled=yes; selector: name=dns2002.wikimedia.org | 
  [production] | 
            
  | 16:14 | 
  <jbond@cumin1001> | 
  conftool action : set/pooled=no; selector: name=dns2002.wikimedia.org | 
  [production] | 
            
  | 16:13 | 
  <jbond42> | 
  rolling restarts of PDNS recursors/ntpd in codfw/esams/ulsfi/eqsin to pick up openssl security update | 
  [production] | 
            
  | 16:02 | 
  <jbond42> | 
  restarting ntpd on dns2001 | 
  [production] | 
            
  | 16:00 | 
  <jbond@cumin1001> | 
  conftool action : set/pooled=yes; selector: name=dns2001.wikimedia.org | 
  [production] | 
            
  | 15:57 | 
  <jynus> | 
  adding dbstore1004:s2 to tendril | 
  [production] | 
            
  | 15:43 | 
  <jbond@cumin1001> | 
  conftool action : set/pooled=no; selector: name=dns2001.wikimedia.org | 
  [production] | 
            
  | 15:20 | 
  <marostegui> | 
  Truncate wmf_checksum table on dbstore1002 - T213670 | 
  [production] | 
            
  | 14:55 | 
  <marostegui> | 
  Compress InnoDB on a few tables on dbstore1002 to gain some extra space - T213670 | 
  [production] | 
            
  | 14:18 | 
  <marostegui> | 
  Convert tokudb tables into innodb on dbstore1002 - T213706 | 
  [production] | 
            
  | 13:47 | 
  <marostegui> | 
  Convert a bunch of Aria tables to InnoDB on dbstore1002 | 
  [production] | 
            
  | 13:38 | 
  <onimisionipe> | 
  repooling maps1002 | 
  [production] | 
            
  | 13:32 | 
  <gehel> | 
  restarting kartotherian on maps100[234] | 
  [production] | 
            
  | 13:30 | 
  <gehel> | 
  restarting kartotherian on maps1003 | 
  [production] | 
            
  | 13:27 | 
  <marostegui> | 
  Migrate some tokudb tables to innodb on dbstore1002 - T213706 | 
  [production] | 
            
  | 13:18 | 
  <gehel> | 
  running cumin 'P{O:cache::upload} and A:eqiad' 'run-puppet-agent' | 
  [production] | 
            
  | 13:10 | 
  <zeljkof> | 
  EU SWAT finished | 
  [production] | 
            
  | 12:36 | 
  <zfilipin@deploy1001> | 
  Synchronized php-1.33.0-wmf.14/extensions/AbuseFilter: SWAT: [[gerrit:486035|Re-fix the throttle script (T209565)]] (duration: 00m 55s) | 
  [production] | 
            
  | 12:32 | 
  <zfilipin@deploy1001> | 
  Synchronized php-1.33.0-wmf.13/extensions/AbuseFilter/: SWAT: [[gerrit:486034|Re-fix the throttle script (T209565)]] (duration: 00m 54s) | 
  [production] | 
            
  | 12:20 | 
  <ladsgroup@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:485704|Add new namespace abbreviation for Swedish (sv) (T214329)]] (duration: 00m 53s) | 
  [production] | 
            
  | 12:17 | 
  <ladsgroup@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:485847|Fix project talk namespace alias of Persian Wikipedia (T213733)]] (duration: 00m 53s) | 
  [production] | 
            
  | 12:09 | 
  <zfilipin@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:485151|Define ImportSources for nywiki]] (duration: 00m 54s) | 
  [production] | 
            
  | 11:44 | 
  <reedy@deploy1001> | 
  Synchronized wmf-config/CommonSettings.php: T214456 (duration: 00m 53s) | 
  [production] | 
            
  | 11:04 | 
  <arturo> | 
  T214299 reboot cloudnet2001-dev, cloudnet2002-dev and cloudnet1003 for new interface names | 
  [production] | 
            
  | 11:02 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1097:3315 T210713 (duration: 00m 52s) | 
  [production] | 
            
  | 10:52 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1097:3315 T210713 (duration: 00m 52s) | 
  [production] | 
            
  | 10:39 | 
  <arturo> | 
  updating puppet catalog compiler facts: `PUPPET_COMPILER=compiler1002.puppet-diffs.eqiad.wmflabs modules/puppet_compiler/files/compiler-update-facts` | 
  [production] | 
            
  | 10:37 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1096:3315 T210713 (duration: 00m 52s) | 
  [production] | 
            
  | 10:33 | 
  <Amir1> | 
  Deployed patch for T207814 on wmf.14 | 
  [production] |