| 
      
        2017-07-19
      
      §
     | 
  
    
  | 13:05 | 
  <hashar@tin> | 
  Synchronized wmf-config/throttle.php: Extend throttle rule - T170844 (duration: 00m 48s) | 
  [production] | 
            
  | 12:56 | 
  <moritzm> | 
  rebooting nescio (DNS recursor) for kernel update | 
  [production] | 
            
  | 12:55 | 
  <jmm@puppetmaster1001> | 
  conftool action : set/pooled=no; selector: nescio.wikimedia.org | 
  [production] | 
            
  | 12:49 | 
  <jmm@puppetmaster1001> | 
  conftool action : set/pooled=yes; selector: maerlant.wikimedia.org | 
  [production] | 
            
  | 12:40 | 
  <Reedy> | 
  running foreachwiki updateRestrictions.php T166184 | 
  [production] | 
            
  | 12:34 | 
  <moritzm> | 
  rebooting maerlant (DNS recursor) for kernel update | 
  [production] | 
            
  | 12:29 | 
  <jmm@puppetmaster1001> | 
  conftool action : set/pooled=no; selector: maerlant.wikimedia.org | 
  [production] | 
            
  | 12:10 | 
  <ema@neodymium> | 
  conftool action : set/pooled=yes; selector: name=acamar.wikimedia.org | 
  [production] | 
            
  | 12:09 | 
  <ema@neodymium> | 
  conftool action : set/pooled=no; selector: name=acamar.wikimedia.org | 
  [production] | 
            
  | 12:07 | 
  <ariel@tin> | 
  Finished deploy [dumps/dumps@f95292e]: fix api call bug, page range query min pages (duration: 00m 03s) | 
  [production] | 
            
  | 12:07 | 
  <ariel@tin> | 
  Started deploy [dumps/dumps@f95292e]: fix api call bug, page range query min pages | 
  [production] | 
            
  | 11:40 | 
  <kartik@tin> | 
  Finished deploy [cxserver/deploy@1029833]: Update cxserver to d28ad0c (duration: 03m 01s) | 
  [production] | 
            
  | 11:37 | 
  <moritzm> | 
  rebooting acamar (DNS recursor) for kernel update | 
  [production] | 
            
  | 11:37 | 
  <kartik@tin> | 
  Started deploy [cxserver/deploy@1029833]: Update cxserver to d28ad0c | 
  [production] | 
            
  | 10:58 | 
  <marostegui> | 
  Global rename of user Moros - T170941 | 
  [production] | 
            
  | 10:09 | 
  <ema> | 
  ulsfo cache_text/upload: upgrade to varnish 4.1.7-1wm1 and reboot for kernel updates | 
  [production] | 
            
  | 10:06 | 
  <marostegui> | 
  Deploy alter table on s1 - db1051 - T166204 | 
  [production] | 
            
  | 10:04 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1051 - T166204 (duration: 00m 47s) | 
  [production] | 
            
  | 10:01 | 
  <filippo@tin> | 
  Finished deploy [statsv/statsv@0a86be8]: (no justification provided) (duration: 00m 03s) | 
  [production] | 
            
  | 10:01 | 
  <filippo@tin> | 
  Started deploy [statsv/statsv@0a86be8]: (no justification provided) | 
  [production] | 
            
  | 09:58 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1065 - T166204 (duration: 00m 47s) | 
  [production] | 
            
  | 09:16 | 
  <ema> | 
  finish up codfw cache_text/upload varnish/kernel upgrades | 
  [production] | 
            
  | 09:05 | 
  <oblivian@neodymium> | 
  conftool action : set/pooled=false; selector: name=eqiad,dnsdisc=(restbase-async|citoid) | 
  [production] | 
            
  | 09:03 | 
  <XioNoX> | 
  codfw repooled in dns - T170380 | 
  [production] | 
            
  | 09:01 | 
  <hashar> | 
  restarting nodepool for upgrade 0.1.1-wmf7 -> 0.1.1-wmf8 | 
  [production] | 
            
  | 08:48 | 
  <moritzm> | 
  uploaded nodepool 0.1.1+wmf8 to apt.wikimedia.org | 
  [production] | 
            
  | 08:29 | 
  <XioNoX> | 
  asw-c-codfw back online - T170380 | 
  [production] | 
            
  | 08:28 | 
  <XioNoX> | 
  asw-c-codfw restarted 8min ago for switch upgrade - T170380 | 
  [production] | 
            
  | 08:09 | 
  <akosiaris> | 
  disable librenms crons on netmon1002 for a while | 
  [production] | 
            
  | 07:57 | 
  <marostegui> | 
  Drop migrateuser_medium from s7 - T170310 | 
  [production] | 
            
  | 07:49 | 
  <oblivian@neodymium> | 
  conftool action : set/pooled=true; selector: name=eqiad,dnsdisc=(restbase-async|citoid) | 
  [production] | 
            
  | 05:26 | 
  <_joe_> | 
  ran systemctl reset-failed on codfw jobrunners after the jobrunner process was activated by mistake running scap at 21.20 UTC yesterday | 
  [production] | 
            
  | 03:03 | 
  <l10nupdate@tin> | 
  LocalisationUpdate failed: git pull of extensions failed | 
  [production] | 
            
  | 01:27 | 
  <mutante> | 
  netmon1001 - stopping all the services, killing snmpwalk, disarming keyholder | 
  [production] | 
            
  | 00:35 | 
  <reedy@tin> | 
  Synchronized wmf-config/CommonSettings.php: Remove rcs1001 and rcs1002 from CommonSettings wgRCFeeds. Stops a load of logspam T170157 (duration: 00m 48s) | 
  [production] | 
            
  
    | 
      
        2017-07-18
      
      §
     | 
  
    
  | 23:53 | 
  <mutante> | 
  netmon1002 - copied Letsencrypt cert/key for librenms from netmon1001 for migration after netmon1002 has been reinstalled and now has RAID. (T159756) | 
  [production] | 
            
  | 23:40 | 
  <thcipriani@tin> | 
  Synchronized wmf-config/InterwikiSortOrders.php: SWAT: [[gerrit:365451|Add din to InterwikiSortOrders]] T168518 (duration: 00m 46s) | 
  [production] | 
            
  | 23:35 | 
  <thcipriani@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:365942|Add Welsh mobile logo (just changes 'k' to 'c']] PART II (duration: 00m 46s) | 
  [production] | 
            
  | 23:34 | 
  <thcipriani@tin> | 
  Synchronized static/images/mobile/copyright/wikipedia-wordmark-cy.svg: SWAT: [[gerrit:365942|Add Welsh mobile logo (just changes 'k' to 'c']] PART I (duration: 00m 47s) | 
  [production] | 
            
  | 23:27 | 
  <thcipriani@tin> | 
  Synchronized php-1.30.0-wmf.9/extensions/Thanks/extension.json: SWAT: [[gerrit:366168|Add missing jQueryMsg dependency for mobile diff view]] T170917 (duration: 00m 47s) | 
  [production] | 
            
  | 23:22 | 
  <thcipriani@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:360371|Enable OOjs UI EditPage buttons on all Wikipedias]] T162849 (duration: 00m 47s) | 
  [production] | 
            
  | 23:13 | 
  <thcipriani@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:365884|Enable CodeMirror on simplewiki for better testing and more exposure]] (duration: 00m 48s) | 
  [production] | 
            
  | 22:58 | 
  <thcipriani> | 
  restared jobrunner on mw1299.eqiad.wmnet mw1168.eqiad.wmnet mw1164.eqiad.wmnet mw1305.eqiad.wmnet mw1304.eqiad.wmnet mw1301.eqiad.wmnet mw1259.eqiad.wmnet mw1166.eqiad.wmnet mw1300.eqiad.wmnet | 
  [production] | 
            
  | 22:42 | 
  <krinkle@tin> | 
  Finished deploy [jobrunner/jobrunner@5f6099f]: (no justification provided) (duration: 08m 18s) | 
  [production] | 
            
  | 22:34 | 
  <krinkle@tin> | 
  Started deploy [jobrunner/jobrunner@5f6099f]: (no justification provided) | 
  [production] | 
            
  | 22:02 | 
  <krinkle@tin> | 
  Finished deploy [jobrunner/jobrunner@5f6099f]: (no justification provided) (duration: 07m 58s) | 
  [production] | 
            
  | 21:54 | 
  <krinkle@tin> | 
  Started deploy [jobrunner/jobrunner@5f6099f]: (no justification provided) | 
  [production] | 
            
  | 21:43 | 
  <Krinkle> | 
  Attempt to deploy mediawiki/services/jobrunner – https://gerrit.wikimedia.org/r/#/c/349364/ - failed. | 
  [production] | 
            
  | 19:56 | 
  <dzahn@neodymium> | 
  conftool action : set/pooled=yes; selector: name=mw2202.codfw.wmnet | 
  [production] | 
            
  | 19:48 | 
  <robh> | 
  starting wipe on cp400[1-4] per T169020 | 
  [production] |