| 
      
        2017-06-07
      
      §
     | 
  
    
  | 22:18 | 
  <bblack> | 
  puppet node clean+deactivate for cp3003 | 
  [production] | 
            
  | 22:15 | 
  <bblack> | 
  lvs4002 - restarting pybal to remove old maps table entries | 
  [production] | 
            
  | 22:14 | 
  <bblack> | 
  lvs3002 - restarting pybal to remove old maps table entries | 
  [production] | 
            
  | 22:13 | 
  <bblack> | 
  lvs2002 - restarting pybal to remove old maps table entries | 
  [production] | 
            
  | 22:13 | 
  <bblack> | 
  lvs1002 - restarting pybal to remove old maps table entries | 
  [production] | 
            
  | 22:12 | 
  <bblack> | 
  lvs4004 - restarting pybal to remove old maps table entries | 
  [production] | 
            
  | 22:11 | 
  <bblack> | 
  lvs3004 - restarting pybal to remove old maps table entries | 
  [production] | 
            
  | 22:09 | 
  <bblack> | 
  lvs2005 - restarting pybal to remove old maps table entries | 
  [production] | 
            
  | 22:07 | 
  <bblack> | 
  lvs1005 - restarting pybal to remove old maps table entries | 
  [production] | 
            
  | 21:32 | 
  <twentyafterfour@tin> | 
  rebuilt wikiversions.php and synchronized wikiversions files: group1 wikis to 1.30.0-wmf.2 | 
  [production] | 
            
  | 21:31 | 
  <twentyafterfour> | 
  rolling back to wmf.2 due to error spike and popups no longer working refs T166829 | 
  [production] | 
            
  | 21:25 | 
  <twentyafterfour@tin> | 
  rebuilt wikiversions.php and synchronized wikiversions files: group1 wikis to 1.30.0-wmf.4 | 
  [production] | 
            
  | 21:23 | 
  <twentyafterfour@tin> | 
  Synchronized php-1.30.0-wmf.4/: sync 3248a17f5529ab6d5cbabb75dd0e72a7589f8633 refs T167343 (duration: 07m 52s) | 
  [production] | 
            
  | 20:26 | 
  <twentyafterfour@tin> | 
  Synchronized php-1.30.0-wmf.4/extensions/MobileFrontend: Deploy 66ef9cbd7f3de2832154f97392c2418fb1cd56ec refs T167216 (duration: 00m 46s) | 
  [production] | 
            
  | 20:04 | 
  <twentyafterfour> | 
  Preparing to deploy the MediaWiki train for group1 wikis, 1.30.0-wmf.4 refs T166829 | 
  [production] | 
            
  | 18:22 | 
  <thcipriani@tin> | 
  Synchronized wmf-config: SWAT: [[gerrit:357236|Enable archive indexing on delete for select wikis]] T162302 (duration: 00m 47s) | 
  [production] | 
            
  | 18:14 | 
  <thcipriani@tin> | 
  Synchronized portals: SWAT: [[gerrit:357490|Updating portals stats]] T128546 (duration: 00m 44s) | 
  [production] | 
            
  | 18:13 | 
  <thcipriani@tin> | 
  Synchronized portals/prod/wikipedia.org/assets: SWAT: [[gerrit:357490|Updating portals stats]] T128546 (duration: 00m 44s) | 
  [production] | 
            
  | 17:14 | 
  <elukey> | 
  restart nutcracker on thumbor1002 (too many connections approaching the 1024 ulimit) | 
  [production] | 
            
  | 15:37 | 
  <akosiaris> | 
  disable puppet on puppetmaster1001, depool rhodium for tests | 
  [production] | 
            
  | 14:51 | 
  <filippo@puppetmaster1001> | 
  conftool action : set/pooled=no; selector: name=ms-fe2007.codfw.wmnet | 
  [production] | 
            
  | 14:48 | 
  <filippo@puppetmaster1001> | 
  conftool action : set/pooled=no; selector: name=ms-fe1007.codfw.wmnet | 
  [production] | 
            
  | 14:11 | 
  <dcausse> | 
  eu swat done | 
  [production] | 
            
  | 12:56 | 
  <aude@tin> | 
  Synchronized php-1.30.0-wmf.4/extensions/Wikidata: Fix parser function registration T167238 (duration: 02m 20s) | 
  [production] | 
            
  | 12:43 | 
  <marostegui> | 
  Drop table updates on s2 - T139342 | 
  [production] | 
            
  | 12:40 | 
  <aude@tin> | 
  Synchronized wmf-config/InitialiseSettings-labs.php: Enable Wikibase Client on beta wiktionary sites T158323 (duration: 00m 43s) | 
  [production] | 
            
  | 12:40 | 
  <elukey> | 
  upgrade zookeeper packages on conf2002 to 3.4.5+dfsg-2+deb8u2 | 
  [production] | 
            
  | 12:32 | 
  <bblack> | 
  cp1072, cp1063 restarting varnish backend for mailbox lag | 
  [production] | 
            
  | 12:26 | 
  <aude@tin> | 
  Synchronized wmf-config/Wikibase.php: Site links for non-main namespace wiktionary pages T158323 (duration: 00m 43s) | 
  [production] | 
            
  | 12:19 | 
  <aude@tin> | 
  Synchronized wmf-config/Wikibase-labs.php: Site links for non-main namespace wiktionary pages (duration: 00m 44s) | 
  [production] | 
            
  | 11:08 | 
  <gehel> | 
  restarting cron on logstash cluster | 
  [production] | 
            
  | 10:29 | 
  <moritzm> | 
  installing tiff regression security update on trusty | 
  [production] | 
            
  | 10:26 | 
  <ema> | 
  upgrade lvs1*/lvs2* to jessie 8.8 point release T164703 | 
  [production] | 
            
  | 09:49 | 
  <ema> | 
  upgrade lvs[3001-3004] to jessie 8.8 point release T164703 | 
  [production] | 
            
  | 09:28 | 
  <gehel> | 
  upgrading kibana to v5.3.3 on logstash cluster - T167266 | 
  [production] | 
            
  | 09:15 | 
  <ema> | 
  upgrade lvs4001-4004 to jessie 8.8 point release T164703 | 
  [production] | 
            
  | 08:58 | 
  <marostegui> | 
  Deploy alter table on s2 - db1076 - T166205 | 
  [production] | 
            
  | 08:58 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1076 - T166205 (duration: 00m 43s) | 
  [production] | 
            
  | 08:50 | 
  <marostegui> | 
  Deploy alter table s4 - db1056 - T166206 | 
  [production] | 
            
  | 08:34 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1056 - T166206 (duration: 00m 43s) | 
  [production] | 
            
  | 08:02 | 
  <marostegui> | 
  Run redact_sanitarium on db1095 for dewiki - T153743 | 
  [production] | 
            
  | 07:22 | 
  <marostegui> | 
  Deploy alter table on db1047 enwiki.revision - T162807 | 
  [production] | 
            
  | 06:49 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1056 - T166206 (duration: 00m 44s) | 
  [production] | 
            
  | 05:35 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1053, depool db1056 - T166206 (duration: 01m 03s) | 
  [production] | 
            
  | 03:11 | 
  <l10nupdate@tin> | 
  ResourceLoader cache refresh completed at Wed Jun  7 03:11:40 UTC 2017 (duration 6m 54s) | 
  [production] | 
            
  | 03:04 | 
  <l10nupdate@tin> | 
  scap sync-l10n completed (1.30.0-wmf.4) (duration: 14m 29s) | 
  [production] | 
            
  | 02:30 | 
  <l10nupdate@tin> | 
  scap sync-l10n completed (1.30.0-wmf.2) (duration: 07m 57s) | 
  [production] | 
            
  | 00:21 | 
  <RainbowSprinkles> | 
  gerrit: rolled back to 2.13.4-13-gc0c5cc4742 from 2.13.8. T152640 rearing its ugly head again (login issues) | 
  [production] |