| 
      
        2017-01-25
      
      §
     | 
  
    
  | 13:55 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1054 - T156225 (duration: 00m 50s) | 
  [production] | 
            
  | 13:52 | 
  <moritzm> | 
  upgrading openjdk-8 on maps-test* | 
  [production] | 
            
  | 13:11 | 
  <gehel> | 
  pooling new elasticsearch nodes on codfw - T154251 | 
  [production] | 
            
  | 12:38 | 
  <moritzm> | 
  installing libxml security updates | 
  [production] | 
            
  | 12:04 | 
  <Dereckson> | 
  Refresh site statistics on simple. (T156247) | 
  [production] | 
            
  | 11:01 | 
  <moritzm> | 
  upgrading restbase staging cluster to new openjdk (also piggyback reboot to latest 4.4 kernel) | 
  [production] | 
            
  | 10:49 | 
  <moritzm> | 
  uploaded openjdk-8 u121 to apt.wikimedia.org | 
  [production] | 
            
  | 10:28 | 
  <moritzm> | 
  uploaded ca-certificates-java 20161107~bpo8+1 to apt.wikimedia.org | 
  [production] | 
            
  | 10:11 | 
  <ema> | 
  repooled codfw | 
  [production] | 
            
  | 09:25 | 
  <elukey> | 
  updating puppet-compiler facts | 
  [production] | 
            
  | 08:53 | 
  <ema> | 
  upgrade cp3040 to jessie 8.7 and reboot into kernel 4.4.2-3+wmf8 T155401 | 
  [production] | 
            
  | 08:15 | 
  <ema> | 
  upgrade cp3034 to jessie 8.7 and reboot into kernel 4.4.2-3+wmf8 T155401 | 
  [production] | 
            
  | 08:15 | 
  <ema> | 
  upgrade cp3034 to jessie 8.7 and reboot into kernel 4.4.2-3+wmf8 | 
  [production] | 
            
  | 08:02 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-codfw.php: Repool db2054 - T153300 (duration: 00m 51s) | 
  [production] | 
            
  | 07:48 | 
  <_joe_> | 
  restarting pybal on lvs1003 | 
  [production] | 
            
  | 07:28 | 
  <elukey> | 
  upgrading aqs100[56] to node6 | 
  [production] | 
            
  | 06:54 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1066 - T156005 (duration: 00m 42s) | 
  [production] | 
            
  | 05:37 | 
  <mobrovac> | 
  zotero restarting zotero, taking 95% of mem ... | 
  [production] | 
            
  | 02:55 | 
  <l10nupdate@tin> | 
  ResourceLoader cache refresh completed at Wed Jan 25 02:55:57 UTC 2017 (duration 5m 37s) | 
  [production] | 
            
  | 02:50 | 
  <l10nupdate@tin> | 
  scap sync-l10n completed (1.29.0-wmf.9) (duration: 12m 53s) | 
  [production] | 
            
  | 02:20 | 
  <l10nupdate@tin> | 
  scap sync-l10n completed (1.29.0-wmf.8) (duration: 06m 25s) | 
  [production] | 
            
  | 00:59 | 
  <jynus@tin> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1052 after maintenance (duration: 00m 40s) | 
  [production] | 
            
  | 00:37 | 
  <mutante> | 
  planet2001 - re-add new salt key, fix minion | 
  [production] | 
            
  | 00:11 | 
  <dereckson@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: Amend import sources for en.wikisource (T155922) (duration: 00m 47s) | 
  [production] | 
            
  | 00:09 | 
  <mutante> | 
  analytics1015,analytics1026 - revoked puppet cert, removing from puppet, shutting down (T147313) | 
  [production] | 
            
  
    | 
      
        2017-01-24
      
      §
     | 
  
    
  | 23:50 | 
  <mutante> | 
  carbon - stopping puppet, stopping atftpd | 
  [production] | 
            
  | 23:49 | 
  <mutante> | 
  carbon stopping DHCP | 
  [production] | 
            
  | 23:49 | 
  <mutante> | 
  analytics1015 (unused spare system) - use for test OS install | 
  [production] | 
            
  | 23:26 | 
  <jynus> | 
  restarting db1052 for kernel upgrade | 
  [production] | 
            
  | 23:09 | 
  <ebernhardson@tin> | 
  Synchronized php-1.29.0-wmf.9/includes/specials/SpecialSearch.php: Update special:search security patc h to not fatal (duration: 00m 44s) | 
  [production] | 
            
  | 23:04 | 
  <jynus> | 
  reimage db1066 | 
  [production] | 
            
  | 22:41 | 
  <jynus@tin> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1066 for reimage (duration: 00m 55s) | 
  [production] | 
            
  | 22:22 | 
  <Pchelolo> | 
  update RESTBase to 69065e2 | 
  [production] | 
            
  | 22:19 | 
  <Pchelolo> | 
  update RESTBase to 69065e2: canary on restbase1007 | 
  [production] | 
            
  | 22:13 | 
  <Pchelolo> | 
  update RESTBase to 69065e2: staging | 
  [production] | 
            
  | 21:55 | 
  <jynus@tin> | 
  Synchronized wmf-config/db-eqiad.php: repool db1065 as dump/vslow & clean up s1 comments (duration: 00m 43s) | 
  [production] | 
            
  | 21:43 | 
  <twentyafterfour> | 
  Finished group0 to wmf/1.29.0-wmf.9 (refs T15525) Changelog: https://www.mediawiki.org/wiki/MediaWiki_1.29/wmf.9/Changelog | 
  [production] | 
            
  | 21:34 | 
  <twentyafterfour@tin> | 
  rebuilt wikiversions.php and synchronized wikiversions files: group0 wikis to 1.29.0-wmf.9 refs T155525 | 
  [production] | 
            
  | 21:31 | 
  <demon@tin> | 
  Synchronized docroot: Drop labs docroot, unused in prod (duration: 00m 44s) | 
  [production] | 
            
  | 21:22 | 
  <twentyafterfour@tin> | 
  Finished scap: test wikis to 1.29.0-wmf.9 refs T155525 (duration: 32m 37s) | 
  [production] | 
            
  | 20:49 | 
  <twentyafterfour@tin> | 
  Started scap: test wikis to 1.29.0-wmf.9 refs T155525 | 
  [production] | 
            
  | 20:49 | 
  <volans> | 
  disabled puppet on ruthenium to avoid the restart of parsoid-vd and parsoid-vd-client processes T156177 | 
  [production] | 
            
  | 20:09 | 
  <demon@tin> | 
  Synchronized docroot: Adding new wikimediafoundation.org docroot (duration: 01m 05s) | 
  [production] | 
            
  | 19:51 | 
  <volans> | 
  ruthenium: stopped parsoid-vd and parsoid-vd-client to avoid uncontrolled spawning of phantomjs childs | 
  [production] | 
            
  | 19:39 | 
  <volans> | 
  sudo service parsoid-vd stop on ruthenium | 
  [production] | 
            
  | 19:37 | 
  <twentyafterfour> | 
  branching 1.29.0-wmf.9 refs T154683 | 
  [production] | 
            
  | 19:35 | 
  <volans> | 
  killed 822 "/srv/visualdiff/node_modules/phantomjs/lib/phantom/bin/phantomjs" processes on ruthenium. RAM and swap full, host unresponsive | 
  [production] | 
            
  | 19:29 | 
  <jynus> | 
  change replication master of db1095 to db1065 | 
  [production] | 
            
  | 19:07 | 
  <jynus> | 
  change replication master of db1095 to db1052 | 
  [production] | 
            
  | 19:07 | 
  <demon@tin> | 
  Synchronized docroot/foundation/logos: rm some old junk logos (duration: 00m 42s) | 
  [production] |