| 2017-06-22
      
      § | 
    
  | 07:24 | <elukey> | reboot kafka1001 for kernel updates (eventbus eqiad) | [production] | 
            
  | 07:24 | <marostegui> | Deploy alter table s5 - db1026 - T166207 | [production] | 
            
  | 07:21 | <marostegui@tin> | Synchronized wmf-config/db-eqiad.php: Depool db1026 - T166207 (duration: 00m 44s) | [production] | 
            
  | 07:15 | <marostegui@tin> | Synchronized wmf-config/db-eqiad.php: Repool db1045 - T166207 (duration: 01m 03s) | [production] | 
            
  | 07:12 | <marostegui> | Deploy alter table s5 - dbstore1001 - T166207 | [production] | 
            
  | 06:53 | <moritzm> | rebooting mw1205-mw1208 for kernel update | [production] | 
            
  | 06:38 | <moritzm> | rebooting bast2001 for kernel update | [production] | 
            
  | 05:34 | <moritzm> | rebooting mw1238-mw1249 for kernel update | [production] | 
            
  | 05:02 | <moritzm> | rebooting ms-be2015-ms-be2020 for kernel update | [production] | 
            
  | 03:40 | <mutante> | regarding my last log message: this is just true for stretch! ah! | [production] | 
            
  | 03:35 | <mutante> | netmon1002 - installed psmisc to have 'killall' - will clean it up, but also suggest we add psmisc to base packages. it provides killall, fuser, pstree... | [production] | 
            
  | 02:49 | <l10nupdate@tin> | ResourceLoader cache refresh completed at Thu Jun 22 02:49:58 UTC 2017 (duration 6m 53s) | [production] | 
            
  | 02:43 | <l10nupdate@tin> | scap sync-l10n completed (1.30.0-wmf.6) (duration: 07m 23s) | [production] | 
            
  | 02:24 | <l10nupdate@tin> | scap sync-l10n completed (1.30.0-wmf.5) (duration: 08m 10s) | [production] | 
            
  | 00:15 | <twentyafterfour> | finished phabricator deployments | [production] | 
            
  | 00:13 | <twentyafterfour> | deploying https://phabricator.wikimedia.org/D687 | [production] | 
            
  
    | 2017-06-21
      
      § | 
    
  | 23:19 | <twentyafterfour@tin> | Synchronized static/images/project-logos/wikimania2017wiki.png: swat (duration: 00m 45s) | [production] | 
            
  | 23:10 | <twentyafterfour@tin> | Synchronized static/images/project-logos/wikimania2017wiki.png: swat (duration: 00m 45s) | [production] | 
            
  | 22:38 | <mutante> | new language din.wikipedia.org has been created in DNS - Dinka is a Nilotic dialect cluster spoken by the Dinka people, the major ethnic group of South Sudan. (T168518) - https://en.wikipedia.org/wiki/Dinka_language | [production] | 
            
  | 22:34 | <mutante> | DNS  - authdns-gen-zones -f /srv/authdns/git/templates /etc/gdnsd/zones && gdnsd checkconf && gdnsd reload-zones to trigger template recreation after edit to langs.tmpl | [production] | 
            
  | 22:31 | <chasemp> | remove manual 10.64.37.26 definition from eth1 on labstore1005 in /etc/network/interfaces | [production] | 
            
  | 22:27 | <chasemp> | reboot labstore1004 to reset network config from boot | [production] | 
            
  | 21:44 | <RainbowSprinkles> | cobalt: updated to 2.13.8-11-gde96955fb2 (T168360, T161206) | [production] | 
            
  | 21:40 | <RainbowSprinkles> | gerrit2001: updated to 2.13.8-11-gde96955fb2 (T168360, T161206) | [production] | 
            
  | 21:14 | <mutante> | apt.wm.org - reprepro copy stretch-wikimedia jessie-wikimedia gerrit - make gerrit available in stretch | [production] | 
            
  | 21:05 | <mutante> | apt.wm.org - reprepro, include gerrit_2.13.8+git1-wmf.6 for jessie-wikimedia | [production] | 
            
  | 21:02 | <mutante> | install1002 - rsynced gerrit packages from copper, closed firewall again, cleaned up rsyncd config from old unused things | [production] | 
            
  | 20:57 | <arlolra> | Updated Parsoid to 881ade32 (T127421, T167933, T167714) | [production] | 
            
  | 20:50 | <mutante> | install1002 - allow rsync from copper (build host) to /srv/wikimedia/incoming , temp for package upload | [production] | 
            
  | 20:49 | <arlolra@tin> | Finished deploy [parsoid/deploy@2c4c0de]: Updating Parsoid to 881ade32 (duration: 12m 02s) | [production] | 
            
  | 20:37 | <arlolra@tin> | Started deploy [parsoid/deploy@2c4c0de]: Updating Parsoid to 881ade32 | [production] | 
            
  | 20:35 | <mutante> | install1002 - removing rsyncd config fragments from carbon migration, running puppet | [production] | 
            
  | 20:25 | <twentyafterfour@tin> | rebuilt wikiversions.php and synchronized wikiversions files: group1 wikis to 1.30.0-wmf.6 | [production] | 
            
  | 20:15 | <bearND> | rolled back deploy since scap could not connect to scb1003 | [production] | 
            
  | 20:14 | <twentyafterfour@tin> | Synchronized php-1.30.0-wmf.6/includes/gallery/ImageGalleryBase.php: deploy https://gerrit.wikimedia.org/r/#/c/360695/ refs T168479 to unblock the train (duration: 00m 56s) | [production] | 
            
  | 20:13 | <bsitzmann@tin> | Finished deploy [mobileapps/deploy@7bfe571]: Update mobileapps to 21f771d (duration: 08m 43s) | [production] | 
            
  | 20:13 | <andrewbogott> | deleting the old IAD wikitech-static server so we stop paying rackspace for it | [production] | 
            
  | 20:11 | <ppchelko@tin> | Finished deploy [changeprop/deploy@63e6a7b]: Actually start black-listing and rate-limiting articles. T161710 (duration: 01m 16s) | [production] | 
            
  | 20:09 | <ppchelko@tin> | Started deploy [changeprop/deploy@63e6a7b]: Actually start black-listing and rate-limiting articles. T161710 | [production] | 
            
  | 20:04 | <bsitzmann@tin> | Started deploy [mobileapps/deploy@7bfe571]: Update mobileapps to 21f771d | [production] | 
            
  | 19:41 | <mutante> | copper: building gerrit_2.13.8+git1-wmf.6 for stretch (experimental) | [production] | 
            
  | 19:39 | <mutante> | copper: building gerrit_2.13.8+git1-wmf.6 for jessie | [production] | 
            
  | 19:30 | <twentyafterfour> | The train for wmf.6 (T167535) is currently blocked by T168479 | [production] | 
            
  | 19:13 | <madhuvishy> | Rebooting labstore1004 (secondary in drbd pair) | [production] | 
            
  | 18:55 | <andrewbogott> | rebooting labnet1001, which will cause a labs-wide network outage | [production] | 
            
  | 18:46 | <gehel> | restarting wdqs-updater on all wdqs servers | [production] | 
            
  | 18:38 | <krinkle@tin> | Synchronized static/images/: I737e6f9fce (duration: 00m 46s) | [production] | 
            
  | 18:20 | <gehel@tin> | Finished deploy [wdqs/wdqs@d67d4a4]: (no justification provided) (duration: 01m 50s) | [production] | 
            
  | 18:18 | <gehel@tin> | Started deploy [wdqs/wdqs@d67d4a4]: (no justification provided) | [production] | 
            
  | 18:17 | <gehel> | deploying wdqs to fix missing lib | [production] |