| 2017-02-13
      
      § | 
    
  | 15:06 | <addshore@tin> | Synchronized wmf-config/InitialiseSettings.php: T101634 [[gerrit:337345|Update Wikiquote talk namespace in Sanskrit Wikisource]] and [[gerrit:337402|Support legacy Wikiquote talk namespace in Sanskrit Wikisource]] (duration: 00m 40s) | [production] | 
            
  | 14:37 | <addshore@tin> | Synchronized portals: Updating portal stats [[gerrit:337366|Gerrit]] (duration: 00m 40s) | [production] | 
            
  | 14:36 | <addshore@tin> | Synchronized portals/prod/wikipedia.org/assets: Updating portal stats [[gerrit:337366|Gerrit]] (duration: 00m 40s) | [production] | 
            
  | 14:28 | <addshore@tin> | Synchronized wmf-config/InitialiseSettings.php: T155721 [[gerrit:332910|Enable TwoColConflict on metawiki]] (duration: 00m 40s) | [production] | 
            
  | 14:23 | <addshore@tin> | Synchronized php-1.29.0-wmf.11/extensions/TwoColConflict/includes/TwoColConflictHooks.php: [[gerrit:337186|Change beta feature info and talk links (duration: 00m 40s) | [production] | 
            
  | 14:19 | <addshore@tin> | Synchronized php-1.29.0-wmf.11/extensions/RevisionSlider/modules/ext.RevisionSlider.lazy.css: T157800 [[gerrit:337030|Dont set min-height and min-width for oo-ui buttons]] 2/2 (duration: 00m 55s) | [production] | 
            
  | 14:18 | <addshore@tin> | Synchronized php-1.29.0-wmf.11/extensions/RevisionSlider/modules/ext.RevisionSlider.css: T157800 [[gerrit:337030|Dont set min-height and min-width for oo-ui buttons]] 1/2 (duration: 01m 07s) | [production] | 
            
  | 13:57 | <moritzm> | rolling restart of zookeeper in eqiad to pick up Java security updates | [production] | 
            
  | 13:57 | <marostegui> | Shutdown db2060 for maintenance - T156161 | [production] | 
            
  | 13:47 | <ema> | lvs100[56]: upgrade to jessie 8.7, pybal 1.13.4, reboot into kernel 4.4.2-3+wmf8 T155401 | [production] | 
            
  | 13:37 | <ema> | lvs1004: upgrade to jessie 8.7, pybal 1.13.4, reboot into kernel 4.4.2-3+wmf8 T155401 | [production] | 
            
  | 13:33 | <moritzm> | rolling restart of zookeeper in codfw to pick up Java security updates | [production] | 
            
  | 12:32 | <elukey> | updating elastic search ACLs on cr1/cr2 for the analytics-ip4 filter | [production] | 
            
  | 11:59 | <moritzm> | removing unneeded PHP packages from mw1261-mw1265 (these were installed before we changed puppet trim most PHP packages in favour of HHVM) | [production] | 
            
  | 11:21 | <moritzm> | installing PHP security updates | [production] | 
            
  | 11:18 | <elukey> | stopped ircecho on einsteinium | [production] | 
            
  | 11:11 | <gehel@puppetmaster1001> | conftool action : set/pooled=no; selector: name=wdqs1002.eqiad.wmnet | [production] | 
            
  | 10:47 | <godog> | remove big/spammy log files from thubmro100[12] - T157949 | [production] | 
            
  | 10:35 | <gehel@puppetmaster1001> | conftool action : set/pooled=yes; selector: name=wdqs1001.eqiad.wmnet | [production] | 
            
  | 09:34 | <marostegui@tin> | Synchronized wmf-config/db-eqiad.php: Repool db1078 - T136428 (duration: 00m 41s) | [production] | 
            
  | 09:28 | <marostegui@tin> | Synchronized wmf-config/db-eqiad.php: Depool db1078 - T136428 (duration: 00m 40s) | [production] | 
            
  | 09:18 | <marostegui@tin> | Synchronized wmf-config/db-eqiad.php: Repool db1077 - T136428 (duration: 00m 40s) | [production] | 
            
  | 09:09 | <marostegui@tin> | Synchronized wmf-config/db-eqiad.php: Depool db1077 - T136428 (duration: 00m 40s) | [production] | 
            
  | 09:00 | <marostegui> | Deploy alter table s3 officewiki and mediawikiwiki for echo_notification tables on eqiad - T136428 | [production] | 
            
  | 08:10 | <elukey> | removed empty log files from elastic1022,1024,2001,1026,1040 to fix logrotate cronspam | [production] | 
            
  | 07:55 | <moritzm> | upgrade HHVM on remaining mw servers in eqiad | [production] | 
            
  | 07:03 | <marostegui> | Compressing commonswiki tables on labsdb1010 and labsdb1011 - T153743 | [production] | 
            
  | 02:38 | <l10nupdate@tin> | ResourceLoader cache refresh completed at Mon Feb 13 02:38:43 UTC 2017 (duration 5m 18s) | [production] | 
            
  | 02:33 | <l10nupdate@tin> | scap sync-l10n completed (1.29.0-wmf.11) (duration: 13m 18s) | [production] | 
            
  
    | 2017-02-11
      
      § | 
    
  | 10:02 | <gehel@puppetmaster1001> | conftool action : set/pooled=yes; selector: name=elastic20(33|34|35|36).codfw.wmnet | [production] | 
            
  | 09:53 | <elukey> | mw1236 back in production (scap pull executed before pooled=yes) - T156610 | [production] | 
            
  | 09:52 | <elukey@puppetmaster1001> | conftool action : set/pooled=yes; selector: name=mw1236.eqiad.wmnet | [production] | 
            
  | 09:35 | <elukey> | rebooting mw1236 to make sure that it comes up cleanly - T156610 | [production] | 
            
  | 09:15 | <gehel@puppetmaster1001> | conftool action : set/pooled=no; selector: name=elastic20(33|34|35|36).codfw.wmnet | [production] | 
            
  | 09:09 | <gehel> | cleanup logs on elastic20(01|25) - T139043 | [production] | 
            
  | 02:37 | <l10nupdate@tin> | ResourceLoader cache refresh completed at Sat Feb 11 02:37:10 UTC 2017 (duration 5m 19s) | [production] | 
            
  | 02:31 | <l10nupdate@tin> | scap sync-l10n completed (1.29.0-wmf.11) (duration: 11m 31s) | [production] | 
            
  | 00:00 | <mutante> | switching apt.wikimedia.org from carbon to install1002 - there might be a short time until the LE SSL cert is also adjusted | [production] | 
            
  
    | 2017-02-10
      
      § | 
    
  | 23:41 | <RainbowSprinkles> | gerrit: Restarting to pick up config changes | [production] | 
            
  | 23:12 | <RainbowSprinkles> | gerrit: restarting service | [production] | 
            
  | 22:42 | <godog> | start rsync of whisper metrics graphite2001 -> graphite1001 - T157022 | [production] | 
            
  | 22:29 | <mutante> | carbon - stopping puppet and most services, adding deprecation warning to motd, rsyncing data one last time (T132757) | [production] | 
            
  | 22:17 | <mutante> | install1001 - shutdown ganeti instance and deleting it and its disk  (T132757) | [production] | 
            
  | 21:27 | <mutante> | install1001, install2001 - removed from Icinga, shutting down  (T84380, T132757) | [production] | 
            
  | 21:18 | <mutante> | install1001, install2001 - revoke puppet certs, puppet node deactivate, delete salt keys (T84380, T132757) | [production] | 
            
  | 21:03 | <ladsgroup@tin> | Synchronized php-1.29.0-wmf.11/extensions/Nuke/Nuke_body.php: [[gerrit:337076]] Fixing Special:Nuke (T156112, T156949, T156314) (duration: 00m 58s) | [production] | 
            
  | 21:02 | <Amir1> | ladsgroup@tin:/srv/mediawiki-staging$ scap sync-file php-1.29.0-wmf.11/extensions/Nuke/Nuke_body.php '[[gerrit:337076]] Fixing Special:Nuke (T156112, T156949, T156314)' | [production] |