| 
      
        2017-02-13
      
      ยง
     | 
  
    
  | 23:29 | 
  <Pchelolo> | 
  update RESTBase to 0e9106ab8 - staging | 
  [production] | 
            
  | 23:27 | 
  <bsitzmann@tin> | 
  Finished deploy [mobileapps/deploy@cd3b897]: Update mobileapps to 776211b (duration: 03m 19s) | 
  [production] | 
            
  | 23:24 | 
  <bsitzmann@tin> | 
  Started deploy [mobileapps/deploy@cd3b897]: Update mobileapps to 776211b | 
  [production] | 
            
  | 22:39 | 
  <Pchelolo> | 
  rollback RESTBase to ea980cc5d - staging | 
  [production] | 
            
  | 22:26 | 
  <Pchelolo> | 
  update RESTBase to 0e9106ab8 - staging | 
  [production] | 
            
  | 22:14 | 
  <kaldari@tin> | 
  Synchronized dblists/: Turning on Echo for loginwiki (duration: 00m 41s) | 
  [production] | 
            
  | 22:14 | 
  <kaldari> | 
  scap sync-dir dblists/ 'Turning on Echo for loginwiki' | 
  [production] | 
            
  | 21:43 | 
  <bsitzmann@tin> | 
  Finished deploy [mobileapps/deploy@f6b4435]: Update mobileapps to 3af473f (duration: 03m 44s) | 
  [production] | 
            
  | 21:39 | 
  <bsitzmann@tin> | 
  Started deploy [mobileapps/deploy@f6b4435]: Update mobileapps to 3af473f | 
  [production] | 
            
  | 20:51 | 
  <mutante> | 
  carbon/install - adjusted Letsencrypt cert creation, deactivated reprepro to protect from accidental use, switching rsync direction from install1002->install2002, disabled cron on carbon  (T132757) | 
  [production] | 
            
  | 20:33 | 
  <Reedy> | 
  dropped old out of date echo tables from extension1.loginwiki T157105 | 
  [production] | 
            
  | 19:38 | 
  <mutante> | 
  carbon - synced /srv/ data to install1002/2002 for the last time, switching apt.wikimedia.org CNAME to install1002 - carbon deprecated (T132757) | 
  [production] | 
            
  | 19:22 | 
  <legoktm> | 
  running namespaceDupes.php on fiwiki (T103786) | 
  [production] | 
            
  | 19:11 | 
  <thcipriani@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:337423|properly set wgCirrusSearchUseIcuFolding]] T155515 (duration: 00m 41s) | 
  [production] | 
            
  | 19:00 | 
  <moritzm> | 
  upgrading mw1236 with the security updates it missed while it was powered off | 
  [production] | 
            
  | 18:33 | 
  <chasemp> | 
  labstore1005 service maintain-dbusers restart | 
  [production] | 
            
  | 18:18 | 
  <mutante> | 
  scandium - shutdown -h now (T150936) | 
  [production] | 
            
  | 18:05 | 
  <mutante> | 
  scandium - ex-zuul merger - removing from puppet, revoking puppet cert, salt key.. | 
  [production] | 
            
  | 16:08 | 
  <ema> | 
  lvs1003: upgrade to jessie 8.7, pybal 1.13.4, reboot into kernel 4.4.2-3+wmf8 T155401 | 
  [production] | 
            
  | 15:58 | 
  <ema> | 
  lvs1002: upgrade to jessie 8.7, pybal 1.13.4, reboot into kernel 4.4.2-3+wmf8 T155401 | 
  [production] | 
            
  | 15:52 | 
  <gehel> | 
  copy old blazegraph metrics to new path (wikidata.query.(triple|lag).* -> servers.<server_name>...) | 
  [production] | 
            
  | 15:36 | 
  <ema> | 
  lvs1001: upgrade to jessie 8.7, pybal 1.13.4, reboot into kernel 4.4.2-3+wmf8 T155401 | 
  [production] | 
            
  | 15:18 | 
  <moritzm> | 
  switched krypton to exim4-daemon-light (the -heavy variant was installed from an earlier role it carried) | 
  [production] | 
            
  | 15:06 | 
  <addshore> | 
  EU SWAT done | 
  [production] | 
            
  | 15:06 | 
  <addshore@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: T101634 [[gerrit:337345|Update Wikiquote talk namespace in Sanskrit Wikisource]] and [[gerrit:337402|Support legacy Wikiquote talk namespace in Sanskrit Wikisource]] (duration: 00m 40s) | 
  [production] | 
            
  | 14:37 | 
  <addshore@tin> | 
  Synchronized portals: Updating portal stats [[gerrit:337366|Gerrit]] (duration: 00m 40s) | 
  [production] | 
            
  | 14:36 | 
  <addshore@tin> | 
  Synchronized portals/prod/wikipedia.org/assets: Updating portal stats [[gerrit:337366|Gerrit]] (duration: 00m 40s) | 
  [production] | 
            
  | 14:28 | 
  <addshore@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: T155721 [[gerrit:332910|Enable TwoColConflict on metawiki]] (duration: 00m 40s) | 
  [production] | 
            
  | 14:23 | 
  <addshore@tin> | 
  Synchronized php-1.29.0-wmf.11/extensions/TwoColConflict/includes/TwoColConflictHooks.php: [[gerrit:337186|Change beta feature info and talk links (duration: 00m 40s) | 
  [production] | 
            
  | 14:19 | 
  <addshore@tin> | 
  Synchronized php-1.29.0-wmf.11/extensions/RevisionSlider/modules/ext.RevisionSlider.lazy.css: T157800 [[gerrit:337030|Dont set min-height and min-width for oo-ui buttons]] 2/2 (duration: 00m 55s) | 
  [production] | 
            
  | 14:18 | 
  <addshore@tin> | 
  Synchronized php-1.29.0-wmf.11/extensions/RevisionSlider/modules/ext.RevisionSlider.css: T157800 [[gerrit:337030|Dont set min-height and min-width for oo-ui buttons]] 1/2 (duration: 01m 07s) | 
  [production] | 
            
  | 13:57 | 
  <moritzm> | 
  rolling restart of zookeeper in eqiad to pick up Java security updates | 
  [production] | 
            
  | 13:57 | 
  <marostegui> | 
  Shutdown db2060 for maintenance - T156161 | 
  [production] | 
            
  | 13:47 | 
  <ema> | 
  lvs100[56]: upgrade to jessie 8.7, pybal 1.13.4, reboot into kernel 4.4.2-3+wmf8 T155401 | 
  [production] | 
            
  | 13:37 | 
  <ema> | 
  lvs1004: upgrade to jessie 8.7, pybal 1.13.4, reboot into kernel 4.4.2-3+wmf8 T155401 | 
  [production] | 
            
  | 13:33 | 
  <moritzm> | 
  rolling restart of zookeeper in codfw to pick up Java security updates | 
  [production] | 
            
  | 12:32 | 
  <elukey> | 
  updating elastic search ACLs on cr1/cr2 for the analytics-ip4 filter | 
  [production] | 
            
  | 11:59 | 
  <moritzm> | 
  removing unneeded PHP packages from mw1261-mw1265 (these were installed before we changed puppet trim most PHP packages in favour of HHVM) | 
  [production] | 
            
  | 11:21 | 
  <moritzm> | 
  installing PHP security updates | 
  [production] | 
            
  | 11:18 | 
  <elukey> | 
  stopped ircecho on einsteinium | 
  [production] | 
            
  | 11:11 | 
  <gehel@puppetmaster1001> | 
  conftool action : set/pooled=no; selector: name=wdqs1002.eqiad.wmnet | 
  [production] | 
            
  | 10:47 | 
  <godog> | 
  remove big/spammy log files from thubmro100[12] - T157949 | 
  [production] | 
            
  | 10:35 | 
  <gehel@puppetmaster1001> | 
  conftool action : set/pooled=yes; selector: name=wdqs1001.eqiad.wmnet | 
  [production] | 
            
  | 09:34 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1078 - T136428 (duration: 00m 41s) | 
  [production] | 
            
  | 09:28 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1078 - T136428 (duration: 00m 40s) | 
  [production] | 
            
  | 09:18 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1077 - T136428 (duration: 00m 40s) | 
  [production] | 
            
  | 09:09 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1077 - T136428 (duration: 00m 40s) | 
  [production] | 
            
  | 09:00 | 
  <marostegui> | 
  Deploy alter table s3 officewiki and mediawikiwiki for echo_notification tables on eqiad - T136428 | 
  [production] | 
            
  | 08:10 | 
  <elukey> | 
  removed empty log files from elastic1022,1024,2001,1026,1040 to fix logrotate cronspam | 
  [production] | 
            
  | 07:55 | 
  <moritzm> | 
  upgrade HHVM on remaining mw servers in eqiad | 
  [production] |