| 2018-02-21
      
      ยง | 
    
  | 18:48 | <thcipriani@tin> | Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:412676|Add namespace localization for sdwiki]] T186943 (duration: 01m 13s) | [production] | 
            
  | 18:39 | <thcipriani@tin> | Synchronized wmf-config/throttle.php: SWAT: [[gerrit:412947|Added new throttle rule for Wikipedia Women in Red editathon]] T187803 (duration: 01m 12s) | [production] | 
            
  | 18:37 | <chasemp> | labsdb rm -fR /usr/local/lib/mediawiki-config && puppet agent --test | [production] | 
            
  | 18:24 | <thcipriani@tin> | Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:412439|Set Topic namespace alias of zhwiki]] T187546 (duration: 01m 13s) | [production] | 
            
  | 18:12 | <_joe_> | stopped testing on mwdebug1001 for SWAT window | [production] | 
            
  | 17:43 | <ema> | eqsin LVSs: upgrade pybal to 1.14.4 | [production] | 
            
  | 17:34 | <_joe_> | resuming tests on mwdebug1001 | [production] | 
            
  | 17:17 | <ema> | eqiad LVSs: bounce pybal for labweb proxfetch config changes | [production] | 
            
  | 17:12 | <ppchelko@tin> | Finished deploy [changeprop/deploy@e9a6bb0]: Use post for ORES precache rules T158437 (duration: 01m 23s) | [production] | 
            
  | 17:11 | <ppchelko@tin> | Started deploy [changeprop/deploy@e9a6bb0]: Use post for ORES precache rules T158437 | [production] | 
            
  | 17:07 | <_joe_> | finished testing on mwdebug1001 for swat | [production] | 
            
  | 16:56 | <oblivian@puppetmaster1001> | conftool action : edit; selector: name=ReadOnly,scope=eqiad | [production] | 
            
  | 16:40 | <_joe_> | testing various etcd failure scenarios on mwdebug1001, T185078 | [production] | 
            
  | 16:39 | <ppchelko@tin> | Finished deploy [changeprop/deploy@1be63aa]: Simplify ORES precaching by using the new endpoint T158437 (duration: 01m 33s) | [production] | 
            
  | 16:37 | <ppchelko@tin> | Started deploy [changeprop/deploy@1be63aa]: Simplify ORES precaching by using the new endpoint T158437 | [production] | 
            
  | 16:27 | <ema> | lvs1010: restart pybal | [production] | 
            
  | 16:00 | <godog> | restart rsyslogd on lithium and wezen - T136312 | [production] | 
            
  | 15:50 | <gilles@tin> | Synchronized wmf-config/filebackend.php: Thumbor private wiki support deployment: [[gerrit:413115|Serve private wiki thumbnails with Thumbor (T169144)]] (duration: 01m 12s) | [production] | 
            
  | 15:44 | <no_justification> | pruned old 1.29.x and 1.30.x versions that somehow stuck around. Also 1.31.0-wmf.* cache/ directories for unused branches. T157030 | [production] | 
            
  | 15:37 | <gilles@tin> | Synchronized wmf-config/filebackend.php: Thumbor private wiki support deployment: [[gerrit:413168|Serve officewiki thumbnails with Thumbor (T169144)]] (duration: 01m 11s) | [production] | 
            
  | 15:27 | <gilles@tin> | Synchronized private/PrivateSettings.php.example: Thumbor private wiki support deployment: [[gerrit:413168|Add Thumbor/Mediawiki shared secret (T169144)]] (duration: 01m 11s) | [production] | 
            
  | 15:24 | <chasemp> | reboot labtestservices2002 | [production] | 
            
  | 15:24 | <gilles@tin> | Synchronized wmf-config/filebackend.php: Thumbor private wiki support deployment: [[gerrit:413168|Add Thumbor/Mediawiki shared secret (T169144)]] (duration: 01m 12s) | [production] | 
            
  | 15:19 | <gilles> | Thumbor private wiki support deployment | [production] | 
            
  | 15:08 | <zeljkof> | EU SWAT finished | [production] | 
            
  | 15:08 | <zfilipin@tin> | Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:412983|Removing Mobile beta feedback link (T187712)]] (duration: 01m 12s) | [production] | 
            
  | 15:03 | <zfilipin@tin> | Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:412996|Disable Page Previews EventLogging instrumentation (T185973)]] (duration: 01m 13s) | [production] | 
            
  | 14:52 | <_joe_> | rolling restart another 4 api appservers | [production] | 
            
  | 14:49 | <oblivian@tin> | Synchronized wmf-config: Serve configuration to mwdebug hosts via etcd (duration: 01m 16s) | [production] | 
            
  | 14:42 | <_joe_> | restarted hhvm on mwdebug1001 too | [production] | 
            
  | 14:38 | <_joe_> | restarting hhvm on mwdebug1002 | [production] | 
            
  | 14:06 | <_joe_> | restarting hhvm on misbehaving api appservers | [production] | 
            
  | 14:02 | <zfilipin@tin> | Synchronized wmf-config/throttle.php: SWAT: [[gerrit:413147|Add new throttle rule (T187870)]] (duration: 01m 13s) | [production] | 
            
  | 13:28 | <marostegui> | Reboot db2092 for a kernel upgrade | [production] | 
            
  | 13:26 | <moritzm> | powercycling ganeti1007 | [production] | 
            
  | 12:43 | <_joe_> | rolling restart of hhvm on api servers under high load | [production] | 
            
  | 12:38 | <elukey> | restart hhvm on mw1234 - high load | [production] | 
            
  | 12:35 | <marostegui@tin> | Synchronized wmf-config/db-eqiad.php: Clarify that db1067 is now s1 candidate master - T186321 (duration: 01m 13s) | [production] | 
            
  | 12:26 | <elukey> | restart hhvm on mw1231 - high load, hhvm-dump-debug in /home/elukey/hhvm.6759.bt | [production] | 
            
  | 12:21 | <elukey> | restart hhvm on mw1227 - high load, hhvm-dump-debug in /home/elukey/hhvm.23382.bt | [production] | 
            
  | 12:10 | <moritzm> | uploading retpoline-enabled gcc-4.9 to apt.wikimedia.org / jessie-wikimedia to be able to use it on boron for building Linux (trying to adapt our pbuilder setup to also include security.debian.org ran into a few proxy-related problems and this is really a rare corner case anyway) | [production] | 
            
  | 12:02 | <ema> | lvs5003: pybal upgraded to 1.14.4 | [production] | 
            
  | 12:01 | <ema> | pybal 1.14.4 uploaded to apt.w.o | [production] | 
            
  | 11:17 | <moritzm> | installing db5.3 security updates | [production] | 
            
  | 11:12 | <jynus> | cloning db2011 to db2044 | [production] | 
            
  | 10:40 | <kart_> | Finished running CLL preference migration script dry-run on terbium (T187677) | [production] | 
            
  | 10:33 | <marostegui> | Reload haproxy on dbproxy1005 - T187722 | [production] | 
            
  | 10:26 | <marostegui> | Remove db2030 from tendril - T187768 | [production] | 
            
  | 10:09 | <moritzm> | installing openssh bugfix updates from jessie/stretch point releases | [production] | 
            
  | 10:01 | <kart_> | Running CLL preference migration script dry-run on terbium (T187677) | [production] |