| 
      
        2016-11-29
      
      §
     | 
  
    
  | 12:05 | 
  <elukey> | 
  complete rolling restart of apache in eqiad | 
  [production] | 
            
  | 11:48 | 
  <elukey> | 
  re-enable puppet on mw1* hosts and apply Apache config change (https://gerrit.wikimedia.org/r/#/c/314519) | 
  [production] | 
            
  | 11:23 | 
  <elukey> | 
  disabled puppet on mw1* hosts as pre-step for https://gerrit.wikimedia.org/r/#/c/314519 | 
  [production] | 
            
  | 10:53 | 
  <_joe_> | 
  restarting pybal on low-traffic codfw | 
  [production] | 
            
  | 10:50 | 
  <_joe_> | 
  restarting pybal on low-traffic eqiad | 
  [production] | 
            
  | 10:40 | 
  <oblivian@puppetmaster1001> | 
  conftool action : set/pooled=yes; selector: cluster=scb,service=pdfrender | 
  [production] | 
            
  | 10:02 | 
  <_joe_> | 
  upgrading firejail on all other scb servers in eqiad | 
  [production] | 
            
  | 09:48 | 
  <_joe_> | 
  upgrading firejail on scb1004, restarting all dependent services | 
  [production] | 
            
  | 06:41 | 
  <marostegui> | 
  Stopping replication db1095 - s1 instance for maintenance - T150802 | 
  [production] | 
            
  | 06:37 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-codfw.php: Depool db2048 - T149553 (duration: 00m 46s) | 
  [production] | 
            
  | 03:09 | 
  <demon@tin> | 
  Synchronized docroot/foundation: rm old legalcode junk (duration: 01m 33s) | 
  [production] | 
            
  | 03:03 | 
  <mutante> | 
  druid1001 - restarting all druid services | 
  [production] | 
            
  | 02:09 | 
  <mutante> | 
  rolling restart of hhvm service across eqiad | 
  [production] | 
            
  | 01:54 | 
  <mutante> | 
  terbium, mw1260: purging libicu48 package in 'rc' status | 
  [production] | 
            
  | 01:52 | 
  <mutante> | 
  deploying icu upgrade on all eqiad mw servers | 
  [production] | 
            
  | 01:46 | 
  <mutante> | 
  restarting hhvm service across codfw | 
  [production] | 
            
  | 01:45 | 
  <demon@tin> | 
  Synchronized wmf-config/CommonSettings.php: fix implicit message parsing error (duration: 00m 48s) | 
  [production] | 
            
  | 01:44 | 
  <thcipriani@tin> | 
  Synchronized README: scap 3.4 sync file (duration: 00m 53s) | 
  [production] | 
            
  | 01:42 | 
  <godog> | 
  upgrade prometheus-varnish-exporter on cache boxes in ulsfo T150479 | 
  [production] | 
            
  | 01:34 | 
  <mutante> | 
  mw2152 - remove libicu48 (for some reason this one host was different from all the others) | 
  [production] | 
            
  | 01:27 | 
  <godog> | 
  upgrade prometheus-varnish-exporter on cache_misc/ulsfo T150479 | 
  [production] | 
            
  | 01:24 | 
  <mutante> | 
  rolling out security upgrades for libicu52 [DSA 3725-1] (CVE-2014-9911 CVE-2015-2632 CVE-2015-4844 CVE-2016-0494 CVE-2016-6293 CVE-2016-7415) | 
  [production] | 
            
  | 01:17 | 
  <demon@tin> | 
  Synchronized w/MWVersion.php: bleh, still used (duration: 00m 47s) | 
  [production] | 
            
  | 01:05 | 
  <demon@tin> | 
  Synchronized w/: remove old MWVersion entry point (duration: 00m 47s) | 
  [production] | 
            
  | 01:04 | 
  <demon@tin> | 
  Synchronized rpc/RunJobs.php: more relative mwversion stuff (duration: 00m 43s) | 
  [production] | 
            
  | 01:03 | 
  <demon@tin> | 
  Synchronized multiversion/: use MWVersion relative path (duration: 00m 59s) | 
  [production] | 
            
  | 01:02 | 
  <demon@tin> | 
  Synchronized scap/plugins/clean.py: typofix (duration: 00m 43s) | 
  [production] | 
            
  | 00:42 | 
  <maxsem@tin> | 
  Synchronized wmf-config/: https://gerrit.wikimedia.org/r/#/c/315985/ (duration: 00m 51s) | 
  [production] | 
            
  | 00:33 | 
  <maxsem@tin> | 
  Synchronized wmf-config/: https://gerrit.wikimedia.org/r/#/c/314748/ (duration: 00m 46s) | 
  [production] | 
            
  | 00:20 | 
  <mutante> | 
  re-enabled icinga notifications for labtest* services (first double checked they are _not_ paging anymore) (T120047) | 
  [production] | 
            
  | 00:17 | 
  <demon@tin> | 
  Finished scap: moving some stuff around, pruned old branches too (duration: 20m 29s) | 
  [production] | 
            
  
    | 
      
        2016-11-28
      
      §
     | 
  
    
  | 23:56 | 
  <demon@tin> | 
  Started scap: moving some stuff around, pruned old branches too | 
  [production] | 
            
  | 23:33 | 
  <demon@tin> | 
  Synchronized w/: (no message) (duration: 00m 48s) | 
  [production] | 
            
  | 23:27 | 
  <mattflaschen@tin> | 
  Synchronized wmf-config/InitialiseSettings-labs.php: Beta Cluster only (duration: 00m 44s) | 
  [production] | 
            
  | 23:20 | 
  <demon@tin> | 
  Synchronized scap/plugins/clean.py: Completeness, testing, etc (duration: 00m 43s) | 
  [production] | 
            
  | 23:14 | 
  <reedy@tin> | 
  Synchronized wmf-config/CommonSettings.php: fix undefined user (duration: 00m 48s) | 
  [production] | 
            
  | 23:08 | 
  <ejegg> | 
  updated payments-wiki from d7ed14407aa7be9a790778cae644c2b320bb7aa4 to bd8012ce876db59142e28bf6f6e4a2bd549f4481 | 
  [production] | 
            
  | 22:49 | 
  <eileen2> | 
  disable job Project Dedupe CiviCRM contacts (name-match) | 
  [production] | 
            
  | 22:43 | 
  <reedy@tin> | 
  Synchronized wmf-config/CommonSettings.php: Log users elevated groups on login attempts (duration: 00m 47s) | 
  [production] | 
            
  | 22:07 | 
  <reedy@tin> | 
  Synchronized wmf-config/CommonSettings.php: Only add oathauth-enable if the group exists on the wiki (duration: 00m 43s) | 
  [production] | 
            
  | 22:01 | 
  <reedy@tin> | 
  Synchronized wmf-config/CommonSettings.php: OATHAuth for more groups and bump their password requirements (duration: 00m 45s) | 
  [production] | 
            
  | 22:00 | 
  <reedy@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: OATHAuth for more groups and bump their password requirements (duration: 00m 45s) | 
  [production] | 
            
  | 21:27 | 
  <mdholloway> | 
  mobileapps deployed 1d09b98 | 
  [production] | 
            
  | 20:56 | 
  <filippo@tin> | 
  Synchronized php-1.29.0-wmf.3/api.php: Revert bandaid from ori (duration: 00m 53s) | 
  [production] | 
            
  | 20:22 | 
  <Krenair> | 
  wikitech-static: moved to REL1_28 | 
  [production] | 
            
  | 20:09 | 
  <hoo> | 
  Started dumpwikidatajson.sh on snapshot1007 (T151787) | 
  [production] | 
            
  | 20:05 | 
  <ottomata> | 
  rolling restart of eventlogging-service-eventbus in eqiad to pick up new python-tornado version bump from jessie backports (so it doesn't bite us unexpectedly later) | 
  [production] | 
            
  | 20:05 | 
  <chasemp> | 
  T150679 changes for user_properties view on labsdb1001 and 1003 | 
  [production] | 
            
  | 19:50 | 
  <thcipriani@tin> | 
  Synchronized php-1.29.0-wmf.3/extensions/Wikidata: SWAT: [[gerrit:323880|Update Wikibase: Use the "redirect" table in SqlEntityIdPager]] T151356 (duration: 02m 11s) | 
  [production] | 
            
  | 19:42 | 
  <urandom> | 
  T151086: bootstrap of restbase2012-a.codfw.wmnet starting... | 
  [production] |