| 2017-09-28
      
      § | 
    
  | 23:45 | <awight@tin> | Started deploy [ores/deploy@42c5663]: Cause ORES service restart | [production] | 
            
  | 22:18 | <MaxSem> | Deployed fix for T176247 | [production] | 
            
  | 21:30 | <ejegg> | updated CiviCRM from 3f48cb03b4a216a34a61f9744a297de4fb99d8a6 to 213886998cad62a37aab5b214290b8431ab41199 | [production] | 
            
  | 21:23 | <XenoRyet> | Updated SmashPig  from dc6220391036c630c8ad14c0d894d33c1e8d8c2c to d056a131ccbbd4b505da4f99e04f5337fdd9c32e | [production] | 
            
  | 19:36 | <demon@tin> | rebuilt wikiversions.php and synchronized wikiversions files: group2 to wmf.1 | [production] | 
            
  | 19:34 | <legoktm> | mysql:wikiadmin@db1052 [enwiki]> delete from user_groups where ug_user =17629530 limit 1; (T176798) | [production] | 
            
  | 19:31 | <bblack> | cp4026 - depool->reboot for bios updates, isolcpus fix | [production] | 
            
  | 19:28 | <legoktm> | mysql:wikiadmin@db1062 [centralauth]> UPDATE localuser SET lu_attached_method="new" WHERE lu_attached_method is NULL AND lu_name="Rua"; | [production] | 
            
  | 19:14 | <ejegg> | updated fundraising tools 5708cf1 Add 'first_donation_date' column to Silverpop export | [production] | 
            
  | 19:14 | <robh> | cp4024 initial tests done (takes about 2 hours or so) then prompted for more in depth testing (hit yes, no failures so far)  T174891 | [production] | 
            
  | 19:00 | <bblack> | cp4025 - depool->reboot for bios updates, isolcpus fix | [production] | 
            
  | 18:54 | <thcipriani@tin> | Synchronized php-1.31.0-wmf.1/skins/MinervaNeue/includes/skins/SkinMinerva.php: SWAT: [[gerrit:381259|Revision::newFromTitle may return null]] T176882 (duration: 00m 50s) | [production] | 
            
  | 18:42 | <bblack> | cp4023 - depool->reboot for bios updates, isolcpus fix | [production] | 
            
  | 18:37 | <thcipriani@tin> | Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:381251|Create rollbacker group at viwiktionary]] T176979 (duration: 00m 50s) | [production] | 
            
  | 18:26 | <thcipriani@tin> | Synchronized docroot/noc: SWAT: [[gerrit:376337|copy squid.php->reverse-proxy.php, squid-labs->reverse-proxy-staging]] PART II T104148 (duration: 00m 49s) | [production] | 
            
  | 18:25 | <thcipriani@tin> | Synchronized wmf-config: SWAT: [[gerrit:376337|copy squid.php->reverse-proxy.php, squid-labs->reverse-proxy-staging]] T104148 (duration: 00m 51s) | [production] | 
            
  | 17:57 | <arlolra> | Updated Parsoid to 2f4b9a8c (T176363, T176151, T170832) | [production] | 
            
  | 17:52 | <mutante> | librenms - renewed LE TLS cert ..via puppet, after Icinga alerted and accidentally still had "do_acme: false" in Hiera | [production] | 
            
  | 17:46 | <arlolra@tin> | Finished deploy [parsoid/deploy@84342fe]: Updating Parsoid to 2f4b9a8c (duration: 15m 25s) | [production] | 
            
  | 17:43 | <bblack> | cp4028 - depool->reboot for bios updates, isolcpus fix | [production] | 
            
  | 17:40 | <moritzm> | installing apache updates on cobalt | [production] | 
            
  | 17:32 | <bblack> | cp4022 - depool->reboot for bios updates, isolcpus fix | [production] | 
            
  | 17:30 | <arlolra@tin> | Started deploy [parsoid/deploy@84342fe]: Updating Parsoid to 2f4b9a8c | [production] | 
            
  | 17:25 | <marostegui> | Run fixStuckGlobalRename.php --wiki=enwiktionary --logwiki=metawiki "CodeCat" "Rua"  on terbium - T176985 | [production] | 
            
  | 17:20 | <bblack> | cp4027 - depool->reboot for bios updates, isolcpus fix | [production] | 
            
  | 17:08 | <bblack> | cp4021 - depool->reboot for bios update | [production] | 
            
  | 17:05 | <robh> | bios firmware uploaded and awaiting reboot for application on cp402[1235678] | [production] | 
            
  | 16:48 | <bd808> | Created centralauth.{analytics,web}.db.svc.eqiad.wmflabs Designate CNAME (T176978) | [production] | 
            
  | 16:26 | <robh> | cp4024 has had ilom and bios firmware updaed per epsa error checking, now re-running epsa utility T174891 | [production] | 
            
  | 16:09 | <robh> | cp4024 bios update in progress (has been for last 5 minutes), letting it alone for a solid 30 to let it finish | [production] | 
            
  | 16:01 | <marostegui> | Global rename of CodeCat → Rua - T176985 | [production] | 
            
  | 15:22 | <hoo> | Ran scap pull on mwdebug1001 after experiments with T176844 | [production] | 
            
  | 14:54 | <dcausse> | restarting elastic on relforge servers to reinstall prod ltr plugin | [production] | 
            
  | 14:51 | <marostegui> | Stop MySQL on db1035 as server is going to be decommissioned - T176931 | [production] | 
            
  | 14:45 | <moritzm> | installing apache updates on puppet mastes (will trigger a few failing puppet runs, but ircecho disabled temporarily) | [production] | 
            
  | 14:42 | <marostegui> | Remove db1035 from tendril - T176931 | [production] | 
            
  | 14:31 | <moritzm> | upgrading mw1276 (API canary) to HHVM 3.18.5 | [production] | 
            
  | 14:28 | <moritzm> | upgrading mwdebug* to HHVM 3.18.5 | [production] | 
            
  | 13:56 | <marostegui@tin> | Synchronized wmf-config/db-eqiad.php: Remove db1035 from config as it will be decommissioned - T176931 (duration: 00m 49s) | [production] | 
            
  | 13:55 | <marostegui@tin> | Synchronized wmf-config/db-codfw.php: Remove db1035 from config as it will be decommissioned - T176931 (duration: 00m 50s) | [production] | 
            
  | 13:40 | <zeljkof> | EU SWAT finished | [production] | 
            
  | 13:38 | <zfilipin@tin> | Synchronized php-1.31.0-wmf.1/extensions/VisualEditor/modules/ve-mw/init/targets/ve.init.mw.DesktopArticleTarget.init.js: SWAT: [[gerrit:381210|ve.init.mw.DesktopArticleTarget: Remove hack for reversed tabs in RTL in Vector (T50017)]] (duration: 00m 50s) | [production] | 
            
  | 13:37 | <moritzm> | upgrading mw1262 to mw1265 (canary app servers) to HHVM 3.18.5 | [production] | 
            
  | 13:35 | <moritzm> | uploaded HHVM 3.18.5 for jessie-wikimedia to apt.wikimedia.org | [production] | 
            
  | 13:26 | <marostegui> | Optimize table on s4 codfw master (db2051), with replication (lag might be generated) - T174509 | [production] | 
            
  | 13:13 | <ema> | bounce pybal on ulsfo primaries to pick up bgp-local-ips config change T103882 | [production] | 
            
  | 13:12 | <marostegui> | Drop  redundant indexes from pagelinks and templatelinks on s4 - T174509 | [production] | 
            
  | 13:10 | <moritzm> | upgrading mw1261 to HHVM 3.18.5 | [production] | 
            
  | 13:08 | <ema> | bounce pybal on ulsfo secondaries to pick up bgp-local-ips config change T103882 | [production] | 
            
  | 13:02 | <marostegui> | Optimize template links and pagelinks on db2053 and db2046 - T174509 | [production] |