| 2015-12-17
      
      § | 
    
  | 11:20 | <jynus> | performing schema change on x1-master wikishared.cx_translations | [production] | 
            
  | 10:47 | <jynus> | performing schema change on s7-master metawiki.oauth_registered_consumer | [production] | 
            
  | 10:30 | <godog> | nodetool stop -- CLEANUP restbase1004 | [production] | 
            
  | 02:53 | <l10nupdate@tin> | ResourceLoader cache refresh completed at Thu Dec 17 02:53:11 UTC 2015 (duration 7m 12s) | [production] | 
            
  | 02:45 | <mwdeploy@tin> | sync-l10n completed (1.27.0-wmf.9) (duration: 08m 22s) | [production] | 
            
  | 02:27 | <mwdeploy@tin> | sync-l10n completed (1.27.0-wmf.8) (duration: 10m 32s) | [production] | 
            
  | 00:28 | <ori@tin> | Synchronized php-1.27.0-wmf.8/includes/api/ApiStashEdit.php: I552cf6b0420: Upgrade some ApiStashEdit logging calls to info() (duration: 00m 30s) | [production] | 
            
  | 00:05 | <catrope@tin> | Synchronized wmf-config/CommonSettings.php: Password policy for sysadmin group (duration: 00m 29s) | [production] | 
            
  
    | 2015-12-16
      
      § | 
    
  | 23:37 | <ori@tin> | Synchronized php-1.27.0-wmf.8/includes/api/ApiStashEdit.php: local hack some extra debug logging into ApiStashEdit (take 2) (duration: 00m 30s) | [production] | 
            
  | 23:32 | <ori@tin> | Synchronized php-1.27.0-wmf.8/includes/api/ApiStashEdit.php: local hack some extra debug logging into ApiStashEdit (duration: 00m 30s) | [production] | 
            
  | 23:27 | <yurik@tin> | Synchronized php-1.27.0-wmf.9/extensions/Graph/: Deployed Graph ext to master - protocol issue (duration: 00m 32s) | [production] | 
            
  | 23:10 | <legoktm> | running foreachwiki extensions/CentralAuth/maintenance/checkLocalUser.php --verbose=1 --delete=1 (T119736) | [production] | 
            
  | 23:07 | <robh> | fermium returned to normal service with new/renewed ssl cert for lists.w.o | [production] | 
            
  | 22:58 | <robh> | disabled puppet on fermium for the lists.w.o cert update | [production] | 
            
  | 22:58 | <robh> | librenms returned to normal service (puppet renabled on netmon1001) | [production] | 
            
  | 22:48 | <robh> | puppet disabled on netmon1001 for librenms certificate update | [production] | 
            
  | 21:31 | <subbu> | finished deploying parsoid 64029e12 | [production] | 
            
  | 21:29 | <bearND> | mobileapps deployed sha1 9f91ad5 | [production] | 
            
  | 21:23 | <subbu> | restarted parsoid on wtp1005 as a canary | [production] | 
            
  | 21:19 | <subbu> | starting parsoid deploy | [production] | 
            
  | 21:14 | <bearND> | starting mobileapps deploy | [production] | 
            
  | 19:46 | <ostriches> | iridium: repacking MW / OPUP repositories as phd user. This needs to be a cron. | [production] | 
            
  | 19:07 | <thcipriani@tin> | rebuilt wikiversions.php and synchronized wikiversions files: group1 wikis to 1.27.0-wmf.9 | [production] | 
            
  | 19:00 | <thcipriani> | updating group1 wikis to 1.27.0-wmf.9 | [production] | 
            
  | 18:39 | <jynus> | performing schema change on ruwiki.geo_tags on db2046 | [production] | 
            
  | 17:46 | <jynus> | setting mysql db1041 as db2029's master | [production] | 
            
  | 17:21 | <jynus> | restarting and configuring mysql on db2029 (there will be an increase of errors- from pings, not real traffic) | [production] | 
            
  | 17:08 | <jynus> | setting mysql db1022 as db2028's master | [production] | 
            
  | 16:50 | <demon@tin> | Synchronized wmf-config/InitialiseSettings.php: graph namespace removal/cleanup (duration: 00m 31s) | [production] | 
            
  | 16:41 | <gwicke> | cleared snapshots on cassandra cluster | [production] | 
            
  | 16:32 | <gwicke> | restarted `nodetool decommission` on restbase1004 | [production] | 
            
  | 16:31 | <jynus> | restarting and reconfiguring mysql on db1041 | [production] | 
            
  | 15:54 | <jynus@tin> | Synchronized wmf-config/db-eqiad.php: Repool db1022, depool db1041 (duration: 00m 30s) | [production] | 
            
  | 15:53 | <godog> | cassandra on restbase1004 couldn't finish decomissioning, out of disk space, running nodetool clean | [production] | 
            
  | 15:28 | <jynus> | restarting and reconfiguring mysql on db2028 | [production] | 
            
  | 15:17 | <yurik> | updated graphoid service | [production] | 
            
  | 15:16 | <moritzm> | removed ecryptfs-utils across the cluster | [production] | 
            
  | 15:09 | <godog> | roll-restart swift daemons on ms-be1* | [production] | 
            
  | 15:04 | <jynus> | setting db2023 master to be now db1049 instead of m5-master | [production] | 
            
  | 14:56 | <mobrovac> | restbase roll-restarting restbase | [production] | 
            
  | 14:52 | <godog> | reenable and run puppet on restbase* and aqs* | [production] | 
            
  | 14:49 | <mobrovac> | restbase restart rb1001 | [production] | 
            
  | 14:46 | <urandom> | starting `nodetool cleanup' on restbase1009-a.eqiad (https://phabricator.wikimedia.org/T121535) | [production] | 
            
  | 14:41 | <urandom> | starting `nodetool cleanup' on restbase1006.eqiad (https://phabricator.wikimedia.org/T121535) | [production] | 
            
  | 14:17 | <mobrovac> | restbase reenabling and running puppet on canary rb1001 | [production] | 
            
  | 14:08 | <godog> | reenable puppet on restbase test cluster | [production] | 
            
  | 14:06 | <akosiaris> | delete empty panel in https://grafana-admin.wikimedia.org/dashboard/db/graphoid | [production] | 
            
  | 14:04 | <akosiaris> | repool scb1001 for mobileapps | [production] | 
            
  | 13:59 | <akosiaris> | depool scb1001 for mobileapps (oid) | [production] | 
            
  | 13:58 | <akosiaris> | repool sca1002 for citoid, mathoid, graphoid, cxserver | [production] |