| 
      
        2015-12-03
      
      §
     | 
  
    
  | 14:48 | 
  <mobrovac> | 
  restbase start deployment of 262da91a | 
  [production] | 
            
  | 14:06 | 
  <moritzm> | 
  installed dpkg updates across the cluster | 
  [production] | 
            
  | 11:35 | 
  <moritzm> | 
  restarting cassandra on aqs cluster (subsequently) to effect openjdk security update | 
  [production] | 
            
  | 10:51 | 
  <jynus> | 
  restarting, upgrading and general maintenance for es1013 (depooled) | 
  [production] | 
            
  | 10:36 | 
  <_joe_> | 
  imported dh-python into precise/universe from the ubuntu cloud archive | 
  [production] | 
            
  | 10:26 | 
  <jynus@tin> | 
  Synchronized wmf-config/db-eqiad.php: Depool es1013 for maintenance (duration: 00m 30s) | 
  [production] | 
            
  | 05:50 | 
  <l10nupdate@tin> | 
  ResourceLoader cache refresh completed at Thu Dec  3 05:50:08 UTC 2015 (duration 50m 7s) | 
  [production] | 
            
  | 02:25 | 
  <mwdeploy@tin> | 
  sync-l10n completed (1.27.0-wmf.7) (duration: 09m 54s) | 
  [production] | 
            
  
    | 
      
        2015-12-02
      
      §
     | 
  
    
  | 22:09 | 
  <jynus> | 
  unscheduled restart of dbstore1002 (analytics-slave) | 
  [production] | 
            
  | 21:44 | 
  <jynus> | 
  disabling all alert notifications for dbstore1002 | 
  [production] | 
            
  | 21:30 | 
  <bblack> | 
  rebooting lvs1007 for interface config test (not active, no BGP) | 
  [production] | 
            
  | 20:37 | 
  <Jeff_Green> | 
  enable mail queue monitoring for fundraising | 
  [production] | 
            
  | 19:13 | 
  <jynus> | 
  restarting mysql on db2067 to test a configuration change | 
  [production] | 
            
  | 18:07 | 
  <mutante> | 
  mw1136 - hhvm restart | 
  [production] | 
            
  | 14:45 | 
  <Coren> | 
  deploying cleanup of labs PAM configuration - this should be a functional noop but may cause some puppet noise | 
  [production] | 
            
  | 12:41 | 
  <akosiaris> | 
  restart cassandra on maps-test200{1,2,3,4}.codfw.wmnet | 
  [production] | 
            
  | 11:23 | 
  <moritzm> | 
  restarting cassandra on restbase100[78] (subsequently) (to effect openjdk security updates plus related libs) | 
  [production] | 
            
  | 11:06 | 
  <moritzm> | 
  restarting cassandra on restbase100[2-4] (subsequently) (to effect openjdk security updates plus related libs) | 
  [production] | 
            
  | 11:04 | 
  <moritzm> | 
  restart cassandra on restbase1001 (to effect openjdk security updates plus related libs) | 
  [production] | 
            
  | 10:21 | 
  <aude@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: Enabling data access for wikinews, wikispecies and mediawiki.org (duration: 00m 27s) | 
  [production] | 
            
  | 10:20 | 
  <aude@tin> | 
  Synchronized dblists/arbitraryaccess.dblist: Enabling data access for wikinews, wikispecies and mediawiki.org (duration: 00m 30s) | 
  [production] | 
            
  | 10:14 | 
  <_joe_> | 
  clearing the job cache for ocg1003 | 
  [production] | 
            
  | 09:22 | 
  <_joe_> | 
  stopped ocg service on ocg1003 | 
  [production] | 
            
  | 08:38 | 
  <_joe_> | 
  planet1001 recovered as soon as I did get into console via gnt-instance | 
  [production] | 
            
  | 06:25 | 
  <ori@tin> | 
  Synchronized php-1.27.0-wmf.7/extensions/NavigationTiming: Idb675cdce: Add isHiDPI and isHttp2 properties; drop isHttps (T119014) (duration: 00m 50s) | 
  [production] | 
            
  | 05:53 | 
  <l10nupdate@tin> | 
  ResourceLoader cache refresh completed at Wed Dec  2 05:53:21 UTC 2015 (duration 53m 20s) | 
  [production] | 
            
  | 02:27 | 
  <mutante|away> | 
  labcontrol2001 - disable puppet, kill from puppet stored configs  | 
  [production] | 
            
  | 02:26 | 
  <mwdeploy@tin> | 
  sync-l10n completed (1.27.0-wmf.7) (duration: 10m 47s) | 
  [production] | 
            
  | 00:29 | 
  <mutante> | 
  puppetstoredconfigclean.rb labcontrol2001.wikimedia.org  fixes icinga config | 
  [production] | 
            
  
    | 
      
        2015-12-01
      
      §
     | 
  
    
  | 22:41 | 
  <mutante> | 
  stop/start ntp service on lvs2002 | 
  [production] | 
            
  | 20:54 | 
  <mutante> | 
  re-enabled puppet on cache::misc, replaced planet cert | 
  [production] | 
            
  | 20:19 | 
  <mutante> | 
  temp. disabling puppet on cache::misc  | 
  [production] | 
            
  | 18:14 | 
  <twentyafterfour> | 
  restarted apache on iridium to deploy https://phabricator.wikimedia.org/rPHEXd724c51a4144f7de548546875097da57ee1b38d7 | 
  [production] | 
            
  | 17:05 | 
  <moritzm> | 
  restarting cassandra instances on restbase200[3-6] (to effect openjdk security updates and a few depending libs) | 
  [production] | 
            
  | 17:00 | 
  <moritzm> | 
  restarting cassandra instances on restbase2002 (to effect openjdk security updates and a few depending libs) | 
  [production] | 
            
  | 16:55 | 
  <moritzm> | 
  restarting cassandra instances on restbase2001 (to effect openjdk security updates and a few depending libs) | 
  [production] | 
            
  | 16:22 | 
  <thcipriani@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: SWAT: Enable VisualEditor for all accounts on eswiki. Part II [[gerrit:250474]] (duration: 00m 28s) | 
  [production] | 
            
  | 16:21 | 
  <thcipriani@tin> | 
  Synchronized dblists/visualeditor-default.dblist: SWAT: Enable VisualEditor for all accounts on eswiki. Part I [[gerrit:250474]] (duration: 01m 29s) | 
  [production] | 
            
  | 12:59 | 
  <godog> | 
  reimage mw2208 | 
  [production] | 
            
  | 10:50 | 
  <moritzm> | 
  restarting on cassandra instances on restbase1008 (to effect openjdk security updates and a few depending libs) | 
  [production] | 
            
  | 06:52 | 
  <l10nupdate@tin> | 
  ResourceLoader cache refresh completed at Tue Dec  1 06:52:40 UTC 2015 (duration 52m 39s) | 
  [production] | 
            
  | 02:26 | 
  <mwdeploy@tin> | 
  sync-l10n completed (1.27.0-wmf.7) (duration: 10m 14s) | 
  [production] | 
            
  | 02:18 | 
  <mutante> | 
  restarted hhvm on mw1147 | 
  [production] | 
            
  | 01:47 | 
  <ori@tin> | 
  Synchronized docroot and w: (no message) (duration: 00m 30s) | 
  [production] | 
            
  | 00:47 | 
  <catrope@tin> | 
  Synchronized php-1.27.0-wmf.7/extensions/VisualEditor: SWAT (duration: 00m 29s) | 
  [production] | 
            
  | 00:47 | 
  <catrope@tin> | 
  Synchronized php-1.27.0-wmf.7/resources/src/mediawiki/mediawiki.ForeignStructuredUpload.js: SWAT (duration: 00m 30s) | 
  [production] |