| 
      
        2015-07-10
      
      §
     | 
  
    
  | 17:32 | 
  <demon> | 
  Synchronized wmf-config/CommonSettings.php: prevent race condition on writing settings (duration: 00m 13s) | 
  [production] | 
            
  | 17:26 | 
  <moritzm> | 
  installed python security updates on mc* | 
  [production] | 
            
  | 17:25 | 
  <Coren> | 
  rebooting labstore2001 (experiments with the new raid setup caused the mapper table to fill) | 
  [production] | 
            
  | 16:35 | 
  <mobrovac> | 
  restbase deploying hotfix for T105509 | 
  [production] | 
            
  | 15:29 | 
  <mobrovac> | 
  restbase restarted restabse on restbase1004 | 
  [production] | 
            
  | 15:25 | 
  <godog> | 
  bounce cassandra on restbae1004 | 
  [production] | 
            
  | 13:43 | 
  <godog> | 
  bounce cassandra on restbae1004 | 
  [production] | 
            
  | 13:37 | 
  <_joe_> | 
  temporarily repooled mw1031 | 
  [production] | 
            
  | 12:40 | 
  <godog> | 
  bounce cassandra on restbae1004 | 
  [production] | 
            
  | 07:43 | 
  <godog> | 
  reimage ms-be2013 T105213 | 
  [production] | 
            
  | 04:36 | 
  <LocalisationUpdate> | 
  ResourceLoader cache refresh completed at Fri Jul 10 04:36:49 UTC 2015 (duration 36m 48s) | 
  [production] | 
            
  | 04:33 | 
  <springle> | 
  Synchronized wmf-config/db-eqiad.php: depool db1037; repool db1030 (revert below) (duration: 00m 12s) | 
  [production] | 
            
  | 04:28 | 
  <springle> | 
  Synchronized wmf-config/db-eqiad.php: repool db1037; depool db1030 (duration: 00m 13s) | 
  [production] | 
            
  | 03:14 | 
  <mutante> | 
  re-enabling puppet on tools-exec-1213, working around adminbot package install fail | 
  [production] | 
            
  | 02:59 | 
  <elee> | 
  please log this with the year | 
  [production] | 
            
  | 02:53 | 
  <andrewbogott> | 
  testing the log by logging a test | 
  [production] | 
            
  | 01:50 | 
  <gwicke> | 
  bounced cassandra on restbase1004 | 
  [production] | 
            
  | 01:38 | 
  <jgage> | 
  cassandra restarted on restbase1004 | 
  [production] | 
            
  | 00:39 | 
  <urandom> | 
  starting restbase1004 | 
  [production] | 
            
  | 00:35 | 
  <krenair> | 
  Synchronized php-1.26wmf13/extensions/VisualEditor/modules/ve-mw/ui/inspectors/ve.ui.MWLinkAnnotationInspector.js: https://gerrit.wikimedia.org/r/#/c/223983/ (duration: 00m 12s) | 
  [production] | 
            
  | 00:15 | 
  <hoo> | 
  Updated WikibaseQualityConstraints data on wikidata (wikidatawiki.wbqc_constraints) | 
  [production] | 
            
  
    | 
      
        2015-07-09
      
      §
     | 
  
    
  | 23:41 | 
  <legoktm> | 
  deployed patch for T105413 | 
  [production] | 
            
  | 23:07 | 
  <gwicke> | 
  bounced cassandra on restbase1004 | 
  [production] | 
            
  | 23:02 | 
  <catrope> | 
  Synchronized wmf-config/CommonSettings.php: TitleBlacklist: Don't block account auto-creation (duration: 00m 13s) | 
  [production] | 
            
  | 22:09 | 
  <oblivian> | 
  Synchronized wmf-config/PoolCounterSettings-eqiad.php: I don't think we want to keep poolcounter running on an imagescaler (duration: 00m 12s) | 
  [production] | 
            
  | 21:30 | 
  <tgr> | 
  Synchronized php-1.26wmf13/extensions/OAuth/api/MWOAuthAPI.setup.php: no canonical redirects for requests with OAuth headers (duration: 00m 12s) | 
  [production] | 
            
  | 21:05 | 
  <tgr> | 
  backporting https://gerrit.wikimedia.org/r/#/c/223952/- fixes OAuth which is broken for 1.26wmf13 | 
  [production] | 
            
  | 20:48 | 
  <gwicke> | 
  temporarily disabled puppet on cassandra nodes while tweaking settings | 
  [production] | 
            
  | 19:53 | 
  <legoktm> | 
  manually fixing global merge of Yuvipanda->YuviPanda (T104686) | 
  [production] | 
            
  | 19:04 | 
  <gwicke> | 
  bounced cassandra on restbase1004 | 
  [production] | 
            
  | 18:29 | 
  <twentyafterfour> | 
  rebuilt wikiversions.cdb and synchronized wikiversions files: all wikis to 1.26wmf13 | 
  [production] | 
            
  | 17:54 | 
  <gwicke> | 
  bounced restbase on restbase1005 | 
  [production] | 
            
  | 17:32 | 
  <ori> | 
  installed poolcounter on mw1154 | 
  [production] | 
            
  | 17:31 | 
  <ori> | 
  Synchronized wmf-config/PoolCounterSettings-eqiad.php: (no message) (duration: 00m 12s) | 
  [production] | 
            
  | 17:22 | 
  <cmjohnson1> | 
  shutting down helium for a few minutes to move within the same row | 
  [production] | 
            
  | 16:53 | 
  <gwicke> | 
  bounced cassandra on restbase1004 | 
  [production] | 
            
  | 16:48 | 
  <godog> | 
  reboot ms-be2013 T105213 | 
  [production] | 
            
  | 16:38 | 
  <gwicke> | 
  bounced cassandra on restbase1006 | 
  [production] | 
            
  | 16:07 | 
  <_joe_> | 
  repooling mw1152 | 
  [production] | 
            
  | 15:57 | 
  <godog> | 
  restart cassandra on restbase1002 | 
  [production] | 
            
  | 15:34 | 
  <gwicke> | 
  bounced cassandra on restbase1004 | 
  [production] | 
            
  | 15:24 | 
  <krenair> | 
  Synchronized php-1.26wmf12/extensions/ContentTranslation: https://gerrit.wikimedia.org/r/#/c/223739/ (duration: 00m 12s) | 
  [production] | 
            
  | 15:23 | 
  <krenair> | 
  Synchronized php-1.26wmf13/extensions/ContentTranslation: https://gerrit.wikimedia.org/r/#/c/223737/ (duration: 00m 12s) | 
  [production] | 
            
  | 15:23 | 
  <krenair> | 
  Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/223742/ (duration: 00m 12s) | 
  [production] | 
            
  | 15:09 | 
  <gwicke> | 
  bounced cassandra on restbase1004 | 
  [production] | 
            
  | 14:44 | 
  <gwicke> | 
  re-enabled compaction throttling (60mb/s) on cassandra nodes | 
  [production] | 
            
  | 14:44 | 
  <bblack> | 
  reprepro: jessie-wikimedia/backports openssl pkg, 1.0.2c-1 => 1.0.2d-1~wmf1 | 
  [production] | 
            
  | 14:29 | 
  <_joe_> | 
  reimaging mw1152 for wiping any leftover local hacks. Depooling, scheduling downtime | 
  [production] | 
            
  | 14:28 | 
  <moritzm> | 
  installed python-django security updates on labmon, netmon and californium | 
  [production] | 
            
  | 14:24 | 
  <godog> | 
  really upgrade python-django on graphite2001 | 
  [production] |