| 
      
        2013-03-15
      
      §
     | 
  
    
  | 23:24 | 
  <laner> | 
  synchronized wmf-config/CommonSettings.php  'Disabling aft5' | 
  [production] | 
            
  | 22:45 | 
  <asher> | 
  synchronized wmf-config/db-eqiad.php  'returning es1007,10 at lower weights' | 
  [production] | 
            
  | 22:35 | 
  <binasher> | 
  rebooted es1007,1010 for kernel upgrades | 
  [production] | 
            
  | 22:33 | 
  <asher> | 
  synchronized wmf-config/db-eqiad.php  'temporarily pulling es1007, es1010' | 
  [production] | 
            
  | 22:23 | 
  <Ryan_Lane> | 
  hid domain dropdown for account creation and login on wikitech | 
  [production] | 
            
  | 22:23 | 
  <Ryan_Lane> | 
  changed wikitech configuration to require email address on account creation | 
  [production] | 
            
  | 22:22 | 
  <Ryan_Lane> | 
  deployed change 54093 to wikitech | 
  [production] | 
            
  | 22:03 | 
  <hashar> | 
  Zuul : opened a bug about this week slowness, breakage:  https://bugzilla.wikimedia.org/show_bug.cgi?id=46176 | 
  [production] | 
            
  | 21:51 | 
  <aaron> | 
  synchronized php-1.21wmf11/includes  'deployed 08295adcdc4a4130954f52fb1eb5377638cd97bc' | 
  [production] | 
            
  | 21:36 | 
  <binasher> | 
  db59 -> mariadb 5.5.30 | 
  [production] | 
            
  | 21:29 | 
  <hashar> | 
  restarted Zuul with the safe version ff79197 | 
  [production] | 
            
  | 21:27 | 
  <hashar> | 
  downgrading Zuul to safe version ff79197 (aka stop doing `git remote update` on ref-update events) | 
  [production] | 
            
  | 21:25 | 
  <binasher> | 
  added new build of mariadb 5.5.30 to precise-wikimedia on brewster | 
  [production] | 
            
  | 21:20 | 
  <hashar> | 
  hard restarted Zuul. It was locked down again :/ | 
  [production] | 
            
  | 21:18 | 
  <mutante> | 
  installing package upgrades on gallium | 
  [production] | 
            
  | 21:14 | 
  <hashar> | 
  restarting Zuul, it is locked somehow. Definitely need to update it :( | 
  [production] | 
            
  | 21:09 | 
  <RobH> | 
  professor services seem back up | 
  [production] | 
            
  | 20:51 | 
  <RobH> | 
  professor shutting down for fan swap, short downtime (hopefully) | 
  [production] | 
            
  | 20:47 | 
  <hashar> | 
  integration.mediawiki.org is not redirecting to the new integration.wikimedia.org (apache conf is disabled). Patch in {{gerrit|54068}} | 
  [production] | 
            
  | 20:33 | 
  <mutante> | 
  puppet runs on neon broken due to mysql puppet issue - mysql-client-5.5 can not create alias mysql-client: object already exists at /var/lib/git/operations/puppet/manifests/mysql.pp:513 | 
  [production] | 
            
  | 20:31 | 
  <mutante> | 
  restarting pdns on ns0 | 
  [production] | 
            
  | 20:30 | 
  <mutante> | 
  DNS update - removing ipv4/ipv6/ipv6and4/results.labs.wm | 
  [production] | 
            
  | 19:37 | 
  <mutante> | 
  maerlant - re-adding 2620:0:862:1::80:2 to eth0, starting nginx | 
  [production] | 
            
  | 19:22 | 
  <sbernardin> | 
  taking down raskin for decommissioning | 
  [production] | 
            
  | 19:13 | 
  <mutante> | 
  maerlant - dpkg: error processing ganglia-monitor | backing up /var/www/ to tridge /data/other then deleting contents | removing sites-enabled/ipv6and4 | nginx can't bind to [2620:0:862:1::80:2]:443 | 
  [production] | 
            
  | 18:50 | 
  <mutante> | 
  installing more package upgrades on maerlant (wikimedia-lvs-realserver, libc6, nginx...) | 
  [production] | 
            
  | 18:46 | 
  <mutante> | 
  installing package upgrades on formey | 
  [production] | 
            
  | 18:44 | 
  <mutante> | 
  fixing broken dpkg/APT on maerlant - dpkg --configure -a , re-run puppet and let it install package upgrades | 
  [production] | 
            
  | 18:33 | 
  <mlitn> | 
  synchronized php-1.21wmf11/extensions/ArticleFeedbackv5/ | 
  [production] | 
            
  | 18:19 | 
  <mlitn> | 
  synchronized php-1.21wmf11/extensions/ArticleFeedbackv5/ | 
  [production] | 
            
  | 18:19 | 
  <notpeter> | 
  ran puppetd --enable on all mw hosts | 
  [production] | 
            
  | 18:15 | 
  <mlitn> | 
  synchronized php-1.21wmf11/extensions/ArticleFeedbackv5/ | 
  [production] | 
            
  | 18:07 | 
  <mlitn> | 
  Finished syncing Wikimedia installation... : Update ArticleFeedbackv5 to master | 
  [production] | 
            
  | 17:52 | 
  <mlitn> | 
  Started syncing Wikimedia installation... : Update ArticleFeedbackv5 to master | 
  [production] | 
            
  | 17:36 | 
  <hashar> | 
  jenkins: updated jenkins configuration to have its URL default to wikimedia.org domain instead of the legacy mediawiki.org | 
  [production] | 
            
  | 17:22 | 
  <mlitn> | 
  synchronized php-1.21wmf11/extensions/ArticleFeedbackv5/  'Update ArticleFeedbackv5 to master' | 
  [production] | 
            
  | 17:21 | 
  <hashar> | 
  gallium : reloading Zuul so it reports the status url with the wikimedia.org domain | 
  [production] | 
            
  | 17:19 | 
  <hashar> | 
  integration website going down while we deploy a new layout. | 
  [production] |