| 
      
        2013-06-18
      
      §
     | 
  
    
  | 18:49 | 
  <mutante> | 
  rebooting iron | 
  [production] | 
            
  | 18:44 | 
  <mutante> | 
  installing kernel and package upgrades on iron | 
  [production] | 
            
  | 18:32 | 
  <mwalker> | 
  updating payments cluster (listeners) from aa0039104e71417845ec14a5ad666c2de464cfab to 8ca174296d570b6fda365cab37b2222d81c47c3d for adyen USD capture bug | 
  [production] | 
            
  | 18:11 | 
  <greg-g> | 
  it's better than bad, it's good! | 
  [production] | 
            
  | 18:08 | 
  <RobH> | 
  log log log log | 
  [production] | 
            
  | 18:08 | 
  <mutante> | 
  LVS monitoring pages caused by neon recovering | 
  [production] | 
            
  | 18:07 | 
  <RobH> | 
  ignore eqiad based search errors for now, as tampa is hosting all search traffic while hardware is repaired | 
  [production] | 
            
  | 18:07 | 
  <RobH> | 
  search is based in tampa | 
  [production] | 
            
  | 17:39 | 
  <cmjohnson1> | 
  dns update | 
  [production] | 
            
  | 17:27 | 
  <cmjohnson1> | 
  wtp1008 is going down for main board replacement | 
  [production] | 
            
  | 17:08 | 
  <mutante> | 
  reloading squid frontends in squid_esams_text and eqiad.text | 
  [production] | 
            
  | 17:06 | 
  <mutante> | 
  reloading squid frontends in pmtpa.text | 
  [production] | 
            
  | 14:00 | 
  <cmjohnson1> | 
  powering down search1001-1024 to relocate to row c5 | 
  [production] | 
            
  | 08:25 | 
  <nikerabbit> | 
  synchronized wmf-config/InitialiseSettings.php  'ULS phase 2' | 
  [production] | 
            
  | 08:15 | 
  <nikerabbit> | 
  synchronized php-1.22wmf6/extensions/UniversalLanguageSelector/  'ULS to master' | 
  [production] | 
            
  | 08:13 | 
  <nikerabbit> | 
  synchronized php-1.22wmf7/extensions/UniversalLanguageSelector/  'ULS to master' | 
  [production] | 
            
  | 07:18 | 
  <ori-l> | 
  Force-confirmed e-mail for 160 accounts. full list in fluorine:/home/olivneh/users-forced-confirm-email-18-Jun-2013.list | 
  [production] | 
            
  | 07:11 | 
  <ori-l> | 
  Marking e-mail addresses of accounts created in last 24h with no password as "confirmed" on all wikis on which VE is enabled. | 
  [production] | 
            
  | 04:12 | 
  <ori-l> | 
  Manually marked the e-mail address of 54 enwiki users with e-mail address configured but no password (due to bug 49727) as confirmed. | 
  [production] | 
            
  | 03:40 | 
  <mflaschen> | 
  synchronized php-1.22wmf7/extensions/VisualEditor/VisualEditor.hooks.php  'Hotfix to fix account creation' | 
  [production] | 
            
  | 03:14 | 
  <mflaschen> | 
  synchronized php-1.22wmf6/extensions/VisualEditor/VisualEditor.hooks.php  'Emergency hotfix to fix account creation' | 
  [production] | 
            
  | 03:09 | 
  <mflaschen> | 
  synchronized php-1.22wmf6/extensions/VisualEditor/VisualEditor.php  'Emergency hotfix to fix account creation' | 
  [production] | 
            
  | 03:09 | 
  <superm401> | 
  Started hotfix for account creation on wmf6. | 
  [production] | 
            
  | 02:25 | 
  <LocalisationUpdate> | 
  ResourceLoader cache refresh completed at Tue Jun 18 02:25:42 UTC 2013 | 
  [production] | 
            
  | 02:13 | 
  <LocalisationUpdate> | 
  completed (1.22wmf6) at Tue Jun 18 02:13:24 UTC 2013 | 
  [production] | 
            
  | 02:07 | 
  <LocalisationUpdate> | 
  completed (1.22wmf7) at Tue Jun 18 02:07:31 UTC 2013 | 
  [production] | 
            
  | 01:03 | 
  <Krinkle> | 
  Gracefully reloading Zuul to deploy {{gerrit|I5695a3b988e9ec3138f}} | 
  [production] | 
            
  | 00:50 | 
  <mutante> | 
  reloading squid config on pmtpa.txt | 
  [production] | 
            
  | 00:47 | 
  <mutante> | 
  reloading squid config on esams text squids | 
  [production] | 
            
  | 00:39 | 
  <mwalker> | 
  updating the payments cluster (listeners) from 5f67ceddd9d495cffd22f6b303443aba58041d47 to aa0039104e71417845ec14a5ad666c2de464cfab for additional smashpig content | 
  [production] | 
            
  | 00:11 | 
  <mutante> | 
  touch /etc/exim4/defer_domains on streber | 
  [production] | 
            
  
    | 
      
        2013-06-17
      
      §
     | 
  
    
  | 23:51 | 
  <mutante> | 
  restarting exim on streber | 
  [production] | 
            
  | 23:51 | 
  <catrope> | 
  Finished syncing Wikimedia installation... : Updating VisualEditor to master | 
  [production] | 
            
  | 23:39 | 
  <catrope> | 
  Started syncing Wikimedia installation... : Updating VisualEditor to master | 
  [production] | 
            
  | 23:35 | 
  <^demon> | 
  restarting gerrit | 
  [production] | 
            
  | 23:30 | 
  <RoanKattouw> | 
  Running scap to update VisualEditor | 
  [production] | 
            
  | 23:11 | 
  <mutante> | 
  install package upgrades on streber | 
  [production] | 
            
  | 23:06 | 
  <olivneh> | 
  synchronized wmf-config/InitialiseSettings.php  'Change I95a8785c7 / Bug 49312: Add 'Programs' NS to MetaWiki' | 
  [production] | 
            
  | 23:04 | 
  <Krinkle> | 
  Gracefully reloading Zuul to deploy {{gerrit|I773e84e6f75512}} | 
  [production] | 
            
  | 22:21 | 
  <gwicke> | 
  updated Parsoid to b2a7b101e | 
  [production] | 
            
  | 22:19 | 
  <mutante> | 
  built package and upgrading to php-luasandbox 1.7-1 | 
  [production] | 
            
  | 19:37 | 
  <Jeff_Green> | 
  dist-upgrade and reboot indium | 
  [production] | 
            
  | 19:28 | 
  <gwicke> | 
  updated Parsoid to d1ae92720c4e | 
  [production] | 
            
  | 18:49 | 
  <hashar> | 
  upgrading packages on gallium | 
  [production] | 
            
  | 18:40 | 
  <py> | 
  synchronized wmf-config/db-pmtpa.php  'bookkeeping changes for pmtpa dbs' | 
  [production] | 
            
  | 18:15 | 
  <aaron> | 
  synchronized php-1.22wmf7/includes/db/DatabaseMysql.php  'ac676a2d83af283701dee111a28a52ea1730d657' | 
  [production] | 
            
  | 18:00 | 
  <reedy> | 
  rebuilt wikiversions.cdb and synchronized wikiversions files: everything non 'pedia to 1.22wmf7 | 
  [production] | 
            
  | 17:14 | 
  <notpeter> | 
  rebooting es7 for lvm snapshot cleanup (again) | 
  [production] | 
            
  | 17:05 | 
  <apergos> | 
  neon should be back in business | 
  [production] | 
            
  | 16:40 | 
  <apergos> | 
  neon  out of inodes again, clearing out /var/spool/snmpttp, will return to service shortly | 
  [production] |