| 
      
        2015-04-16
      
      §
     | 
  
    
  | 18:38 | 
  <legoktm> | 
  creating "Maintenance script" account on all SUL wikis for globaluserpage | 
  [production] | 
            
  | 16:46 | 
  <csteipp> | 
  removed oauth-headers.php since that allowed stealing httponly cookies | 
  [production] | 
            
  | 16:44 | 
  <csteipp> | 
  Synchronized w: (no message) (duration: 00m 11s) | 
  [production] | 
            
  | 16:42 | 
  <kartik> | 
  Finished scap: Update ContentTranslation (duration: 22m 13s) | 
  [production] | 
            
  | 16:19 | 
  <kartik> | 
  Started scap: Update ContentTranslation | 
  [production] | 
            
  | 16:06 | 
  <legoktm> | 
  starting to run forceRenameUsers.php (SUL finalization) | 
  [production] | 
            
  | 15:39 | 
  <^d> | 
  restarting gerrit | 
  [production] | 
            
  | 15:09 | 
  <krenair> | 
  Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/203323 (duration: 00m 12s) | 
  [production] | 
            
  | 15:03 | 
  <krenair> | 
  Synchronized wmf-config: https://gerrit.wikimedia.org/r/#/c/203861/ - should be a no-op, just config file cleanup (duration: 00m 13s) | 
  [production] | 
            
  | 12:35 | 
  <akosiaris> | 
  uploaded etherpad-lite_1.4.1-2 on apt.wikimedia.org | 
  [production] | 
            
  | 11:58 | 
  <Krenair> | 
  restarted apache on silver, wikitech login seems to work again | 
  [production] | 
            
  | 11:56 | 
  <andrewbogott> | 
  disabling puppet on virt1000 so that I can prevent a questionable cron (purging tokens from the keystone db) from running while I sleep. | 
  [production] | 
            
  | 04:00 | 
  <legoktm> | 
  Synchronized php-1.26wmf2/includes/DefaultSettings.php: The 'spambot_username' message is a reserved username (duration: 00m 11s) | 
  [production] | 
            
  | 03:29 | 
  <legoktm> | 
  Synchronized php-1.26wmf1/includes/DefaultSettings.php: The 'spambot_username' message is a reserved username (duration: 00m 12s) | 
  [production] | 
            
  | 03:07 | 
  <bd808> | 
  Updated iegreview to e126f7c (Fix aggregated reports to work on the new reviews system) | 
  [production] | 
            
  | 02:56 | 
  <LocalisationUpdate> | 
  completed (1.26wmf2) at 2015-04-16 02:55:08+00:00 | 
  [production] | 
            
  | 02:52 | 
  <l10nupdate> | 
  Synchronized php-1.26wmf2/cache/l10n: (no message) (duration: 04m 38s) | 
  [production] | 
            
  | 02:34 | 
  <legoktm> | 
  starting forceRenameUsers.php (SUL finalization) on non-test*wikis | 
  [production] | 
            
  | 02:33 | 
  <LocalisationUpdate> | 
  completed (1.26wmf1) at 2015-04-16 02:32:26+00:00 | 
  [production] | 
            
  | 02:29 | 
  <l10nupdate> | 
  Synchronized php-1.26wmf1/cache/l10n: (no message) (duration: 05m 57s) | 
  [production] | 
            
  | 02:24 | 
  <andrewbogott> | 
  but the ‘token’ table is still too big to manage | 
  [production] | 
            
  | 02:24 | 
  <andrewbogott> | 
  restarted mysql on virt1000 because keystone was stuck.  It seems to have helped, eventually | 
  [production] | 
            
  | 02:24 | 
  <andrewbogott> | 
  restarted keystone and nova-scheduler in a failed attempt to unstick things | 
  [production] | 
            
  | 02:23 | 
  <andrewbogott> | 
  testing the log by logging a test | 
  [production] | 
            
  
    | 
      
        2015-04-15
      
      §
     | 
  
    
  | 18:25 | 
  <legoktm> | 
  running forceRenameUsers.php (SUL finalization) on test* wikis | 
  [production] | 
            
  | 17:15 | 
  <legoktm> | 
  running migrateAccount.php --auto (CentralAuth) | 
  [production] | 
            
  | 16:49 | 
  <ottomata> | 
  rebooting analyics1020 | 
  [production] | 
            
  | 16:45 | 
  <nuria> | 
  restarted eventlogging && deployed d241d75ee2fab554bc47cf8d1ba83f5df2130633 | 
  [production] | 
            
  | 16:29 | 
  <demon> | 
  Synchronized php-1.26wmf1/extensions/CentralAuth/: (no message) (duration: 00m 13s) | 
  [production] | 
            
  | 16:22 | 
  <gwicke> | 
  running revision render thin-out script on wikipedia HTML | 
  [production] | 
            
  | 15:35 | 
  <bblack> | 
  re-enabling puppet on caches, canary nodes were no-op \\o/ | 
  [production] | 
            
  | 15:27 | 
  <bblack> | 
  disabling puppet on caches JIC for https://gerrit.wikimedia.org/r/204068 merge | 
  [production] | 
            
  | 14:54 | 
  <legoktm> | 
  running deleteEmptyAccounts.php --fix on metawiki (CentralAuth) | 
  [production] | 
            
  | 13:54 | 
  <andrewbogott> | 
  purging expired keystone tokens on virt1000 | 
  [production] | 
            
  | 12:59 | 
  <andrewbogott> | 
  restarted pdns on virt1000 and labcontrol2001 to recover from the opendj restart | 
  [production] | 
            
  | 12:58 | 
  <akosiaris> | 
  restarted keystone, nova services on virt1000 | 
  [production] | 
            
  | 12:58 | 
  <akosiaris> | 
  restarted opendj on neptunium | 
  [production] | 
            
  | 12:58 | 
  <andrewbogott> | 
  dropping labswiki and labswiki_eqiad from mysql on virt1000 | 
  [production] | 
            
  | 11:20 | 
  <_joe_> | 
  restarted apache2 on virt1000, passenger gone to hell | 
  [production] | 
            
  | 10:31 | 
  <godog> | 
  bounce jobchron on mw1001 | 
  [production] | 
            
  | 10:30 | 
  <godog> | 
  restart keystone on virt1000 (#2) | 
  [production] | 
            
  | 08:12 | 
  <godog> | 
  bounce keystone on virt1000 | 
  [production] | 
            
  | 04:16 | 
  <ebernhardson> | 
  Synchronized wmf-config/CommonSettings.php: Bump flow cache version (duration: 00m 11s) | 
  [production] | 
            
  | 03:11 | 
  <LocalisationUpdate> | 
  completed (1.26wmf1) at 2015-04-15 03:10:08+00:00 | 
  [production] | 
            
  | 03:04 | 
  <l10nupdate> | 
  Synchronized php-1.26wmf1/cache/l10n: (no message) (duration: 09m 22s) | 
  [production] | 
            
  | 02:39 | 
  <LocalisationUpdate> | 
  completed (1.25wmf24) at 2015-04-15 02:37:59+00:00 | 
  [production] | 
            
  | 02:32 | 
  <l10nupdate> | 
  Synchronized php-1.25wmf24/cache/l10n: (no message) (duration: 09m 03s) | 
  [production] |