| 
      
        2015-04-02
      
      §
     | 
  
    
  | 23:11 | 
  <mutante> | 
  temp. disabling puppet on restbase servers | 
  [production] | 
            
  | 22:50 | 
  <bd808> | 
  lots of SYSTEM ERROR responses from nutcracker on mw1147 | 
  [production] | 
            
  | 22:13 | 
  <greg-g> | 
  Account creation is broken/not working for either iOS or Android WP apps, investigation in -mobile | 
  [production] | 
            
  | 19:37 | 
  <ori> | 
  Synchronized wmf-config/InitialiseSettings.php: I3bbf2418d: Set $wgLogoHD for enwiki (duration: 00m 12s) | 
  [production] | 
            
  | 18:52 | 
  <mutante> | 
  running puppet on mw2095 - proxy error | 
  [production] | 
            
  | 18:43 | 
  <RoanKattouw> | 
  Running clearMessageBlobs.php | 
  [production] | 
            
  | 18:37 | 
  <demon> | 
  Synchronized wmf-config/CirrusSearch-labs.php: cleanups for labs, no-op (duration: 00m 12s) | 
  [production] | 
            
  | 18:37 | 
  <demon> | 
  Synchronized wmf-config/CirrusSearch-common.php: turn off "yay new search!!" msg. old news now (duration: 00m 11s) | 
  [production] | 
            
  | 17:15 | 
  <kartik> | 
  Synchronized php-1.25wmf23/extensions/ContentTranslation/modules/campaigns/ext.cx.campaigns.contributionsmenu.js: (no message) (duration: 00m 15s) | 
  [production] | 
            
  | 16:52 | 
  <kartik> | 
  Finished scap: Update ContentTranslation (duration: 40m 01s) | 
  [production] | 
            
  | 16:25 | 
  <godog> | 
  reload uwsgi on graphite1001 | 
  [production] | 
            
  | 16:12 | 
  <kartik> | 
  Started scap: Update ContentTranslation | 
  [production] | 
            
  | 15:51 | 
  <manybubbles> | 
  actually that last patch seems to be working too.  cool.  sweet.  still running the cirrus script just in case. | 
  [production] | 
            
  | 15:50 | 
  <manybubbles> | 
  last sync accidentally picked up 'Add 100/106 namespaces to be searched by default at frwiktionary' - that one might require a cirrus script to finish running before its working properly | 
  [production] | 
            
  | 15:49 | 
  <manybubbles> | 
  Synchronized wmf-config/InitialiseSettings.php: SWAT Set $wgRestrictDisplayTitle to false at cawikimedia (duration: 00m 11s) | 
  [production] | 
            
  | 15:42 | 
  <manybubbles> | 
  Synchronized wmf-config/InitialiseSettings.php: SWAT disable mobile ip editing at kowiki 2/2 (duration: 00m 12s) | 
  [production] | 
            
  | 15:42 | 
  <manybubbles> | 
  Synchronized wmf-config/CommonSettings.php: SWAT disable mobile ip editing at kowiki 1/2 (duration: 00m 11s) | 
  [production] | 
            
  | 15:40 | 
  <manybubbles> | 
  Synchronized php-1.25wmf23/extensions/OpenStackManager/: SWAT update openstackmanager extension (duration: 00m 14s) | 
  [production] | 
            
  | 15:36 | 
  <manybubbles> | 
  Synchronized php-1.25wmf24/extensions/OpenStackManager/: SWAT update openstackmanager extension (duration: 00m 11s) | 
  [production] | 
            
  | 15:18 | 
  <manybubbles> | 
  Synchronized wmf-config/InitialiseSettings.php: SWAT enable newarticle campaign on cawiki 3/3 (duration: 00m 14s) | 
  [production] | 
            
  | 15:17 | 
  <manybubbles> | 
  Synchronized wmf-config/CommonSettings-labs.php: SWAT enable newarticle campaign on cawiki 2/3 (duration: 00m 12s) | 
  [production] | 
            
  | 15:17 | 
  <manybubbles> | 
  Synchronized wmf-config/CommonSettings.php: SWAT enable newarticle campaign on cawiki 1/3 (duration: 00m 13s) | 
  [production] | 
            
  | 15:16 | 
  <manybubbles> | 
  ignore last log - its a noop failure on my part | 
  [production] | 
            
  | 15:16 | 
  <manybubbles> | 
  Synchronized wmf-config/CommonSettings.php: SWAT enable newarticle campaign on cawiki 1/3 (duration: 00m 12s) | 
  [production] | 
            
  | 15:15 | 
  <manybubbles> | 
  Synchronized php-1.25wmf23/includes/User.php: SWAT user preferences load from the master by default (duration: 00m 12s) | 
  [production] | 
            
  | 14:13 | 
  <hashar> | 
  Jenkins: migrated Zuul cloner on Precise labs slaves (100[1-4] to a version provided by a Debian package. Jobs console output should now shows <tt>Zuul version: 2.0.0-304-g685ca22-wmf1precise1</tt> | 
  [production] | 
            
  | 12:51 | 
  <andrewbogott> | 
  restarted opendj, pdns on neptunium, nembus, virt1000, labcontrol2001 | 
  [production] | 
            
  | 12:48 | 
  <paravoid> | 
  repooling esams | 
  [production] | 
            
  | 12:42 | 
  <paravoid> | 
  upgrading junos on mr1-esams | 
  [production] | 
            
  | 12:15 | 
  <mark> | 
  Shutting down cp3014 for 10G upgrade | 
  [production] | 
            
  | 11:02 | 
  <mark> | 
  Shutting down cp3012 for 10G upgrade | 
  [production] | 
            
  | 10:38 | 
  <_joe_> | 
  stopping pybal on lvs2003, running manually to help debugging | 
  [production] | 
            
  | 09:43 | 
  <paravoid> | 
  asw-d-eqiad: routing-engine backup switch FPC 8 -> FPC 4 | 
  [production] | 
            
  | 09:42 | 
  <paravoid> | 
  asw-d-eqiad: routing-engine backup switch FPC 7 -> FPC 5, master switchover FPC 8 -> FPC 5 | 
  [production] | 
            
  | 08:31 | 
  <LocalisationUpdate> | 
  ResourceLoader cache refresh completed at Thu Apr  2 08:30:42 UTC 2015 (duration 30m 41s) | 
  [production] | 
            
  | 07:48 | 
  <springle> | 
  on sync-file from tin: mw2213.codfw.wmnet returned [255]: Host key verification failed | 
  [production] | 
            
  | 07:46 | 
  <springle> | 
  Synchronized wmf-config/db-eqiad.php: repool db1027, warm up (duration: 00m 11s) | 
  [production] | 
            
  | 06:41 | 
  <bblack> | 
  re-depooled esams ... | 
  [production] | 
            
  | 06:34 | 
  <Jamesofur> | 
  manually merged Ximilian global account per request and account confirmation | 
  [production] | 
            
  | 06:13 | 
  <bblack> | 
  re-pooling esams (GTT event never happened AFAICS) | 
  [production] | 
            
  | 05:13 | 
  <springle> | 
  xtrabackup clone db1027 to db1035 | 
  [production] | 
            
  | 03:45 | 
  <springle> | 
  Synchronized wmf-config/db-eqiad.php: depool db1027 (duration: 00m 11s) | 
  [production] | 
            
  | 03:32 | 
  <springle> | 
  reinstall db1035, reclone data | 
  [production] | 
            
  | 03:18 | 
  <LocalisationUpdate> | 
  completed (1.25wmf24) at 2015-04-02 03:08:52+00:00 | 
  [production] |