| 
      
        2014-09-08
      
      §
     | 
  
    
  | 23:59 | 
  <ori> | 
  restarted rsync on mw1070 to unblock scap | 
  [production] | 
            
  | 23:40 | 
  <ebernhardson> | 
  Started scap: Repeat SWAT scap deployment due to possible sync-common failure | 
  [production] | 
            
  | 23:39 | 
  <ebernhardson> | 
  Finished scap: SWAT deploy updates to Flow, Echo and Thanks (duration: 24m 00s) | 
  [production] | 
            
  | 23:34 | 
  <mutante> | 
  disabled mw1070 in pybal because it refused sync | 
  [production] | 
            
  | 23:31 | 
  <ebernhardson> | 
  scap failed to connect to mw1070.  Repeated message: rsync: failed to connect to mw1070.eqiad.wmnet (10.64.16.50): Connection refused (111) | 
  [production] | 
            
  | 23:15 | 
  <ebernhardson> | 
  Started scap: SWAT deploy updates to Flow, Echo and Thanks | 
  [production] | 
            
  | 23:03 | 
  <ebernhardson> | 
  Synchronized wmf-config/InitialiseSettings.php: gerrit:159089 Enable $wgContentHandlerUseDB on mediawikiwiki, testwiki, & test2wiki (duration: 00m 05s) | 
  [production] | 
            
  | 20:14 | 
  <subbu> | 
  deployed Parsoid ce108cb5 | 
  [production] | 
            
  | 18:01 | 
  <demon> | 
  Synchronized php-1.24wmf19/extensions/Wikidata: Updating Wikidata to f1d2110 (duration: 00m 09s) | 
  [production] | 
            
  | 17:19 | 
  <mutante> | 
  disabled notifications for puppet freshness on neon | 
  [production] | 
            
  | 16:19 | 
  <demon> | 
  Synchronized wmf-config/InitialiseSettings.php: svwiki: Cirrus as primary (duration: 00m 04s) | 
  [production] | 
            
  | 15:42 | 
  <manybubbles> | 
  Synchronized php-1.24wmf20/extensions/Wikidata/: SWAT update wikidata to fix add links widget (duration: 00m 06s) | 
  [production] | 
            
  | 15:32 | 
  <manybubbles> | 
  Synchronized php-1.24wmf20/extensions/LiquidThreads/: SWAT update liquidthreads to fix some missing images (duration: 00m 04s) | 
  [production] | 
            
  | 15:28 | 
  <manybubbles> | 
  15:13:53 Synchronized php-1.24wmf19/extensions/WikiLove/: SWAT fix for WikiLove (duration: 00m 04s) | 
  [production] | 
            
  | 15:28 | 
  <manybubbles> | 
  this is the missing log: | 
  [production] | 
            
  | 15:27 | 
  <manybubbles> | 
  sync logging was down so it missed some syncing I just did. | 
  [production] | 
            
  | 15:25 | 
  <manybubbles> | 
  Synchronized php-1.24wmf20/extensions/WikiLove/: (no message) (duration: 00m 05s) | 
  [production] | 
            
  | 15:20 | 
  <manybubbles> | 
  Synchronized wmf-config: SWAT another cirrus setting update (duration: 00m 04s) | 
  [production] | 
            
  | 15:10 | 
  <manybubbles> | 
  Synchronized wmf-config: SWAT finish updating Cirrus settings (duration: 00m 05s) | 
  [production] | 
            
  | 15:10 | 
  <manybubbles> | 
  Synchronized wmf-config/InitialiseSettings.php: SWAT update some cirrus settings (duration: 00m 04s) | 
  [production] | 
            
  | 15:10 | 
  <cmjohnson1> | 
  shutting down neon for memory upgrade  | 
  [production] | 
            
  | 14:41 | 
  <springle> | 
  Synchronized wmf-config/db-eqiad.php: repool db1072 (duration: 00m 09s) | 
  [production] | 
            
  | 12:23 | 
  <springle> | 
  Synchronized wmf-config/db-eqiad.php: repool db1073, depool db1072 (duration: 00m 06s) | 
  [production] | 
            
  | 11:07 | 
  <springle> | 
  Synchronized wmf-config/db-eqiad.php: depool db1073wq (duration: 00m 09s) | 
  [production] | 
            
  | 10:55 | 
  <_joe_> | 
  re-enabled puppet, the change results in a no-op as expected | 
  [production] | 
            
  | 10:42 | 
  <_joe_> | 
  disabling puppet on all appservers while updating apache config. | 
  [production] | 
            
  | 04:58 | 
  <springle> | 
  Synchronized wmf-config/db-eqiad.php: move enwiki api traffic to db1051/db1066 (duration: 00m 09s) | 
  [production] | 
            
  | 03:37 | 
  <LocalisationUpdate> | 
  ResourceLoader cache refresh completed at Mon Sep  8 03:36:13 UTC 2014 (duration 36m 12s) | 
  [production] | 
            
  | 02:45 | 
  <LocalisationUpdate> | 
  completed (1.24wmf20) at 2014-09-08 02:44:47+00:00 | 
  [production] | 
            
  | 02:33 | 
  <LocalisationUpdate> | 
  completed (1.24wmf19) at 2014-09-08 02:32:37+00:00 | 
  [production] | 
            
  | 02:21 | 
  <LocalisationUpdate> | 
  completed (1.24wmf15) at 2014-09-08 02:19:52+00:00 | 
  [production] | 
            
  
    | 
      
        2014-09-07
      
      §
     | 
  
    
  | 23:35 | 
  <Tim> | 
  upgrading liblua everywhere | 
  [production] | 
            
  | 20:36 | 
  <ori> | 
  mw1017: upgraded HHVM from 3.3-dev+20140728+wmf5 to 3.3-dev+20140728+wmf6 | 
  [production] | 
            
  | 15:12 | 
  <apergos> | 
  manually changed /etc/hosts entry on analytics1004 from having "analyticas1004.eqiad.wmnet" to "analytics1004.eqiad.wmnet" | 
  [production] | 
            
  | 06:15 | 
  <godog> | 
  powercycle ms-be1005, not even responsive on console | 
  [production] | 
            
  | 03:31 | 
  <LocalisationUpdate> | 
  ResourceLoader cache refresh completed at Sun Sep  7 03:29:51 UTC 2014 (duration 29m 50s) | 
  [production] | 
            
  | 02:43 | 
  <LocalisationUpdate> | 
  completed (1.24wmf20) at 2014-09-07 02:42:12+00:00 | 
  [production] | 
            
  | 02:31 | 
  <LocalisationUpdate> | 
  completed (1.24wmf19) at 2014-09-07 02:30:15+00:00 | 
  [production] | 
            
  | 02:19 | 
  <LocalisationUpdate> | 
  completed (1.24wmf15) at 2014-09-07 02:17:44+00:00 | 
  [production] |