| 2014-10-03
      
      § | 
    
  | 14:35 | <manybubbles> | Synchronized wmf-config/CirrusSearch-labs.php: Noop - just keeps beta config in sync (duration: 00m 04s) | [production] | 
            
  | 14:18 | <Jeff_Green> | launched iodine:/opt/otrs/bin/otrs.RebuildFulltextIndex.pl per bugzilla #64473 | [production] | 
            
  | 13:07 | <Reedy> | Updated minor_mime to varbinary(100) on image|filearchive|oldimage on foundationwiki | [production] | 
            
  | 12:31 | <springle> | Synchronized wmf-config/db-eqiad.php: depool es1004 (duration: 00m 06s) | [production] | 
            
  | 11:51 | <reedy> | Synchronized wmf-config/interwiki.cdb: Updating interwiki cache (duration: 00m 15s) | [production] | 
            
  | 11:49 | <aude> | Synchronized wmf-config/interwiki.cdb: Updating interwiki cache (duration: 00m 10s) | [production] | 
            
  | 11:37 | <springle> | shutdown db60 db68 db69 db71 db72 db73 db74 es4 es7 es10 | [production] | 
            
  | 10:43 | <mark> | Shutdown amaranth.toolserver.org's switchport on asw-d-pmtpa | [production] | 
            
  | 09:35 | <springle> | Synchronized wmf-config/db-eqiad.php: repool db1063 (duration: 00m 07s) | [production] | 
            
  | 08:31 | <akosiaris> | deleting snmp community from nas1-a, nas1-b. I guess librenms is going to start complaining | [production] | 
            
  | 08:20 | <akosiaris> | unexporting, offline, destroying /vol/home_pmtpa on nas1-a | [production] | 
            
  | 06:59 | <_joe_> | depooling mw1022, then reimaging it | [production] | 
            
  | 04:22 | <springle> | upgrade db1063 mariadb 10 | [production] | 
            
  | 04:16 | <LocalisationUpdate> | ResourceLoader cache refresh completed at Fri Oct  3 04:16:35 UTC 2014 (duration 16m 34s) | [production] | 
            
  | 04:04 | <springle> | Synchronized wmf-config/db-eqiad.php: depool db1063 (duration: 00m 08s) | [production] | 
            
  | 03:17 | <LocalisationUpdate> | completed (1.25wmf2) at 2014-10-03 03:17:28+00:00 | [production] | 
            
  | 02:44 | <LocalisationUpdate> | completed (1.25wmf1) at 2014-10-03 02:44:39+00:00 | [production] | 
            
  
    | 2014-10-02
      
      § | 
    
  | 23:15 | <maxsem> | Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/164483 (duration: 00m 03s) | [production] | 
            
  | 23:06 | <maxsem> | Synchronized php-1.25wmf2/extensions/MobileFrontend/: (no message) (duration: 00m 04s) | [production] | 
            
  | 22:53 | <reedy> | Synchronized wmf-config/: Experimentally enable vips for larger (>50MP) tiff files (duration: 00m 15s) | [production] | 
            
  | 22:38 | <mutante> | icinga_broken_due_to_missing_hostgroup_counter incremented | [production] | 
            
  | 21:43 | <demon> | Synchronized wmf-config/InitialiseSettings.php: Disabling prerendering of images from this mornings swat (duration: 00m 04s) | [production] | 
            
  | 21:33 | <bblack> | added LVS BGP config setup to cr[12]-codfw | [production] | 
            
  | 20:46 | <ori> | Synchronized php-1.24wmf22/extensions/NavigationTiming: Update NavigationTiming for cherry-picks (duration: 00m 03s) | [production] | 
            
  | 20:44 | <ori> | Synchronized php-1.25wmf1/extensions/NavigationTiming: Update NavigationTiming for cherry-picks (duration: 00m 04s) | [production] | 
            
  | 20:11 | <bblack> | stopping -> starting uwsgi/apache -type stuff on tungsten | [production] | 
            
  | 19:55 | <aude> | populated sites table for fawikivoyage | [production] | 
            
  | 19:27 | <bd808> | hosts that failed Trebuchet update of scap: virt0.wikimedia.org, fenari.wikimedia.org, mw1110.eqiad.wmnet, mw1053.eqiad.wmnet. mw1053.eqiad.wmnet only failed checkout | [production] | 
            
  | 19:23 | <bd808> | Trebuchet reports for scap sync "231/234 minions completed fetch; 230/234 minions completed checkout" Some stale entries need to be removed from Trebuchet redis cache | [production] | 
            
  | 19:21 | <bd808> | Updated scap to eff0d01 (Fix format specifier for error message) | [production] | 
            
  | 19:12 | <reedy> | Synchronized wmf-config/InitialiseSettings.php: fawikivoyage (duration: 00m 16s) | [production] | 
            
  | 19:12 | <reedy> | rebuilt wikiversions.cdb and synchronized wikiversions files: fawikivoyage | [production] | 
            
  | 19:11 | <reedy> | Synchronized database lists: fawikivoyage (duration: 00m 16s) | [production] | 
            
  | 18:47 | <mutante> | graceful'ed apache on mw1030,mw1164 | [production] | 
            
  | 18:44 | <reedy> | Purged l10n cache for 1.24wmf21 | [production] | 
            
  | 18:43 | <reedy> | rebuilt wikiversions.cdb and synchronized wikiversions files: group0 to 1.25wmf2 | [production] | 
            
  | 18:41 | <reedy> | rebuilt wikiversions.cdb and synchronized wikiversions files: wikipedias to 1.25wmf2 | [production] | 
            
  | 18:40 | <reedy> | Finished scap: testwiki to 1.25wmf2 (duration: 24m 19s) | [production] | 
            
  | 18:32 | <cmjohnson1> | replacing failed disk es1005 | [production] | 
            
  | 18:15 | <reedy> | Started scap: testwiki to 1.25wmf2 | [production] | 
            
  | 17:57 | <manybubbles> | going to try to restart lsearchd on the misc pool machines to see if that makes it responsive | [production] | 
            
  | 17:42 | <ejegg> | updated payments-wiki from 83464deed3b66da655ca5d1086852237c4793b71 to 9417bbd95057a87824be157dbbb5965a1f09d202 | [production] | 
            
  | 17:26 | <aaron> | Synchronized php-1.25wmf1/maintenance/findMissingFiles.php: aa2eb3c0de08256822a2b0c985ebb3a6145d28cd (duration: 00m 05s) | [production] | 
            
  | 16:30 | <ori> | graceful'd all apaches for I98bcdbfc7: mediawiki: add vhost_combined log format to apache2.conf | [production] | 
            
  | 15:32 | <demon> | Synchronized php-1.25wmf1/extensions/Wikidata: (no message) (duration: 00m 20s) | [production] | 
            
  | 15:32 | <demon> | Synchronized php-1.25wmf1/includes/jobqueue/jobs/ThumbnailRenderJob.php: (no message) (duration: 00m 05s) | [production] | 
            
  | 15:21 | <godog> | shutting down nfs1 | [production] | 
            
  | 15:15 | <demon> | Synchronized wmf-config/InitialiseSettings.php: (no message) (duration: 00m 06s) | [production] | 
            
  | 15:11 | <demon> | Synchronized wmf-config/CommonSettings.php: (no message) (duration: 00m 04s) | [production] | 
            
  | 15:11 | <demon> | Synchronized wmf-config/InitialiseSettings.php: (no message) (duration: 00m 04s) | [production] |