| 2014-05-29
      
      § | 
    
  | 20:05 | <anomie> | synchronized php-1.24wmf7/extensions/EducationProgram/includes/api/ApiListStudents.php  'Backport fix for [[bugzilla:65906]]' | [production] | 
            
  | 19:40 | <^d> | hewiki elastic index was missing geodata mappings. re-map + in place reindex failed spectacularly. rebuilding from scratch now. | [production] | 
            
  | 19:38 | <ori> | mw1163: mkdir -p /usr/local/apache/common-local && chown mwdeploy:mwdeploy /usr/local/apache/common-local | [production] | 
            
  | 19:09 | <cmjohnson1> | powering down mw1151 for disk replacement | [production] | 
            
  | 19:01 | <reedy> | rebuilt wikiversions.cdb and synchronized wikiversions files: group0 to 1.24wmf7 | [production] | 
            
  | 18:49 | <cmjohnson1> | removing mw1151 from pybal and dsh groups to replace disk and reinstall | [production] | 
            
  | 18:47 | <reedy> | rebuilt wikiversions.cdb and synchronized wikiversions files: Wikipedias to 1.24wmf6 take 3 | [production] | 
            
  | 18:45 | <reedy> | synchronized php-1.24wmf6/includes/api/ | [production] | 
            
  | 18:19 | <reedy> | rebuilt wikiversions.cdb and synchronized wikiversions files: rv that | [production] | 
            
  | 18:18 | <reedy> | rebuilt wikiversions.cdb and synchronized wikiversions files: Wikipedias to 1.24wmf6 | [production] | 
            
  | 15:50 | <reedy> | rebuilt wikiversions.cdb and synchronized wikiversions files: (no message) | [production] | 
            
  | 15:42 | <reedy> | Finished scap: testwiki to 1.24wmf7 and build l10n cache (duration: 24m 24s) | [production] | 
            
  | 15:17 | <reedy> | Started scap: testwiki to 1.24wmf7 and build l10n cache | [production] | 
            
  | 15:06 | <anomie> | synchronized php-1.24wmf6/extensions/VisualEditor/modules/ve-mw/  'SWAT: VisualEditor URL decoding and image alignment fixes. [[gerrit:135922]] [[gerrit:135946]]' | [production] | 
            
  | 04:27 | <springle> | synchronized README  'test sync-file' | [production] | 
            
  | 04:20 | <ori> | updated scap to 9ba9014: Partially revert "Convert sync-dir and sync-file to python" | [production] | 
            
  | 04:13 | <ori> | re-enabled puppet on tin | [production] | 
            
  | 03:59 | <LocalisationUpdate> | ResourceLoader cache refresh completed at Thu May 29 03:57:59 UTC 2014 (duration 57m 58s) | [production] | 
            
  | 03:55 | <ori> | Synchronized README: Debugging sync-file (duration: 00m 06s) | [production] | 
            
  | 03:52 | <springle> | db1009 mariadb 5.5.37 live trial with low load | [production] | 
            
  | 03:49 | <springle> | synchronized wmf-config/db-eqiad.php  'repool db1009 in s2, take #2' | [production] | 
            
  | 03:45 | <ori> | disabled puppet on tin and copied sync-common-file from mediawiki/tools/scap@8f2a8356c38 into /usr/local/bin to debug sync issue | [production] | 
            
  | 03:44 | <springle> | Synchronized wmf-config/db-eqiad.php: repool db1009 in s2 (duration: 00m 08s) | [production] | 
            
  | 03:19 | <springle> | Synchronized wmf-config/db-eqiad.php: repool db1009 in s2 (duration: 00m 08s) | [production] | 
            
  | 03:11 | <LocalisationUpdate> | completed (1.24wmf6) at 2014-05-29 03:10:15+00:00 | [production] | 
            
  | 02:36 | <LocalisationUpdate> | completed (1.24wmf5) at 2014-05-29 02:35:45+00:00 | [production] | 
            
  | 01:22 | <awight> | updated SmashPig from e9964bfec47b3796dab0a19a9545cc3abb23fde6 to 98b1f348aa55f6a3aac441db08a59ca309fade7a | [production] | 
            
  | 01:16 | <awight> | (rollback) | [production] | 
            
  | 01:16 | <awight> | updated SmashPig from 03015f3827fedea9d0f89c791604ad08ec97ba71 to e9964bfec47b3796dab0a19a9545cc3abb23fde6 | [production] | 
            
  | 01:04 | <awight> | update SmashPig from f64f79f13cf4ab560d0bb5bd69690c827a821629 to 03015f3827fedea9d0f89c791604ad08ec97ba71 | [production] | 
            
  
    | 2014-05-28
      
      § | 
    
  | 23:39 | <demon> | Synchronized wmf-config/CommonSettings.php: Including external CologneBlue/Modern skins, if they exist (duration: 00m 07s) | [production] | 
            
  | 22:27 | <awight> | updated crm from 65a433b5564f42c3aa4f310cd4bb938ae70f841d to 5b231163e9e880de5b9787d40b679a6723748aca | [production] | 
            
  | 22:27 | <awight> | updated tools from 1e8029544dc19a84f6d1adf2783266e16d19ef1f to d257e8445e028b758b1d1fa90c857667d4faac62 | [production] | 
            
  | 21:23 | <RoanKattouw> | Restarting Parsoid Varnishes per gwicke's request | [production] | 
            
  | 20:38 | <mutante> | enabling puppet on osmium | [production] | 
            
  | 20:18 | <hashar> | Jenkins: killed all phantomjs process on gallium. They were eating all available memory.  All three process were VisualEditor qunit tests. | [production] | 
            
  | 20:13 | <subbu> | deployed parsoid a234af8c0 (deploy sha f17506eb) | [production] | 
            
  | 20:12 | <hashar> | gallium (Jenkins master) sent to swap somehow :-( | [production] | 
            
  | 20:02 | <bd808> | Finished scap: no-op scap deleted.dblist (duration: 10m 40s) | [production] | 
            
  | 19:52 | <bd808|deploy> | Horrible log message; should be "no-op scap to test code changes" | [production] | 
            
  | 19:51 | <bd808> | Started scap: no-op scap deleted.dblist | [production] | 
            
  | 19:49 | <bd808> | Synchronized database lists: (no message) (duration: 00m 03s) | [production] | 
            
  | 19:48 | <bd808> | Synchronized robots.txt: Testing sync-file in php (duration: 00m 03s) | [production] | 
            
  | 19:08 | <bd808|deploy> | Symlinks for mergeCdbFileUpdates, mwversionsinuse, refreshCdbJsonFiles, scap-rebuild-cdbs, scap-recompile and sync-common on tin still pointing to /srv/scap/bin instead of /srv/deployment/scap/scap/bin | [production] | 
            
  | 19:06 | <robh> | mw1053 reinstalling | [production] | 
            
  | 18:54 | <bd808> | Synchronized robots.txt: Testing sync-file in php (duration: 00m 05s) | [production] | 
            
  | 18:29 | <mutante> | running puppet on jobrunners | [production] | 
            
  | 18:27 | <^d> | jobrunners back up now, should slowly catch back up | [production] | 
            
  | 18:24 | <bd808|deploy> | Scap updated to fd7e538; Trebuchet fetch and checkout failed for mw1053.eqiad.wmnet | [production] | 
            
  | 18:06 | <bd808|deploy> | Restarted logstash on logstash1001; log event volume suspiciously low for the last ~35 minutes | [production] |