| 
      
        2014-12-01
      
      §
     | 
  
    
  | 22:15 | 
  <awight> | 
  Synchronized php-1.25wmf10/extensions/DonationInterface: push DonationInterface translations (duration: 00m 09s) | 
  [production] | 
            
  | 22:15 | 
  <awight> | 
  Synchronized php-1.25wmf9/extensions/DonationInterface: push DonationInterface translations (duration: 00m 07s) | 
  [production] | 
            
  | 21:06 | 
  <K4-713> | 
  updated payments to 00415dd54bec2d4cf0a | 
  [production] | 
            
  | 20:08 | 
  <yurik> | 
  Synchronized php-1.25wmf10/extensions/ZeroPortal/: updatidng ZeroPortal to master (duration: 00m 05s) | 
  [production] | 
            
  | 20:06 | 
  <yurik> | 
  Synchronized php-1.25wmf10/extensions/ZeroBanner/: updatidng ZeroBanner to master (duration: 00m 06s) | 
  [production] | 
            
  | 20:05 | 
  <yurik> | 
  Synchronized php-1.25wmf9/extensions/ZeroPortal/: updatidng ZeroPortal to master (duration: 00m 05s) | 
  [production] | 
            
  | 20:04 | 
  <yurik> | 
  Synchronized php-1.25wmf9/extensions/ZeroBanner/: updatidng ZeroBanner to master (duration: 00m 08s) | 
  [production] | 
            
  | 20:00 | 
  <yurik> | 
  Synchronized mobilelanding.php: https://gerrit.wikimedia.org/r/#/c/175797/ (duration: 00m 06s) | 
  [production] | 
            
  | 19:43 | 
  <maxsem> | 
  Synchronized php-1.25wmf10/extensions/Popups/: https://gerrit.wikimedia.org/r/#/c/176715/ (duration: 00m 06s) | 
  [production] | 
            
  | 19:43 | 
  <maxsem> | 
  Synchronized php-1.25wmf9/extensions/Popups/: https://gerrit.wikimedia.org/r/#/c/176715/ (duration: 00m 05s) | 
  [production] | 
            
  | 19:41 | 
  <MaxSem> | 
  Stashed Tim's uncommitted tidy-related changes on tin | 
  [production] | 
            
  | 19:19 | 
  <K4-713> | 
  updated DjangoBannerStats to 3db799dc8705c728c | 
  [production] | 
            
  | 18:25 | 
  <bblack> | 
  ulsfo LVS updated for 'sh' for SSL as well | 
  [production] | 
            
  | 18:22 | 
  <bblack> | 
  eqiad+esams LVS back to normal, with new config for 'sh' for SSL | 
  [production] | 
            
  | 18:15 | 
  <bblack> | 
  ditto on pybal 'sh' stuff for esams | 
  [production] | 
            
  | 18:10 | 
  <bblack> | 
  stopping pybal on primary eqiad LVSes to test 'sh' change for SSL (already restarted for change on backup LVSes) | 
  [production] | 
            
  | 17:02 | 
  <andrewbogott> | 
  created empty jessie-wikimedia repo on Carbon | 
  [production] | 
            
  | 16:38 | 
  <demon> | 
  Synchronized wmf-config/CommonSettings.php: opensearchxml conditional include (duration: 00m 06s) | 
  [production] | 
            
  | 16:37 | 
  <anomie> | 
  Synchronized php-1.25wmf9/extensions/SyntaxHighlight_GeSHi/geshi/geshi.php: SWAT: Fix highly recursive number highlighting regex in GeSHi (duration: 00m 07s) | 
  [production] | 
            
  | 16:35 | 
  <anomie> | 
  Synchronized php-1.25wmf10/extensions/SyntaxHighlight_GeSHi/geshi/geshi.php: SWAT: Fix highly recursive number highlighting regex in GeSHi (duration: 00m 10s) | 
  [production] | 
            
  | 16:04 | 
  <demon> | 
  Synchronized wmf-config/abusefilter.php: (no message) (duration: 00m 05s) | 
  [production] | 
            
  | 16:04 | 
  <demon> | 
  Synchronized wmf-config/InitialiseSettings.php: (no message) (duration: 00m 05s) | 
  [production] | 
            
  | 15:49 | 
  <bd808> | 
  restarted logstash on logstash1001; log2udp events were not being processed | 
  [production] | 
            
  | 15:26 | 
  <_joe_> | 
  depooling mw1047-mw1052 | 
  [production] | 
            
  | 15:24 | 
  <_joe_> | 
  repooling mw1041-mw1046 | 
  [production] | 
            
  | 14:26 | 
  <_joe_> | 
  depooling mw1041-1046 | 
  [production] | 
            
  | 14:16 | 
  <_joe_> | 
  repooling mw1036-mw1040 | 
  [production] | 
            
  | 13:46 | 
  <_joe_> | 
  removing the same files from ocg1002,3 as well | 
  [production] | 
            
  | 13:44 | 
  <_joe_> | 
  removing cache files from ocg1001, when they're older than 3 days | 
  [production] | 
            
  | 09:55 | 
  <_joe_> | 
  reimaging mw1033-mw1040 to HHVM, depooling from the main pool now | 
  [production] | 
            
  | 09:31 | 
  <_joe_> | 
  upgrading hhvm to the latest version across the cluster | 
  [production] | 
            
  | 04:46 | 
  <tstarling> | 
  Synchronized php-1.25wmf10/includes/parser/MWTidy.php: change previously pulled but scap was apparently not run (duration: 00m 05s) | 
  [production] | 
            
  | 04:44 | 
  <tstarling> | 
  Synchronized php-1.25wmf9/includes/parser/MWTidy.php: change previously pulled but scap was apparently not run (duration: 00m 06s) | 
  [production] | 
            
  | 03:34 | 
  <LocalisationUpdate> | 
  ResourceLoader cache refresh completed at Mon Dec  1 03:34:57 UTC 2014 (duration 34m 56s) | 
  [production] | 
            
  | 02:17 | 
  <LocalisationUpdate> | 
  completed (1.25wmf10) at 2014-12-01 02:17:56+00:00 | 
  [production] | 
            
  | 02:17 | 
  <l10nupdate> | 
  Synchronized php-1.25wmf10/cache/l10n: (no message) (duration: 00m 03s) | 
  [production] | 
            
  | 02:10 | 
  <LocalisationUpdate> | 
  completed (1.25wmf9) at 2014-12-01 02:10:33+00:00 | 
  [production] | 
            
  | 02:10 | 
  <l10nupdate> | 
  Synchronized php-1.25wmf9/cache/l10n: (no message) (duration: 00m 01s) | 
  [production] | 
            
  | 00:51 | 
  <tstarling> | 
  Synchronized wmf-config/StartProfiler.php: (no message) (duration: 00m 05s) | 
  [production] | 
            
  
    | 
      
        2014-11-30
      
      §
     | 
  
    
  | 23:17 | 
  <qchris> | 
  Updated EventLogging to 19c23698bc03694017d764af33307d6f035fc224 (on 22:51) and restarted it | 
  [production] | 
            
  | 20:51 | 
  <qchris> | 
  restarted eventlogging mysql-m2-master consumer. It seems it could no longer write to the database. | 
  [production] | 
            
  | 19:17 | 
  <Krinkle> | 
  Disabling and relauching Gearman connection from Jenkins. | 
  [production] | 
            
  | 10:28 | 
  <oblivian> | 
  Synchronized wmf-config/jobqueue-eqiad.php: reverting to rdb1001 (duration: 00m 05s) | 
  [production] | 
            
  | 10:13 | 
  <mark> | 
  Rebooted asw-c4-eqiad | 
  [production] | 
            
  | 09:13 | 
  <_joe_> | 
  jobsqueues work again | 
  [production] | 
            
  | 09:08 | 
  <oblivian> | 
  Synchronized wmf-config/jobqueue-eqiad.php: changing the aggregator address as well (duration: 00m 05s) | 
  [production] | 
            
  | 07:27 | 
  <_joe_> | 
  restarted the jobrunner service on all jobrunners | 
  [production] | 
            
  | 07:15 | 
  <oblivian> | 
  Synchronized wmf-config/jobqueue-eqiad.php: (no message) (duration: 00m 05s) | 
  [production] | 
            
  | 05:50 | 
  <ori> | 
  3:50 UTC: switch asw-c-eqiad lost connectivity with cabinet C4. Impact: phabricator down; gap in web request logs and some perf monitoring. Job queue and Recent Changes stream OK b/c redundant servers are up. | 
  [production] | 
            
  | 03:41 | 
  <LocalisationUpdate> | 
  ResourceLoader cache refresh completed at Sun Nov 30 03:41:03 UTC 2014 (duration 41m 2s) | 
  [production] |