| 2017-09-07
      
      § | 
    
  | 07:20 | <oblivian@tin> | Synchronized rpc/RunSingleJob.php: Adding the RunSingleJob.php rpc endpoint for jobrunners (duration: 00m 56s) | [production] | 
            
  | 06:07 | <eileen> | civicrm updated from 73a24b4ec193586c597eb16da4c0494a640e36fd to c1ece1e0d97e6a4ea397bdb6c04f175bded0f4c4 | [production] | 
            
  | 03:13 | <eileen> | seconds behind=78 | [production] | 
            
  | 03:08 | <eileen> | seconds behind 55 | [production] | 
            
  | 02:52 | <l10nupdate@tin> | ResourceLoader cache refresh completed at Thu Sep  7 02:52:05 UTC 2017 (duration 7m 15s) | [production] | 
            
  | 02:44 | <l10nupdate@tin> | scap sync-l10n completed (1.30.0-wmf.17) (duration: 06m 15s) | [production] | 
            
  | 02:43 | <eileen> | restarting groupmember.load after hacking out cache delete calls | [production] | 
            
  | 02:40 | <eileen> | update civicrm  from 764bfe1f92cc8bbf8d7536744fb47b34ca7a4365 to 73a24b4ec193586c597eb16da4c0494a640e36fd | [production] | 
            
  | 02:24 | <l10nupdate@tin> | scap sync-l10n completed (1.30.0-wmf.16) (duration: 07m 23s) | [production] | 
            
  | 02:05 | <eileen> | replag 217 behind | [production] | 
            
  | 01:56 | <eileen> | replag 197 seconds | [production] | 
            
  | 01:47 | <eileen> | replag 23 seconds | [production] | 
            
  | 01:29 | <eileen> | replag 0 | [production] | 
            
  | 01:22 | <eileen> | replag 550 sec behind master | [production] | 
            
  | 01:13 | <eileen> | replag delay 422 | [production] | 
            
  | 01:12 | <eileen> | job killed | [production] | 
            
  | 00:49 | <eileen> | 8389 contacts now | [production] | 
            
  | 00:49 | <eileen> | 7971 contacts so far | [production] | 
            
  | 00:46 | <eileen> | started Omnigroupmember.load | [production] | 
            
  | 00:24 | <twentyafterfour> | phabricator update deployed | [production] | 
            
  | 00:12 | <twentyafterfour@tin> | Finished deploy [phabricator/deployment@c265c1a]: (no justification provided) (duration: 00m 24s) | [production] | 
            
  | 00:11 | <twentyafterfour@tin> | Started deploy [phabricator/deployment@c265c1a]: (no justification provided) | [production] | 
            
  | 00:07 | <twentyafterfour> | deploying phabricator update 2017-09-06 https://phabricator.wikimedia.org/project/view/2980/ | [production] | 
            
  
    | 2017-09-06
      
      § | 
    
  | 23:41 | <twentyafterfour@tin> | Synchronized php-1.30.0-wmf.16/extensions/WikimediaEvents/: Sync Change-Id: I7ae522155e67610d25b5857d7b3918559bce8bc7 to all webservers refs T174387 (duration: 00m 49s) | [production] | 
            
  | 23:28 | <twentyafterfour@tin> | Synchronized php-1.30.0-wmf.17/extensions/WikimediaEvents/: Sync Change-Id: I7ae522155e67610d25b5857d7b3918559bce8bc7 to wmf.17 refs T174387 (duration: 00m 49s) | [production] | 
            
  | 23:23 | <twentyafterfour@tin> | Synchronized wmf-config/CirrusSearch-common.php: CirrusSearch-common.php goes last (duration: 00m 48s) | [production] | 
            
  | 23:21 | <twentyafterfour@tin> | Synchronized wmf-config/InitialiseSettings.php: Now sync initializesettings (duration: 00m 49s) | [production] | 
            
  | 23:19 | <twentyafterfour@tin> | Synchronized wmf-config/CirrusSearch-rel-survey.php: sync the added file first  (duration: 00m 49s) | [production] | 
            
  | 23:17 | <twentyafterfour@tin> | scap failed: average error rate on 9/11 canaries increased by 10x (rerun with --force to override this check, see https://logstash.wikimedia.org/goto/2cc7028226a539553178454fc2f14459 for details) | [production] | 
            
  | 23:12 | <twentyafterfour@tin> | Synchronized wmf-config/: roll-back due to huge error rate spike (duration: 00m 51s) | [production] | 
            
  | 23:10 | <twentyafterfour@tin> | Synchronized wmf-config/: deploy config for CirrusSearch human relevancy survey. Change-Id: I272c69e5a3bb6e833fca59282142d6b237fd9e60 Bug: T174106 (duration: 00m 52s) | [production] | 
            
  | 22:18 | <no_justification> | prior deploy was no-op | [production] | 
            
  | 22:18 | <demon@tin> | Finished deploy [gerrit/gerrit@d4f9a77]: (no justification provided) (duration: 00m 07s) | [production] | 
            
  | 22:18 | <demon@tin> | Started deploy [gerrit/gerrit@d4f9a77]: (no justification provided) | [production] | 
            
  | 22:11 | <bsitzmann@tin> | Finished deploy [mobileapps/deploy@507a479]: Update mobileapps to 2cb6281 (T168848 T169277 T169274 T162179 T164033 T167921 T174698 T168848 T174808) (duration: 04m 53s) | [production] | 
            
  | 22:06 | <bsitzmann@tin> | Started deploy [mobileapps/deploy@507a479]: Update mobileapps to 2cb6281 (T168848 T169277 T169274 T162179 T164033 T167921 T174698 T168848 T174808) | [production] | 
            
  | 21:39 | <niharika29@tin> | Synchronized wmf-config/CommonSettings-labs.php: Config for ArticleCreationWorkflow T175054 (duration: 00m 50s) | [production] | 
            
  | 21:28 | <ejegg> | updated payments-wiki from bf97cd236bdec421082db8a8a782965de9c98358 to ed2e4811067ad3d74433e8d7f4944ae838af6e28 | [production] | 
            
  | 21:24 | <ejegg> | updated CiviCRM from f9c373ec42f0a7b078a3eb7b31385183dab79bdf to 764bfe1f92cc8bbf8d7536744fb47b34ca7a4365 | [production] | 
            
  | 21:17 | <ottomata> | rebooting kafka-jumbo1004 | [production] | 
            
  | 20:34 | <arlolra> | Updated Parsoid to f9d367ea (T169342) | [production] | 
            
  | 20:27 | <arlolra@tin> | Finished deploy [parsoid/deploy@f07ac8c]: Updating Parsoid to f9d367ea (duration: 08m 27s) | [production] | 
            
  | 20:18 | <arlolra@tin> | Started deploy [parsoid/deploy@f07ac8c]: Updating Parsoid to f9d367ea | [production] | 
            
  | 19:53 | <ottomata> | reimaging kafka-jumbo* with stretch | [production] | 
            
  | 19:48 | <chasemp> | reboot labvirt1018 | [production] | 
            
  | 18:50 | <maxsem@tin> | Synchronized wmf-config/: just to make sure... (duration: 00m 50s) | [production] | 
            
  | 18:48 | <maxsem@tin> | Synchronized wmf-config/CommonSettings.php: revert (duration: 00m 49s) | [production] | 
            
  | 18:33 | <maxsem@tin> | Synchronized wmf-config/CommonSettings.php: https://gerrit.wikimedia.org/r/#/c/370292/3 (duration: 00m 49s) | [production] | 
            
  | 18:25 | <maxsem@tin> | Synchronized wmf-config/CommonSettings-labs.php: https://gerrit.wikimedia.org/r/#/c/370291/2 (duration: 00m 49s) | [production] | 
            
  | 18:19 | <maxsem@tin> | Synchronized wmf-config/Wikibase.php: https://gerrit.wikimedia.org/r/#/c/376317/2 (duration: 00m 48s) | [production] |