| 
      
        2020-09-11
      
      §
     | 
  
    
  | 01:29 | 
  <dzahn@cumin1001> | 
  END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) | 
  [production] | 
            
  | 01:29 | 
  <dzahn@cumin1001> | 
  START - Cookbook sre.hosts.downtime | 
  [production] | 
            
  | 01:29 | 
  <dzahn@cumin1001> | 
  END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) | 
  [production] | 
            
  | 01:29 | 
  <dzahn@cumin1001> | 
  START - Cookbook sre.hosts.downtime | 
  [production] | 
            
  | 01:24 | 
  <milimetric@deploy1001> | 
  Started deploy [analytics/refinery@6057f20]: Simple hql syntax fix | 
  [production] | 
            
  | 00:41 | 
  <milimetric@deploy1001> | 
  Finished deploy [analytics/refinery@7f5a6ca] (thin): Regular analytics weekly train THIN [analytics/refinery@7f5a6ca] (duration: 00m 08s) | 
  [production] | 
            
  | 00:41 | 
  <milimetric@deploy1001> | 
  Started deploy [analytics/refinery@7f5a6ca] (thin): Regular analytics weekly train THIN [analytics/refinery@7f5a6ca] | 
  [production] | 
            
  | 00:40 | 
  <milimetric@deploy1001> | 
  Finished deploy [analytics/refinery@7f5a6ca]: Regular analytics weekly train [analytics/refinery@7f5a6ca] (duration: 08m 25s) | 
  [production] | 
            
  | 00:38 | 
  <mutante> | 
  generating mcrouter certs for parse2001 - parse2019 - mcrouter_generate_certs on puppetmaster1001 (T247441) | 
  [production] | 
            
  | 00:32 | 
  <dzahn@cumin1001> | 
  END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) | 
  [production] | 
            
  | 00:31 | 
  <dzahn@cumin1001> | 
  START - Cookbook sre.hosts.downtime | 
  [production] | 
            
  | 00:31 | 
  <dzahn@cumin1001> | 
  END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) | 
  [production] | 
            
  | 00:31 | 
  <milimetric@deploy1001> | 
  Started deploy [analytics/refinery@7f5a6ca]: Regular analytics weekly train [analytics/refinery@7f5a6ca] | 
  [production] | 
            
  | 00:31 | 
  <dzahn@cumin1001> | 
  START - Cookbook sre.hosts.downtime | 
  [production] | 
            
  | 00:31 | 
  <dzahn@cumin1001> | 
  END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) | 
  [production] | 
            
  | 00:31 | 
  <dzahn@cumin1001> | 
  START - Cookbook sre.hosts.downtime | 
  [production] | 
            
  | 00:31 | 
  <dzahn@cumin1001> | 
  END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) | 
  [production] | 
            
  | 00:31 | 
  <dzahn@cumin1001> | 
  START - Cookbook sre.hosts.downtime | 
  [production] | 
            
  | 00:01 | 
  <pt1979@cumin2001> | 
  END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) | 
  [production] | 
            
  | 00:00 | 
  <pt1979@cumin2001> | 
  START - Cookbook sre.hosts.downtime | 
  [production] | 
            
  
    | 
      
        2020-09-10
      
      §
     | 
  
    
  | 23:44 | 
  <ejegg> | 
  updated payments-wiki from e41ab173e0 to 3c073a6a56 | 
  [production] | 
            
  | 23:14 | 
  <pt1979@cumin2001> | 
  END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) | 
  [production] | 
            
  | 23:11 | 
  <pt1979@cumin2001> | 
  START - Cookbook sre.hosts.downtime | 
  [production] | 
            
  | 22:50 | 
  <jhuneidi@deploy1001> | 
  helmfile [eqiad] Ran 'sync' command on namespace 'blubberoid' for release 'production' . | 
  [production] | 
            
  | 22:43 | 
  <jhuneidi@deploy1001> | 
  helmfile [codfw] Ran 'sync' command on namespace 'blubberoid' for release 'production' . | 
  [production] | 
            
  | 22:33 | 
  <pt1979@cumin2001> | 
  END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) | 
  [production] | 
            
  | 22:31 | 
  <jhuneidi@deploy1001> | 
  helmfile [staging] Ran 'sync' command on namespace 'blubberoid' for release 'staging' . | 
  [production] | 
            
  | 22:31 | 
  <pt1979@cumin2001> | 
  START - Cookbook sre.hosts.downtime | 
  [production] | 
            
  | 22:11 | 
  <ejegg> | 
  updated payments-wiki from be81063168 to e41ab173e0 | 
  [production] | 
            
  | 22:06 | 
  <mutante> | 
  added mcrouter cert for parse2020, ran mcrouter_generate_certs | 
  [production] | 
            
  | 21:51 | 
  <pt1979@cumin2001> | 
  END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) | 
  [production] | 
            
  | 21:49 | 
  <pt1979@cumin2001> | 
  START - Cookbook sre.hosts.downtime | 
  [production] | 
            
  | 21:09 | 
  <pt1979@cumin2001> | 
  END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) | 
  [production] | 
            
  | 21:07 | 
  <pt1979@cumin2001> | 
  START - Cookbook sre.hosts.downtime | 
  [production] | 
            
  | 20:52 | 
  <dzahn@cumin1001> | 
  END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) | 
  [production] | 
            
  | 20:52 | 
  <dzahn@cumin1001> | 
  START - Cookbook sre.hosts.downtime | 
  [production] | 
            
  | 20:25 | 
  <jhuneidi@deploy1001> | 
  rebuilt and synchronized wikiversions files: all wikis to 1.36.0-wmf.8 | 
  [production] | 
            
  | 20:23 | 
  <pt1979@cumin2001> | 
  END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) | 
  [production] | 
            
  | 20:21 | 
  <pt1979@cumin2001> | 
  START - Cookbook sre.hosts.downtime | 
  [production] | 
            
  | 20:20 | 
  <longma> | 
  correction: T257976 - 1.36.0-wmf.8 to all wikis | 
  [production] | 
            
  | 20:20 | 
  <longma> | 
  deploying 1.36.0-wmf.8 to all wikis | 
  [production] | 
            
  | 20:02 | 
  <krinkle@deploy1001> | 
  Synchronized php-1.36.0-wmf.8/includes/resourceloader/ResourceLoaderSkinModule.php: Ibe2c9f8d024f6 (duration: 01m 05s) | 
  [production] | 
            
  | 19:44 | 
  <Urbanecm> | 
  End of [urbanecm@mwmaint2001 ~]$ mwscript updateCollation.php --wiki=trwiktionary --previous-collation=uppercase # T262163 | 
  [production] | 
            
  | 19:12 | 
  <mholloway-shell@deploy1001> | 
  Started restart [recommendation-api/deploy@db7fd80]: (no justification provided) | 
  [production] | 
            
  | 19:07 | 
  <Urbanecm> | 
  Start of [urbanecm@mwmaint2001 ~]$ mwscript updateCollation.php --wiki=trwiktionary --previous-collation=uppercase # T262163 | 
  [production] | 
            
  | 19:05 | 
  <urbanecm@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: 95d2b575c683a1c5c2972a9bf0cf3b87059fbd74: Set $wgCategoryCollation = uca-tr on trwiktionary (T262163) (duration: 01m 05s) | 
  [production] | 
            
  | 18:58 | 
  <Urbanecm> | 
  [urbanecm@mwmaint2001 ~]$ mwscript namespaceDupes.php --wiki=frwiktionary --fix # T262398 | 
  [production] | 
            
  | 18:58 | 
  <urbanecm@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: 09e487e76158026ba161acffad277928d2603891: Add a new namespace to frwiktionary (T262398) (duration: 01m 04s) | 
  [production] | 
            
  | 18:40 | 
  <urbanecm@deploy1001> | 
  Synchronized php-1.36.0-wmf.8/includes/EditPage.php: 824094428c5f41dc9eef7d65c8440dadda4d4dbd: EditPage: Fix member call on boolean when undo is impossible (T262463) (duration: 01m 03s) | 
  [production] | 
            
  | 18:37 | 
  <urbanecm@deploy1001> | 
  Synchronized php-1.36.0-wmf.6/includes/EditPage.php: 824094428c5f41dc9eef7d65c8440dadda4d4dbd: EditPage: Fix member call on boolean when undo is impossible (T262463) (duration: 01m 07s) | 
  [production] |