| 
      
        2019-12-09
      
      ยง
     | 
  
    
  | 22:55 | 
  <brennen@deploy1001> | 
  rebuilt and synchronized wikiversions files: all wikis to 1.35.0-wmf.8 | 
  [production] | 
            
  | 22:54 | 
  <XenoRyet> | 
  updated civicrm from 7eab025ec0 to 09149e0427 | 
  [production] | 
            
  | 22:54 | 
  <shdubsh> | 
  restart prometheus on prometheus2004 -- T238807 | 
  [production] | 
            
  | 22:40 | 
  <sbassett> | 
  Deployed security patch for T192134 to wmf.8 | 
  [production] | 
            
  | 22:37 | 
  <sbassett> | 
  Deployed security patch for T192134 to wmf.5 | 
  [production] | 
            
  | 21:55 | 
  <mholloway-shell@deploy1001> | 
  Finished deploy [mobileapps/deploy@aa65057]: Update mobileapps to f9771ab (duration: 10m 39s) | 
  [production] | 
            
  | 21:48 | 
  <shdubsh> | 
  restart prometheus on prometheus2003 -- T238807 | 
  [production] | 
            
  | 21:45 | 
  <mholloway-shell@deploy1001> | 
  Started deploy [mobileapps/deploy@aa65057]: Update mobileapps to f9771ab | 
  [production] | 
            
  | 21:32 | 
  <ebernhardson@deploy1001> | 
  Finished deploy [wikimedia/discovery/analytics@08cfd70]: Set location of ivy cache for spark (duration: 00m 24s) | 
  [production] | 
            
  | 21:32 | 
  <ebernhardson@deploy1001> | 
  Started deploy [wikimedia/discovery/analytics@08cfd70]: Set location of ivy cache for spark | 
  [production] | 
            
  | 19:18 | 
  <Urbanecm> | 
  Morning SWAT done | 
  [production] | 
            
  | 19:18 | 
  <Urbanecm> | 
  Purge several logo files (T150618) | 
  [production] | 
            
  | 19:18 | 
  <Urbanecm> | 
  Run namespaceDupes.php for eswikisource (T240050) | 
  [production] | 
            
  | 19:16 | 
  <urbanecm@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: SWAT: 32da89f: Upload HD logos for en, fi and nl arbcom wikis (2/2, T150618) (duration: 01m 00s) | 
  [production] | 
            
  | 19:14 | 
  <urbanecm@deploy1001> | 
  Synchronized static/images/project-logos/: SWAT: 32da89f: Upload HD logos for en, fi and nl arbcom wikis (1/2, T150618) (duration: 01m 01s) | 
  [production] | 
            
  | 19:07 | 
  <urbanecm@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: SWAT: f984d18: Add aliases for Help and Project on eswikisource (T240050) (duration: 01m 00s) | 
  [production] | 
            
  | 19:01 | 
  <onimisionipe> | 
  continue osm-import on maps1004 - T239728 | 
  [production] | 
            
  | 18:37 | 
  <herron> | 
  enabling lvs for kibana-next elk7 upgrade environment, in case any alerts fire relating to this please disreagard them | 
  [production] | 
            
  | 18:19 | 
  <onimisionipe@deploy1001> | 
  Finished deploy [wdqs/wdqs@9f9190e]: New WDQS Build (duration: 09m 33s) | 
  [production] | 
            
  | 18:09 | 
  <onimisionipe@deploy1001> | 
  Started deploy [wdqs/wdqs@9f9190e]: New WDQS Build | 
  [production] | 
            
  | 18:09 | 
  <onimisionipe@deploy1001> | 
  Finished deploy [wdqs/wdqs@9f9190e]: New WDQS Build (duration: 03m 02s) | 
  [production] | 
            
  | 18:06 | 
  <onimisionipe@deploy1001> | 
  Started deploy [wdqs/wdqs@9f9190e]: New WDQS Build | 
  [production] | 
            
  | 18:01 | 
  <brennen@deploy1001> | 
  rebuilt and synchronized wikiversions files: Revert "group2 wikis to 1.35.0-wmf.5" | 
  [production] | 
            
  | 17:52 | 
  <brennen@deploy1001> | 
  rebuilt and synchronized wikiversions files: all wikis to 1.35.0-wmf.8 | 
  [production] | 
            
  | 17:09 | 
  <brennen@deploy1001> | 
  Synchronized php: group1 wikis to 1.35.0-wmf.8 (duration: 01m 00s) | 
  [production] | 
            
  | 17:08 | 
  <brennen@deploy1001> | 
  rebuilt and synchronized wikiversions files: group1 wikis to 1.35.0-wmf.8 | 
  [production] | 
            
  | 17:03 | 
  <brennen> | 
  attempting to roll 1.35.0-wmf.8 forward to group1 | 
  [production] | 
            
  | 15:57 | 
  <moritzm> | 
  installing openslp security updates | 
  [production] | 
            
  | 15:40 | 
  <jiji@cumin1001> | 
  END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) | 
  [production] | 
            
  | 15:37 | 
  <jiji@cumin1001> | 
  START - Cookbook sre.hosts.downtime | 
  [production] | 
            
  | 15:09 | 
  <elukey> | 
  upload prometheus-memcached-exporter 0.6.0+git20191209.bac8a8c-1 to buster-wikimedia | 
  [production] | 
            
  | 15:01 | 
  <jiji@cumin1001> | 
  END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) | 
  [production] | 
            
  | 14:58 | 
  <jiji@cumin1001> | 
  START - Cookbook sre.hosts.downtime | 
  [production] | 
            
  | 14:34 | 
  <ladsgroup@deploy1001> | 
  Synchronized php-1.35.0-wmf.5/extensions/Wikibase/repo/includes/ParserOutput/FullEntityParserOutputGenerator.php: T229407, clean up debugging info (duration: 00m 59s) | 
  [production] | 
            
  | 14:26 | 
  <ladsgroup@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: [[gerrit:555941|Disable sanity check cirrus jobs for Wikidata (T239931 T229407)]] (duration: 00m 57s) | 
  [production] | 
            
  | 14:15 | 
  <jiji@cumin1001> | 
  END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) | 
  [production] | 
            
  | 14:13 | 
  <jiji@cumin1001> | 
  START - Cookbook sre.hosts.downtime | 
  [production] | 
            
  | 13:55 | 
  <effie> | 
  reimage mw2270.codfw.wmnet mw2269.codfw.wmnet mw2268.codfw.wmnet | 
  [production] | 
            
  | 11:36 | 
  <jdrewniak@deploy1001> | 
  Synchronized portals: Wikimedia Portals Update: [[gerrit:555907| Bumping portals to master (T128546)]] (duration: 01m 00s) | 
  [production] | 
            
  | 11:35 | 
  <jdrewniak@deploy1001> | 
  Synchronized portals/wikipedia.org/assets: Wikimedia Portals Update: [[gerrit:555907| Bumping portals to master (T128546)]] (duration: 01m 23s) | 
  [production] | 
            
  | 10:46 | 
  <addshore> | 
  T239470 addshore@mwmaint1002:~$ mwscript extensions/Wikibase/repo/maintenance/rebuildItemTerms.php --wiki wikidatawiki --from-id=10000007 --to-id=10000007 | 
  [production] | 
            
  | 10:06 | 
  <rlazarus> | 
  rolling restart php-fpm in mw-eqiad due to APCu fragmentation | 
  [production] | 
            
  | 08:58 | 
  <oblivian@cumin1001> | 
  conftool action : set/weight=10; selector: service=parsoid-php | 
  [production] | 
            
  | 08:49 | 
  <kart_> | 
  Updated cxserver to 2019-12-05-090549-production (T217585, T230195) | 
  [production] | 
            
  | 08:46 | 
  <kartik@deploy1001> | 
  helmfile [EQIAD] Ran 'apply' command on namespace 'cxserver' for release 'production' . | 
  [production] | 
            
  | 08:41 | 
  <kartik@deploy1001> | 
  helmfile [CODFW] Ran 'apply' command on namespace 'cxserver' for release 'production' . | 
  [production] | 
            
  | 08:39 | 
  <kartik@deploy1001> | 
  helmfile [STAGING] Ran 'apply' command on namespace 'cxserver' for release 'staging' . | 
  [production] | 
            
  | 08:33 | 
  <elukey> | 
  powercycle mw1280, mgmt console stuck, dimm errors in getsel | 
  [production] | 
            
  | 08:24 | 
  <ema> | 
  cp3064: ats-tls-restart to clear "tls process restarted" alert T240183 | 
  [production] | 
            
  | 07:44 | 
  <onimisionipe> | 
  resetting cron on wdqs1010 to fix cronspam | 
  [production] |