| 2020-08-11
      
      ยง | 
    
  | 14:55 | <jayme> | updated helmfile to 0.125.2-1 on contint* and deploy* | [production] | 
            
  | 14:52 | <otto@deploy1001> | Finished deploy [analytics/refinery@35c4430]: Deploying to an-launcher1002 to get camus wrapper script changes - T251935 (duration: 01m 14s) | [production] | 
            
  | 14:51 | <otto@deploy1001> | Started deploy [analytics/refinery@35c4430]: Deploying to an-launcher1002 to get camus wrapper script changes - T251935 | [production] | 
            
  | 14:50 | <hashar@deploy1001> | Pruned MediaWiki: 1.36.0-wmf.1 (duration: 02m 07s) | [production] | 
            
  | 14:48 | <jayme> | imported helmfile_0.125.2-1 to buster-wikimedia, jessie-wikimedia, stretch-wikimedia | [production] | 
            
  | 14:47 | <hashar@deploy1001> | Pruned MediaWiki: 1.35.0-wmf.41 (duration: 04m 20s) | [production] | 
            
  | 14:40 | <hashar@deploy1001> | Pruned MediaWiki: 1.35.0-wmf.40 (duration: 10m 24s) | [production] | 
            
  | 14:37 | <papaul> | replacing msw-b5,b6,b7 and b8 | [production] | 
            
  | 14:30 | <hashar> | Cleaning old MediaWiki versions that were never removed | [production] | 
            
  | 14:27 | <hashar@deploy1001> | sync aborted: testwikis wikis to 1.36.0-wmf.4 (duration: 72m 36s) | [production] | 
            
  | 14:10 | <hashar> | mw1319: scap pull | [production] | 
            
  | 13:27 | <jayme@deploy1001> | helmfile [staging] Ran 'sync' command on namespace 'blubberoid' for release 'staging' . | [production] | 
            
  | 13:27 | <jayme@deploy1001> | helmfile [staging] Ran 'sync' command on namespace 'blubberoid' for release 'production' . | [production] | 
            
  | 13:23 | <oblivian@deploy1001> | helmfile [codfw] Ran 'sync' command on namespace 'blubberoid' for release 'production' . | [production] | 
            
  | 13:16 | <oblivian@deploy1001> | helmfile [eqiad] Ran 'sync' command on namespace 'blubberoid' for release 'production' . | [production] | 
            
  | 13:14 | <hashar@deploy1001> | Started scap: testwikis wikis to 1.36.0-wmf.4 | [production] | 
            
  | 13:12 | <hashar> | Applied 1.36.0-wmf.4 security patches # T257972 | [production] | 
            
  | 13:03 | <jayme@deploy1001> | helmfile [staging] Ran 'sync' command on namespace 'blubberoid' for release 'staging' . | [production] | 
            
  | 13:03 | <jayme@deploy1001> | helmfile [staging] Ran 'sync' command on namespace 'blubberoid' for release 'production' . | [production] | 
            
  | 12:52 | <kormat> | uploaded wmfmariadbpy 0.2 packages to apt1001 | [production] | 
            
  | 12:36 | <oblivian@deploy1001> | helmfile [staging] Ran 'sync' command on namespace 'blubberoid' for release 'production' . | [production] | 
            
  | 12:36 | <oblivian@deploy1001> | helmfile [staging] Ran 'sync' command on namespace 'blubberoid' for release 'staging' . | [production] | 
            
  | 11:54 | <marostegui> | Install new MariaDB 10.4.14 on db2102 | [production] | 
            
  | 11:42 | <hnowlan@deploy1001> | helmfile [eqiad] Ran 'sync' command on namespace 'api-gateway' for release 'production' . | [production] | 
            
  | 11:30 | <hnowlan@deploy1001> | helmfile [staging] Ran 'sync' command on namespace 'api-gateway' for release 'staging' . | [production] | 
            
  | 11:18 | <Urbanecm> | EU B&C window done | [production] | 
            
  | 11:08 | <kartik@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit|619255|Enable ContentTranslation in Sundanese WP as a default tool (T258502)]] (duration: 00m 59s) | [production] | 
            
  | 10:39 | <volans> | migrating *all* eqiad mgmt DNS records to the autogenerated ones via Netbox - T233183 | [production] | 
            
  | 10:38 | <volans@cumin1001> | END (PASS) - Cookbook sre.dns.netbox (exit_code=0) | [production] | 
            
  | 10:34 | <volans@cumin1001> | START - Cookbook sre.dns.netbox | [production] | 
            
  | 10:20 | <elukey@cumin1001> | END (PASS) - Cookbook sre.hadoop.change-distro-from-cdh (exit_code=0) | [production] | 
            
  | 10:01 | <elukey@cumin1001> | START - Cookbook sre.hadoop.change-distro-from-cdh | [production] | 
            
  | 10:00 | <elukey@cumin1001> | END (PASS) - Cookbook sre.hadoop.stop-cluster (exit_code=0) | [production] | 
            
  | 09:51 | <elukey@cumin1001> | START - Cookbook sre.hadoop.stop-cluster | [production] | 
            
  | 09:29 | <volans@cumin1001> | END (PASS) - Cookbook sre.dns.netbox (exit_code=0) | [production] | 
            
  | 09:25 | <volans@cumin1001> | START - Cookbook sre.dns.netbox | [production] | 
            
  | 09:11 | <marostegui> | Rename tables on muswiki and mhwiktionary on s3 master (db1123) without replication T260112 | [production] | 
            
  | 09:01 | <volans> | renewed puppet certificate on scb1001.eqiad.wmnet | [production] | 
            
  | 08:52 | <urbanecm@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: e6ec237b6b6fb67a0a80613909589bc724f5eecf: Revert "Turn muswiki and mhwiktionary to read-only" (T259004) (duration: 00m 58s) | [production] | 
            
  | 08:45 | <urbanecm@deploy1001> | Synchronized dblists/: 81f4594b6c583f938821549b3a1800fec5b120bb: Point muswiki and mhwiktionary to s5 (T259004; 3/3) (duration: 00m 58s) | [production] | 
            
  | 08:44 | <urbanecm@deploy1001> | Synchronized wmf-config/db-eqiad.php: 81f4594b6c583f938821549b3a1800fec5b120bb: Point muswiki and mhwiktionary to s5 (T259004; 2/3) (duration: 00m 58s) | [production] | 
            
  | 08:43 | <urbanecm@deploy1001> | Synchronized wmf-config/db-codfw.php: 81f4594b6c583f938821549b3a1800fec5b120bb: Point muswiki and mhwiktionary to s5 (T259004; 1/3) (duration: 01m 02s) | [production] | 
            
  | 08:06 | <urbanecm@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: a04bc1f27e4ef4e38002d546d30bfd2d1dc60d0e: Turn muswiki and mhwiktionary to read-only (T259004) (duration: 01m 01s) | [production] | 
            
  | 08:04 | <ayounsi@cumin1001> | END (PASS) - Cookbook sre.network.prepare-upgrade (exit_code=0) | [production] | 
            
  | 06:54 | <ayounsi@cumin1001> | START - Cookbook sre.network.prepare-upgrade | [production] | 
            
  | 06:45 | <XioNoX> | Re-prioritize peering over transit eqiad/esams - T259614 | [production] | 
            
  | 01:59 | <tstarling@deploy1001> | Synchronized wmf-config/PoolCounterSettings.php: enabling fast stale mode T250248 (duration: 00m 58s) | [production] | 
            
  | 00:33 | <dpifke@deploy1001> | Finished deploy [performance/arc-lamp@fc5f1c6]: Deploying latest attempt to fix T259167 (duration: 01m 03s) | [production] | 
            
  | 00:31 | <dpifke@deploy1001> | Started deploy [performance/arc-lamp@fc5f1c6]: Deploying latest attempt to fix T259167 | [production] | 
            
  | 00:24 | <mutante> | reverting switch of releases.wikimedia.org for today since releases-jenkins.wikimedia.org is tied to it and new jenkins still needs some config and plugins (T247652) | [production] |