| 
      
        2019-08-21
      
      ยง
     | 
  
    
  | 05:26 | 
  <marostegui@cumin1001> | 
  dbctl commit (dc=all): 'More weight to db1122', diff saved to https://phabricator.wikimedia.org/P8951 and previous config saved to /var/cache/conftool/dbconfig/20190821-052613-marostegui.json | 
  [production] | 
            
  | 05:14 | 
  <marostegui@cumin1001> | 
  dbctl commit (dc=all): 'Slowly repool db1122 after restart', diff saved to https://phabricator.wikimedia.org/P8950 and previous config saved to /var/cache/conftool/dbconfig/20190821-051441-marostegui.json | 
  [production] | 
            
  | 05:05 | 
  <marostegui> | 
  Restart MySQL on db1122 for binlog format change - T230785 | 
  [production] | 
            
  | 05:05 | 
  <marostegui@cumin1001> | 
  dbctl commit (dc=all): 'Depool db1122 for binlog format change', diff saved to https://phabricator.wikimedia.org/P8949 and previous config saved to /var/cache/conftool/dbconfig/20190821-050501-marostegui.json | 
  [production] | 
            
  | 05:04 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Clarify db1122 status: candidate master for s2 - T230785 (duration: 00m 55s) | 
  [production] | 
            
  | 02:28 | 
  <ebernhardson@deploy1001> | 
  Finished deploy [search/mjolnir/deploy@adff5ad]: bulk_daemon: Handle non-integer status_code in json response (duration: 04m 09s) | 
  [production] | 
            
  | 02:24 | 
  <ebernhardson@deploy1001> | 
  Started deploy [search/mjolnir/deploy@adff5ad]: bulk_daemon: Handle non-integer status_code in json response | 
  [production] | 
            
  
    | 
      
        2019-08-20
      
      ยง
     | 
  
    
  | 23:53 | 
  <ebernhardson@deploy1001> | 
  Finished deploy [search/mjolnir/deploy@a7bf6cf]: bulk_daemon: Increase bulk request_timeout (duration: 03m 40s) | 
  [production] | 
            
  | 23:50 | 
  <eileen> | 
  that just changes us to php7 csv so watch for any fail mail | 
  [production] | 
            
  | 23:49 | 
  <eileen> | 
  civicrm revision changed from 9c7b2ffbc9 to 0d1b7f107a, config revision is 58cd6b7ae6 | 
  [production] | 
            
  | 23:49 | 
  <ebernhardson@deploy1001> | 
  Started deploy [search/mjolnir/deploy@a7bf6cf]: bulk_daemon: Increase bulk request_timeout | 
  [production] | 
            
  | 23:42 | 
  <Urbanecm> | 
  Evening SWAT aborted due to no logs logged for some period of time (T230847), no patches were reverted | 
  [production] | 
            
  | 23:27 | 
  <urbanecm@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: SWAT: fd2cece: Enable RelatedArticles on all skins on eswikinews (T230660) (duration: 00m 52s) | 
  [production] | 
            
  | 23:22 | 
  <urbanecm@deploy1001> | 
  Synchronized wmf-config/throttle-analyze.php: SWAT: a3927a7: Grant skipcaptcha to everyone coming from whitelisted IP (T227487) (duration: 00m 54s) | 
  [production] | 
            
  | 23:19 | 
  <urbanecm@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: SWAT: 13be059: Disable Wikimedia ReadingDepth (T229042) (duration: 00m 56s) | 
  [production] | 
            
  | 23:16 | 
  <urbanecm@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: SWAT: 0c08257: Remove unused remnant from old menu click tracking (T228681) (duration: 00m 55s) | 
  [production] | 
            
  | 23:10 | 
  <urbanecm@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: SWAT: b94b647: Update wgSkipSkins to experiment with not showing skins to users (T223824) (duration: 00m 58s) | 
  [production] | 
            
  | 21:20 | 
  <ebernhardson@deploy1001> | 
  Finished deploy [search/mjolnir/deploy@9b40607]: bulk_daemon: Increase max_poll_interval_ms to 15 minutes (duration: 06m 22s) | 
  [production] | 
            
  | 21:14 | 
  <ebernhardson@deploy1001> | 
  Started deploy [search/mjolnir/deploy@9b40607]: bulk_daemon: Increase max_poll_interval_ms to 15 minutes | 
  [production] | 
            
  | 20:49 | 
  <XioNoX> | 
  push BGP_Wikimedia_pops to ams - T227808 | 
  [production] | 
            
  | 19:28 | 
  <XioNoX> | 
  push BGP_Wikimedia_pops to eqsin - T227808 | 
  [production] | 
            
  | 19:25 | 
  <thcipriani> | 
  cleanup old (pre 1.34.0-wmf.14) wmf/* branches for core and extensions on gerrit | 
  [production] | 
            
  | 19:25 | 
  <XioNoX> | 
  push BGP_Wikimedia_pops to cr4-ulsfo - T227808 | 
  [production] | 
            
  | 19:04 | 
  <XioNoX> | 
  push BGP_Wikimedia_pops to cr3-ulsfo - T227808 | 
  [production] | 
            
  | 19:00 | 
  <cdanis@deploy1001> | 
  Synchronized docroot/noc/db.php: 80a6743dd noc: read dbctl JSON T229631 (duration: 00m 58s) | 
  [production] | 
            
  | 17:57 | 
  <bblack> | 
  deploying anycast recdns settings to resolv.conf on 41 live hosts in eqiad - https://gerrit.wikimedia.org/r/528524 - T228190 | 
  [production] | 
            
  | 16:54 | 
  <cdanis> | 
  โ๏ธ cdanis@deploy1001.eqiad.wmnet /srv/mediawiki-staging ๐โ sudo chmod g+w -R /srv/mediawiki-staging/ | 
  [production] | 
            
  | 16:49 | 
  <krinkle@deploy1001> | 
  Synchronized php-1.34.0-wmf.17/includes/resourceloader/ResourceLoaderWikiModule.php: T229433 - f84a4abb418de8 (debugging) (duration: 00m 56s) | 
  [production] | 
            
  | 16:42 | 
  <Krinkle> | 
  php-1.34.0-wmf.17/extensions/TimedMediaHandler is dirty. A merged patch was not deployed - https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/TimedMediaHandler/+/530558/ | 
  [production] | 
            
  | 16:24 | 
  <hoo> | 
  Updated the Wikidata property suggester with data from the 2019-08-12 JSON dump and applied the T132839 workarounds | 
  [production] | 
            
  | 16:15 | 
  <krinkle@deploy1001> | 
  Synchronized php-1.34.0-wmf.19/includes/resourceloader/ResourceLoaderWikiModule.php: T229433 - 44607c984016b (debugging) (duration: 00m 55s) | 
  [production] | 
            
  | 16:08 | 
  <urbanecm@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: fa903b7: Enable DNS blacklist for es.wikiquote (T230796) (duration: 00m 55s) | 
  [production] | 
            
  | 16:04 | 
  <oblivian@deploy1001> | 
  Pruned MediaWiki: 1.34.0-wmf.13 (duration: 04m 09s) | 
  [production] | 
            
  | 15:59 | 
  <urbanecm@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: 5ab38dc: Restrict account creation on es.wikiquote to 1 day/IP (T230796) (duration: 01m 00s) | 
  [production] | 
            
  | 15:49 | 
  <urandom> | 
  creating Parsoid/PHP storage schema in restbase-dev -- T230792 | 
  [production] | 
            
  | 15:48 | 
  <Urbanecm> | 
  Run sudo -u mwdeploy chmod g+w /srv/mediawiki-stagging/wmf-config on deploy1001 | 
  [production] | 
            
  | 15:17 | 
  <zfilipin@deploy1001> | 
  rebuilt and synchronized wikiversions files: group0 to 1.34.0-wmf.19 | 
  [production] | 
            
  | 14:54 | 
  <zfilipin@deploy1001> | 
  Finished scap: testwiki to php-1.34.0-wmf.19 and rebuild l10n cache (duration: 30m 31s) | 
  [production] | 
            
  | 14:29 | 
  <jmm@cumin1001> | 
  END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) | 
  [production] | 
            
  | 14:24 | 
  <zfilipin@deploy1001> | 
  Started scap: testwiki to php-1.34.0-wmf.19 and rebuild l10n cache | 
  [production] | 
            
  | 14:21 | 
  <zfilipin@deploy1001> | 
  Pruned MediaWiki: 1.34.0-wmf.15 [keeping static files] (duration: 01m 43s) | 
  [production] | 
            
  | 14:13 | 
  <zfilipin@deploy1001> | 
  Pruned MediaWiki: 1.34.0-wmf.14 (duration: 06m 44s) | 
  [production] | 
            
  | 13:58 | 
  <jmm@cumin1001> | 
  START - Cookbook sre.ganeti.makevm | 
  [production] | 
            
  | 13:32 | 
  <jmm@cumin2001> | 
  END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) | 
  [production] | 
            
  | 13:07 | 
  <cdanis> | 
  โ๏ธ cdanis@cobalt.wikimedia.org ~ ๐ sudo systemctl restart gerrit.service | 
  [production] | 
            
  | 13:03 | 
  <jmm@cumin2001> | 
  START - Cookbook sre.ganeti.makevm | 
  [production] | 
            
  | 12:05 | 
  <awight@deploy1001> | 
  Synchronized php-1.34.0-wmf.17/extensions/Wikibase: SWAT: [[gerrit:530845|Initialize  DatabaseTermIdsResolver and DatabaseTypeIdsStore with repo database name in client. (T230119, T225053)]] (duration: 00m 52s) | 
  [production] | 
            
  | 10:51 | 
  <marostegui> | 
  Stop MySQL on db2051 and db2056 for decommission T230777 T230778 | 
  [production] | 
            
  | 10:30 | 
  <ema> | 
  cp5002: restart trafficserver for compress.so config change | 
  [production] | 
            
  | 10:11 | 
  <tarrow> | 
  termbox 2nd smoketests finished | 
  [production] |