| 
      
        2020-10-07
      
      ยง
     | 
  
    
  | 18:12 | 
  <hashar@deploy1001> | 
  Synchronized php-1.36.0-wmf.10/includes/HeaderCallback.php: Preload class used in HeaderCallback - T261260 (duration: 01m 01s) | 
  [production] | 
            
  | 17:58 | 
  <hashar> | 
  Pulled https://gerrit.wikimedia.org/r/c/mediawiki/core/+/632680  on deployment staging area  and mw2001 | 
  [production] | 
            
  | 17:35 | 
  <elukey@cumin1001> | 
  END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) | 
  [production] | 
            
  | 17:33 | 
  <elukey@cumin1001> | 
  START - Cookbook sre.hosts.downtime | 
  [production] | 
            
  | 16:39 | 
  <jgleeson> | 
  updated civicrm from 39b4f954ed to b86a15a430 | 
  [production] | 
            
  | 16:35 | 
  <mutante> | 
  switching webproxy service names to the new local install servers in esams/eqsin/ulsfo T242602 | 
  [production] | 
            
  | 15:12 | 
  <godog> | 
  upgrade rsyslog to 8.2008.0-1~bpo10+1 on centrallog1001 - T259780 | 
  [production] | 
            
  | 14:45 | 
  <cmjohnson@cumin1001> | 
  END (PASS) - Cookbook sre.dns.netbox (exit_code=0) | 
  [production] | 
            
  | 14:41 | 
  <cmjohnson@cumin1001> | 
  START - Cookbook sre.dns.netbox | 
  [production] | 
            
  | 14:37 | 
  <cmjohnson@cumin1001> | 
  END (PASS) - Cookbook sre.dns.netbox (exit_code=0) | 
  [production] | 
            
  | 14:33 | 
  <cmjohnson@cumin1001> | 
  START - Cookbook sre.dns.netbox | 
  [production] | 
            
  | 14:22 | 
  <jayme@deploy1001> | 
  helmfile [codfw] Ran 'sync' command on namespace 'cxserver' for release 'production' . | 
  [production] | 
            
  | 14:04 | 
  <hoo> | 
  Ran "mwscript extensions/Wikibase/repo/maintenance/changePropertyDataType.php --wiki=wikidatawiki --property-id P1820 --new-data-type external-id" on mwmaint2001 (T263986) | 
  [production] | 
            
  | 14:04 | 
  <jayme@deploy1001> | 
  helmfile [eqiad] Ran 'sync' command on namespace 'cxserver' for release 'production' . | 
  [production] | 
            
  | 14:03 | 
  <akosiaris@deploy1001> | 
  helmfile [codfw] Ran 'sync' command on namespace 'termbox' for release 'production' . | 
  [production] | 
            
  | 14:00 | 
  <jayme@deploy1001> | 
  helmfile [staging] Ran 'sync' command on namespace 'cxserver' for release 'staging' . | 
  [production] | 
            
  | 13:42 | 
  <jayme> | 
  updated envoyproxy to 1.15.1-2 on all eqiad hosts | 
  [production] | 
            
  | 13:39 | 
  <akosiaris@deploy1001> | 
  helmfile [eqiad] Ran 'sync' command on namespace 'termbox' for release 'production' . | 
  [production] | 
            
  | 13:37 | 
  <akosiaris@deploy1001> | 
  helmfile [staging] Ran 'sync' command on namespace 'termbox' for release 'test' . | 
  [production] | 
            
  | 13:37 | 
  <akosiaris@deploy1001> | 
  helmfile [staging] Ran 'sync' command on namespace 'termbox' for release 'staging' . | 
  [production] | 
            
  | 13:18 | 
  <volker-e@deploy1001> | 
  Finished deploy [design/style-guide@e3fda83]: Deploy design/style-guide:  (duration: 00m 04s) | 
  [production] | 
            
  | 13:18 | 
  <volker-e@deploy1001> | 
  Started deploy [design/style-guide@e3fda83]: Deploy design/style-guide: | 
  [production] | 
            
  | 12:33 | 
  <jayme@deploy1001> | 
  helmfile [codfw] Ran 'sync' command on namespace 'blubberoid' for release 'production' . | 
  [production] | 
            
  | 12:24 | 
  <jayme@deploy1001> | 
  helmfile [eqiad] Ran 'sync' command on namespace 'blubberoid' for release 'production' . | 
  [production] | 
            
  | 12:22 | 
  <jayme@deploy1001> | 
  helmfile [staging] Ran 'sync' command on namespace 'blubberoid' for release 'staging' . | 
  [production] | 
            
  | 11:55 | 
  <_joe_> | 
  rolling restart of restbase due to running puppet with changed config-vars (a noop for the actual configuration) | 
  [production] | 
            
  | 11:22 | 
  <Urbanecm> | 
  EU B&C window done | 
  [production] | 
            
  | 11:22 | 
  <urbanecm@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: f85bc3056f809910c0487fb0b0559b3de92b1992: Enable bot passwords at all fishbowl and private wikis (T258356) (duration: 00m 58s) | 
  [production] | 
            
  | 11:15 | 
  <urbanecm@deploy1001> | 
  Synchronized wmf-config/CommonSettings.php: 57297362c0a22ecf16648b7be4a73c4cb80d53ef: Fix OAuthRateLimiter rate limit configuration (duration: 00m 59s) | 
  [production] | 
            
  | 11:14 | 
  <urbanecm@deploy1001> | 
  sync-file aborted: 57297362c0a22ecf16648b7be4a73c4cb80d53ef: Fix OAuthRateLimiter rate limit configuration (duration: 00m 02s) | 
  [production] | 
            
  | 11:07 | 
  <urbanecm@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: 6cdeea2c4c15780a641722157584f12febedab2a: Set CXMTThresholdForPublish to 95% for Vietnamese Wikipedia (T264161) (duration: 00m 59s) | 
  [production] | 
            
  | 10:58 | 
  <marostegui> | 
  Set innodb_change_buffering = inserts on pc2009 T263443 | 
  [production] | 
            
  | 09:53 | 
  <kormat@cumin1001> | 
  dbctl commit (dc=all): 'Remove db2119 from mw load groups T259831', diff saved to https://phabricator.wikimedia.org/P12945 and previous config saved to /var/cache/conftool/dbconfig/20201007-095355-kormat.json | 
  [production] | 
            
  | 09:44 | 
  <kormat@cumin1001> | 
  dbctl commit (dc=all): 'db2138:3314 (re)pooling @ 100%: 75', diff saved to https://phabricator.wikimedia.org/P12944 and previous config saved to /var/cache/conftool/dbconfig/20201007-094412-kormat.json | 
  [production] | 
            
  | 09:21 | 
  <moritzm> | 
  imported icu63 63.1-6+deb10u1~wmf1 to component/icu63 for stretch-wikimedia | 
  [production] | 
            
  | 09:09 | 
  <marostegui@cumin1001> | 
  dbctl commit (dc=all): 'Repool db1076 T264755 ', diff saved to https://phabricator.wikimedia.org/P12943 and previous config saved to /var/cache/conftool/dbconfig/20201007-090943-marostegui.json | 
  [production] | 
            
  | 08:39 | 
  <kormat@cumin1001> | 
  dbctl commit (dc=all): 'db2138:3314 depooling: schema change T259831', diff saved to https://phabricator.wikimedia.org/P12942 and previous config saved to /var/cache/conftool/dbconfig/20201007-083903-kormat.json | 
  [production] | 
            
  | 08:38 | 
  <kormat@cumin1001> | 
  END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) | 
  [production] | 
            
  | 08:38 | 
  <kormat@cumin1001> | 
  START - Cookbook sre.hosts.downtime | 
  [production] | 
            
  | 08:32 | 
  <godog> | 
  roll-restart statsd-exporter across ms-be* after puppet run - T264588 | 
  [production] | 
            
  | 08:09 | 
  <jayme> | 
  updated envoyproxy to 1.15.1-2 on all non mw and restbase hosts | 
  [production] | 
            
  | 08:05 | 
  <volans@cumin1001> | 
  END (PASS) - Cookbook sre.dns.netbox (exit_code=0) | 
  [production] | 
            
  | 07:58 | 
  <volans@cumin1001> | 
  START - Cookbook sre.dns.netbox | 
  [production] | 
            
  | 07:49 | 
  <marostegui@cumin1001> | 
  dbctl commit (dc=all): 'Remove es2015 from dbctl T264700', diff saved to https://phabricator.wikimedia.org/P12941 and previous config saved to /var/cache/conftool/dbconfig/20201007-074951-marostegui.json | 
  [production] | 
            
  | 07:14 | 
  <marostegui> | 
  Stop MySQL es2015 for decommissioning T264700 | 
  [production] | 
            
  | 05:52 | 
  <ayounsi@cumin1001> | 
  END (PASS) - Cookbook sre.dns.netbox (exit_code=0) | 
  [production] | 
            
  | 05:46 | 
  <ayounsi@cumin1001> | 
  START - Cookbook sre.dns.netbox | 
  [production] | 
            
  | 02:37 | 
  <eileen> | 
  civicrm revision changed from a30da7f92a to 39b4f954ed, config revision is 0ca9a3a055 | 
  [production] | 
            
  | 01:00 | 
  <cdanis> | 
  repool esams; cr2-esams router upgrade complete | 
  [production] | 
            
  | 00:43 | 
  <cdanis> | 
  T259621 cdanis@re1.cr2-esams> request chassis routing-engine master switch | 
  [production] |