|
2020-11-30
ยง
|
| 14:02 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1087 (re)pooling @ 50%: After cloning clouddb1016:3318', diff saved to https://phabricator.wikimedia.org/P13477 and previous config saved to /var/cache/conftool/dbconfig/20201130-140226-root.json |
[production] |
| 13:58 |
<ema> |
varnish 6.0.7-1wm1 uploaded to apt.wikimedia.org component/varnish6 T268736 |
[production] |
| 13:48 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Slowly repool db1092', diff saved to https://phabricator.wikimedia.org/P13475 and previous config saved to /var/cache/conftool/dbconfig/20201130-134841-marostegui.json |
[production] |
| 13:47 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1087 (re)pooling @ 25%: After cloning clouddb1016:3318', diff saved to https://phabricator.wikimedia.org/P13474 and previous config saved to /var/cache/conftool/dbconfig/20201130-134722-root.json |
[production] |
| 13:23 |
<jbond42> |
update zeromq on jessie hosts |
[production] |
| 13:21 |
<dcausse> |
depooling wdqs1004 (lag) |
[production] |
| 13:18 |
<moritzm> |
CAS enabled for racktables |
[production] |
| 13:16 |
<gilles@deploy1001> |
Synchronized debug.json: T268167 Add mwdebug1003 to list of debug servers (duration: 00m 56s) |
[production] |
| 12:50 |
<Urbanecm> |
EU B&C window done |
[production] |
| 12:46 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: 3476644e4c27dd28339f7b10c8871be2e9455394: Grant enwikibooks reviewers suppressredirect and raise move rate limit to 100/60 (T268849; 2nd attempt) (duration: 00m 56s) |
[production] |
| 12:43 |
<hnowlan@deploy1001> |
Finished deploy [kartotherian/deploy@0a38bc5]: Redeploy to fix gelf traffic (duration: 00m 24s) |
[production] |
| 12:43 |
<hnowlan@deploy1001> |
Started deploy [kartotherian/deploy@0a38bc5]: Redeploy to fix gelf traffic |
[production] |
| 12:41 |
<hnowlan@puppetmaster1001> |
conftool action : set/pooled=no; selector: dc=eqiad,cluster=maps,service=kartotherian,name=maps1004.eqiad.wmnet |
[production] |
| 12:40 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: 5585fd79119a4e077705789d1c1928c9e9efa956: Enable RelatedArticles on ptwikinews (T268945) (duration: 00m 57s) |
[production] |
| 12:38 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: ba6d0f8fd2a443e5c913a292365063f01f2d076b: Grant enwikibooks reviewers suppressredirect and raise move rate limit to 100/60 (T268849) (duration: 00m 57s) |
[production] |
| 12:37 |
<hnowlan@deploy1001> |
Finished deploy [kartotherian/deploy@0a38bc5]: Newer codfw maps hosts (duration: 02m 05s) |
[production] |
| 12:37 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: 9942d68c914a56073d2d192434ba24ff8cb921ba: Assign patrolmarks right to autoconfirmed users on itwiki (T268734) (duration: 00m 57s) |
[production] |
| 12:35 |
<hnowlan@puppetmaster1001> |
conftool action : set/pooled=yes:weight=10; selector: dc=eqiad,cluster=maps,service=kartotherian,name=maps1005.eqiad.wmnet |
[production] |
| 12:35 |
<hnowlan@deploy1001> |
Started deploy [kartotherian/deploy@0a38bc5]: Newer codfw maps hosts |
[production] |
| 12:34 |
<hnowlan@deploy1001> |
Finished deploy [tilerator/deploy@97575e4]: Newer codfw maps hosts (duration: 00m 24s) |
[production] |
| 12:34 |
<hnowlan@deploy1001> |
Started deploy [tilerator/deploy@97575e4]: Newer codfw maps hosts |
[production] |
| 12:34 |
<hnowlan@deploy1001> |
Finished deploy [tilerator/deploy@97575e4]: Newer codfw maps hosts (duration: 00m 51s) |
[production] |
| 12:33 |
<hnowlan@deploy1001> |
Started deploy [tilerator/deploy@97575e4]: Newer codfw maps hosts |
[production] |
| 12:32 |
<Lucas_WMDE> |
Deployed patch for T260349 |
[production] |
| 12:27 |
<hnowlan@deploy1001> |
Finished deploy [tilerator/deploy@97575e4]: New eqiad maps hosts (duration: 00m 03s) |
[production] |
| 12:27 |
<hnowlan@deploy1001> |
Started deploy [tilerator/deploy@97575e4]: New eqiad maps hosts |
[production] |
| 12:24 |
<hnowlan@deploy1001> |
Finished deploy [kartotherian/deploy@0a38bc5]: New eqiad maps hosts (duration: 00m 03s) |
[production] |
| 12:24 |
<hnowlan@deploy1001> |
Started deploy [kartotherian/deploy@0a38bc5]: New eqiad maps hosts |
[production] |
| 12:21 |
<hnowlan@puppetmaster1001> |
conftool action : set/pooled=no:weight=0; selector: dc=eqiad,cluster=maps,service=kartotherian,name=maps1005.eqiad.wmnet |
[production] |
| 12:12 |
<urbanecm@deploy1001> |
Synchronized wmf-config/CommonSettings.php: 2922abe7b810f1b53b446af783dc9d51e6585225: Remove wgContentTranslationRESTBase config (T266213) (duration: 00m 57s) |
[production] |
| 11:43 |
<marostegui> |
Sanitize clouddb1016:3318 - T267090 |
[production] |
| 11:38 |
<ema> |
A:cp upgrade fifo-log-demux to 0.6.2 T268883 |
[production] |
| 11:36 |
<jdrewniak@deploy1001> |
Synchronized portals: Wikimedia Portals Update: [[gerrit:644196| Bumping portals to master (T128546)]] (duration: 00m 57s) |
[production] |
| 11:35 |
<jdrewniak@deploy1001> |
Synchronized portals/wikipedia.org/assets: Wikimedia Portals Update: [[gerrit:644196| Bumping portals to master (T128546)]] (duration: 01m 01s) |
[production] |
| 11:32 |
<ariel@deploy1001> |
Finished deploy [dumps/dumps@e8c6267]: allow page content fixup script to write output files to arbitrary dir (duration: 00m 04s) |
[production] |
| 11:32 |
<ariel@deploy1001> |
Started deploy [dumps/dumps@e8c6267]: allow page content fixup script to write output files to arbitrary dir |
[production] |
| 11:28 |
<ema> |
upload fifo-log-demux 0.6.2 to buster-wikimedia T268883 |
[production] |
| 11:13 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1134 (re)pooling @ 100%: After schema change', diff saved to https://phabricator.wikimedia.org/P13473 and previous config saved to /var/cache/conftool/dbconfig/20201130-111321-root.json |
[production] |
| 11:00 |
<hnowlan> |
bootstrapping maps1005 cassandra |
[production] |
| 10:58 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1134 (re)pooling @ 75%: After schema change', diff saved to https://phabricator.wikimedia.org/P13472 and previous config saved to /var/cache/conftool/dbconfig/20201130-105818-root.json |
[production] |
| 10:43 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1134 (re)pooling @ 50%: After schema change', diff saved to https://phabricator.wikimedia.org/P13471 and previous config saved to /var/cache/conftool/dbconfig/20201130-104314-root.json |
[production] |
| 10:29 |
<ema@cumin1001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) |
[production] |
| 10:29 |
<marostegui> |
Compare data between clouddb1014:3312 clouddb1018:3312 labsdb1012 T267090 |
[production] |
| 10:29 |
<marostegui> |
Compare data between clouddb1012:3312 clouddb1018:3312 labsdb1012 T267090 |
[production] |
| 10:28 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1134 (re)pooling @ 25%: After schema change', diff saved to https://phabricator.wikimedia.org/P13470 and previous config saved to /var/cache/conftool/dbconfig/20201130-102811-root.json |
[production] |
| 10:24 |
<akosiaris> |
applying https://gerrit.wikimedia.org/r/q/topic:%22k8s_config%22 series of patches |
[production] |
| 10:18 |
<ema@cumin1001> |
START - Cookbook sre.hosts.reboot-single |
[production] |
| 10:18 |
<ema> |
cp4031: reboot to test atsmtail/fifo-log-demux service dependencies -- https://gerrit.wikimedia.org/r/c/operations/puppet/+/643922 T256467 |
[production] |
| 10:11 |
<ema> |
cp4032: upgrade varnish to 6.0.7-1wm1 T268736 |
[production] |
| 10:06 |
<moritzm> |
installing NSS security updates |
[production] |