|
2019-12-06
ยง
|
| 23:10 |
<andrew@deploy1001> |
Started deploy [horizon/deploy@1911591]: (no justification provided) |
[production] |
| 23:00 |
<ppchelko@deploy1001> |
Finished deploy [restbase/deploy@c2bab5d]: Parsoid: Disable mirroring all traffic in split mode (duration: 13m 43s) |
[production] |
| 22:46 |
<ppchelko@deploy1001> |
Started deploy [restbase/deploy@c2bab5d]: Parsoid: Disable mirroring all traffic in split mode |
[production] |
| 22:08 |
<bblack> |
mc1033: ethernet tweaks as well (expect a short link blip) |
[production] |
| 21:54 |
<bblack> |
mc1026: add tc-fq qdisc to eth0 for tx |
[production] |
| 21:41 |
<bblack> |
mc1026: adjusting rx ring to 2047 and disabling ethernet pause (will be a minor blip of eth link state!) |
[production] |
| 21:25 |
<jeh@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 21:23 |
<jeh@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 21:16 |
<cdanis@cumin2001> |
conftool action : set/weight=15; selector: service=nginx,cluster=api_appserver,dc=eqiad,name=mw1231.eqiad.wmnet |
[production] |
| 21:15 |
<cdanis@cumin2001> |
conftool action : set/weight=15; selector: service=nginx,cluster=api_appserver,dc=eqiad,name=mw1227.eqiad.wmnet |
[production] |
| 21:15 |
<cdanis@cumin2001> |
conftool action : set/weight=15; selector: service=nginx,cluster=api_appserver,dc=eqiad,name=mw1222.eqiad.wmnet |
[production] |
| 21:15 |
<cdanis@cumin2001> |
conftool action : set/weight=15; selector: service=nginx,cluster=api_appserver,dc=eqiad,name=mw1233.eqiad.wmnet |
[production] |
| 21:14 |
<cdanis@cumin2001> |
conftool action : set/weight=25; selector: service=nginx,cluster=api_appserver,dc=eqiad,name=mw12[789].* |
[production] |
| 21:12 |
<cdanis@cumin2001> |
conftool action : set/weight=15; selector: service=nginx,cluster=api_appserver,dc=eqiad,name=mw1233 |
[production] |
| 21:12 |
<cdanis@cumin2001> |
conftool action : set/weight=15; selector: service=nginx,cluster=api_appserver,dc=eqiad,name=mw1222 |
[production] |
| 21:12 |
<cdanis@cumin2001> |
conftool action : set/weight=15; selector: service=nginx,cluster=api_appserver,dc=eqiad,name=mw1227 |
[production] |
| 21:04 |
<jeh@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 21:01 |
<jeh@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 20:37 |
<jeh@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 20:34 |
<jeh@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 18:57 |
<jeh@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 18:56 |
<cdanis@cumin2001> |
conftool action : set/weight=20; selector: service=nginx,cluster=api_appserver,dc=eqiad,name=mw12.* |
[production] |
| 18:55 |
<jeh@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 18:36 |
<jeh@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 18:34 |
<jeh@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 18:15 |
<jeh@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 18:13 |
<jeh@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 18:12 |
<cdanis@cumin2001> |
conftool action : set/weight=15; selector: service=nginx,cluster=api_appserver,dc=eqiad,name=mw12.* |
[production] |
| 17:54 |
<bblack> |
install2002 - restart squid3 service |
[production] |
| 17:43 |
<jforrester@deploy1001> |
Synchronized php-1.35.0-wmf.8/includes/libs/rdbms/database/Database.php: T239877 Have Database::makeWhereFrom2d assume is string-based (duration: 01m 11s) |
[production] |
| 17:28 |
<jeh@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 17:26 |
<jeh@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 17:19 |
<bblack> |
editing /e/n/i carefully with sed across the fleet via cumin, to correct legacy "dns-nameservers" line in older installs |
[production] |
| 17:08 |
<jeh@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 17:06 |
<jeh@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 16:50 |
<jeh@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 16:48 |
<jeh@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 16:47 |
<_joe_> |
acpu flush finished |
[production] |
| 16:41 |
<_joe_> |
flush acpu across the api cluster in eqiad |
[production] |
| 16:32 |
<_joe_> |
flushing apcu on mw1339 |
[production] |
| 16:21 |
<ejegg> |
updated fundraising CiviCRM from 30cdc5fa59 to 7eab025ec0 |
[production] |
| 14:40 |
<ema> |
text@esams: rolling ats-backend-restart to apply https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/553132/ T238494 |
[production] |
| 14:12 |
<ema> |
cp3050: ats-backend-restart to apply https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/553132/ T238494 |
[production] |
| 13:41 |
<ema> |
cp2004: adding do_global_ doesn't seem to work with reload, restart ats-be T238494 |
[production] |
| 13:31 |
<gehel> |
starting transfer of blazegraph journal from wdqs1007 to stat1004 - T239898 |
[production] |
| 08:46 |
<andrew@deploy1001> |
Finished deploy [horizon/deploy@1911591]: (no justification provided) (duration: 00m 08s) |
[production] |
| 08:46 |
<andrew@deploy1001> |
Started deploy [horizon/deploy@1911591]: (no justification provided) |
[production] |
| 08:43 |
<andrew@deploy1001> |
Finished deploy [horizon/deploy@1911591]: (no justification provided) (duration: 01m 59s) |
[production] |
| 08:41 |
<andrew@deploy1001> |
Started deploy [horizon/deploy@1911591]: (no justification provided) |
[production] |
| 08:38 |
<andrew@deploy1001> |
Finished deploy [horizon/deploy@1911591]: (no justification provided) (duration: 01m 55s) |
[production] |