|
2020-02-10
ยง
|
| 21:58 |
<robh> |
cp107[56] returned to service, cp107[78] offline for bios update via T243167 |
[production] |
| 21:43 |
<arlolra> |
Updated Parsoid to 612106d2 (T244412, T244413, T242746, T235273, T235307, T238845, T204618, T240054) |
[production] |
| 21:38 |
<robh> |
cp1075 & cp1076 offline for bios updates per T243167 |
[production] |
| 21:36 |
<robh> |
cp1075 and cp1076 going offline for bios updates. This will cause a bit of cp irc icinga noise, but no paging. Not putting into maint mode, as there is no way to maint mode the noisest check (which checks all backends and thus shouldnt be disabled) |
[production] |
| 21:33 |
<arlolra@deploy1001> |
Finished deploy [parsoid/deploy@d2d4870]: Updating Parsoid to 612106d2 (duration: 10m 26s) |
[production] |
| 21:32 |
<XioNoX> |
clamp tcp-mss on cr2-eqiad:xe-3/3/3 |
[production] |
| 21:23 |
<arlolra@deploy1001> |
Started deploy [parsoid/deploy@d2d4870]: Updating Parsoid to 612106d2 |
[production] |
| 21:12 |
<halfak@deploy1001> |
Finished deploy [ores/deploy@a6f4f14]: T242705 (duration: 12m 18s) |
[production] |
| 21:00 |
<halfak@deploy1001> |
Started deploy [ores/deploy@a6f4f14]: T242705 |
[production] |
| 20:55 |
<mholloway-shell@deploy1001> |
Synchronized php-1.35.0-wmf.16/extensions/MachineVision: MachineVision: Fix page id parsing from imageinfo results (T244752) (duration: 01m 11s) |
[production] |
| 20:14 |
<mholloway-shell@deploy1001> |
Synchronized php-1.35.0-wmf.18/extensions/MachineVision: MachineVision: Fix page id parsing from imageinfo results (T244752) (duration: 01m 15s) |
[production] |
| 19:31 |
<ppchelko@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:570393]] Config: Session Store: Switch group0 and group1 to kask-session T243106 (duration: 01m 06s) |
[production] |
| 19:28 |
<mutante> |
Gerrit - added eevans to 'wmf-deployment' group (T244508) |
[production] |
| 19:12 |
<jforrester@deploy1001> |
Synchronized wmf-config/CommonSettings.php: T242122 Load new EventStreamConfig extension if so configured (duration: 01m 06s) |
[production] |
| 19:07 |
<jforrester@deploy1001> |
Scap failed!: Call to mwscript eval.php stderr: not empty |
[production] |
| 19:06 |
<jforrester@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: T242122 Set default of wmgUseEventStreamConfig false everywhere (duration: 01m 06s) |
[production] |
| 18:39 |
<twentyafterfour@deploy1001> |
Synchronized php: group1 wikis to 1.35.0-wmf.18 refs T233866 (duration: 01m 05s) |
[production] |
| 18:38 |
<twentyafterfour@deploy1001> |
rebuilt and synchronized wikiversions files: group1 wikis to 1.35.0-wmf.18 refs T233866 |
[production] |
| 18:25 |
<twentyafterfour@deploy1001> |
rebuilt and synchronized wikiversions files: group0 wikis to 1.35.0-wmf.18 refs T233867 |
[production] |
| 18:21 |
<twentyafterfour> |
MediaWiki train: finally moving forward with group0 wikis to 1.35.0-wmf.18 refs T233866 |
[production] |
| 17:52 |
<jforrester@deploy1001> |
Synchronized wmf-config/CommonSettings.php: T244561 Set Kartographer servers to Wikimedia servers (duration: 01m 06s) |
[production] |
| 16:48 |
<moritzm> |
installing libexif security updates on jessie |
[production] |
| 16:22 |
<vgutierrez> |
pooling cp5002 and cp5009 running buster - T242093 |
[production] |
| 15:45 |
<XioNoX> |
push outbound flowspec support to core routers |
[production] |
| 15:45 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1107 after first day of 10.4 testing - T242702', diff saved to https://phabricator.wikimedia.org/P10366 and previous config saved to /var/cache/conftool/dbconfig/20200210-154552-marostegui.json |
[production] |
| 15:41 |
<pt1979@cumin2001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 15:41 |
<pt1979@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 15:33 |
<godog> |
roll restart cassandra on session* to apply logging changes - T242585 |
[production] |
| 15:23 |
<moritzm> |
uploading debdeploy 0.0.99.13 to apt.wikimedia.org |
[production] |
| 15:22 |
<godog> |
roll restart cassandra on restbase* to apply logging changes - T242585 |
[production] |
| 15:19 |
<vgutierrez@cumin1001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 15:19 |
<vgutierrez@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 15:19 |
<vgutierrez@cumin1001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 15:19 |
<vgutierrez@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 15:06 |
<marostegui> |
Reload haproxy on dbproxy1017 and dbproxy1017 - T244209 |
[production] |
| 15:04 |
<twentyafterfour@deploy1001> |
Finished scap: full scap sync prior to wmf.18 rollout (duration: 20m 13s) |
[production] |
| 15:04 |
<godog> |
roll restart cassandra on maps* to apply logging changes - T242585 |
[production] |
| 15:03 |
<vgutierrez> |
rolling restart of ats-tls - T240950 |
[production] |
| 15:01 |
<godog> |
roll restart cassandra on maps* to apply logging changes - T242585 |
[production] |
| 15:01 |
<godog> |
roll restart cassandra on maps* to apply logging changes - T242585 |
[production] |
| 15:00 |
<marostegui> |
Restart mysql on m5 master (wikitech will go down) - T244209 |
[production] |
| 14:52 |
<vgutierrez> |
rolling restart of ats-tls in ulsfo - T244464 |
[production] |
| 14:46 |
<vgutierrez> |
depool cp5002 and cp5009 and reimage as buster - T242093 |
[production] |
| 14:44 |
<twentyafterfour@deploy1001> |
Started scap: full scap sync prior to wmf.18 rollout |
[production] |
| 14:42 |
<vgutierrez> |
repool cp5003 and cp5010 running buster - T242093 |
[production] |
| 14:41 |
<marostegui> |
Full-upgrade db1133 (without restarting mysql) - T244209 |
[production] |
| 14:40 |
<twentyafterfour> |
MediaWiki Train: Running a full scap to prepare for moving forward to 1.35.0-wmf.18 ( T233866 ) |
[production] |
| 14:32 |
<marostegui> |
Downtime m5 hosts for the upcoming maintenance - T244209 |
[production] |
| 14:19 |
<vgutierrez@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 14:17 |
<vgutierrez@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |