|
2019-02-14
§
|
| 18:03 |
<fsero> |
upgrading tiller to 2.12.2 on eqiad |
[production] |
| 17:35 |
<arturo> |
T215154 tools-sgebastion-07 now running systemd 239 and starts enforcing user limits |
[tools] |
| 17:34 |
<godog> |
bounce rsyslog on wezen/lithium, tls listener timeout in icinga |
[production] |
| 16:59 |
<moritzm> |
restarting apertium-apy on scb1001 to pick up Python security update |
[production] |
| 16:39 |
<marostegui> |
Depool labsdb1009 - T210713 |
[production] |
| 16:26 |
<fsero> |
upgrading tiller on codfw |
[production] |
| 16:11 |
<fsero> |
updating tiller version on staging cluster |
[production] |
| 16:10 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-codfw.php: Repool db2085 - T214840 (duration: 00m 52s) |
[production] |
| 15:50 |
<fsero> |
building and publishing new tiller docker image on boron |
[production] |
| 15:50 |
<END> |
(PASS) - Cookbook sre.hosts.upgrade-and-reboot (exit_code=0) (volans@cumin1001) |
[production] |
| 15:43 |
<START> |
- Cookbook sre.hosts.upgrade-and-reboot (volans@cumin1001) |
[production] |
| 15:33 |
<andrewbogott> |
moving tools-worker-1002, 1003, 1005, 1006, 1007, 1010, 1013, 1014 to different labvirts in order to move labvirt1012 to eqiad1-r |
[tools] |
| 15:28 |
<volans> |
upgraded spicerack to v0.0.15 on cumin[12]001 |
[production] |
| 15:26 |
<volans> |
uploaded spicerack_0.0.15-1_amd64.deb to apt.wikimedia.org stretch-wikimedia |
[production] |
| 15:12 |
<marostegui> |
Clear idrac logs from db2085 - T214840 |
[production] |
| 14:45 |
<godog> |
depool and stop logstash1009 for stretch reimage - T213898 |
[production] |
| 14:20 |
<marostegui> |
Stop MySQL on db2085 for on-site maintenance - T214840 |
[production] |
| 14:12 |
<jijiki> |
Enabling puppet on thumbor* servers - T214597 |
[production] |
| 13:39 |
<arturo> |
T215892 icinga downtime cloudvirt1024 for 2 weeks |
[production] |
| 13:24 |
<thcipriani> |
rearm keyholder on deployment-deploy01: sudo keyholder arm, passwords on https://wikitech.wikimedia.org/wiki/Keyholder |
[releng] |
| 12:22 |
<zeljkof> |
EU SWAT finished |
[production] |
| 12:21 |
<zfilipin@deploy1001> |
Synchronized php-1.33.0-wmf.17/extensions/ExternalGuidance/: SWAT: [[gerrit:490523|Fix the eventlogging schema definition as per manifest_version=2]] (duration: 00m 55s) |
[production] |
| 11:43 |
<_joe_> |
restarting hhvm on mw1338, hot tc exhausted T216084 |
[production] |
| 11:04 |
<_joe_> |
upgrading python3-etcd on stretch T209136 |
[production] |
| 11:03 |
<jbond42> |
rolling security updates for curl |
[production] |
| 11:02 |
<jijiki> |
Disabling puppet on thumbor* servers - T214597 |
[production] |
| 10:59 |
<moritzm> |
installing python3.4 security updates |
[production] |
| 10:53 |
<godog> |
bounce prometheus instances on prometheus2004 to take a snapshot |
[production] |
| 09:07 |
<joal> |
rerun mediawiki-history-wikitext-wf-2019-01 |
[analytics] |
| 09:06 |
<joal> |
Re-run webrequest-load-wf-text-2019-2-14-6 |
[analytics] |
| 08:10 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool db1106 T214840 (duration: 00m 52s) |
[production] |
| 07:57 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool db1087 T210713 (duration: 00m 54s) |
[production] |
| 07:36 |
<marostegui> |
Stop MySQL on db1106 for reboot - T214840 |
[production] |
| 06:10 |
<marostegui> |
Deploy schema change on db1087 with replication, lag will be generated on labsdb:s8 T210713 |
[production] |
| 06:10 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Depool db1087 T210713 (duration: 00m 55s) |
[production] |
| 01:52 |
<mutante> |
scandium - removing parsoid deploy dir and letting puppet re-clone it after merging gerrit fix 484602 - replace manual clone with proper puppetization (T201366) |
[production] |
| 01:52 |
<mutante> |
scandium - removing parsoid deploy dir and letting puppet re-clone it after merging gerrit fix 484602 - replace manual hack with proper puppet |
[production] |
| 01:15 |
<mutante> |
phab1001 - phabricator mail config converted to cluster.mailers to adjust to upstream change (T212989) |
[production] |
| 00:36 |
<bd808@deploy1001> |
Finished deploy [scholarships/scholarships@1d89fe2]: Live hack PHPMailer namespace T215302 (duration: 00m 02s) |
[production] |
| 00:36 |
<bd808@deploy1001> |
Started deploy [scholarships/scholarships@1d89fe2]: Live hack PHPMailer namespace T215302 |
[production] |
| 00:32 |
<catrope@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Enable ORES (damaging only) on itwiki (T211032) (duration: 00m 53s) |
[production] |
| 00:24 |
<catrope@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Enable help panel search on cswiki and kowiki (T209301) (duration: 00m 55s) |
[production] |