|
2019-02-14
§
|
| 16:39 |
<marostegui> |
Depool labsdb1009 - T210713 |
[production] |
| 16:26 |
<fsero> |
upgrading tiller on codfw |
[production] |
| 16:11 |
<fsero> |
updating tiller version on staging cluster |
[production] |
| 16:10 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-codfw.php: Repool db2085 - T214840 (duration: 00m 52s) |
[production] |
| 15:50 |
<fsero> |
building and publishing new tiller docker image on boron |
[production] |
| 15:50 |
<END> |
(PASS) - Cookbook sre.hosts.upgrade-and-reboot (exit_code=0) (volans@cumin1001) |
[production] |
| 15:43 |
<START> |
- Cookbook sre.hosts.upgrade-and-reboot (volans@cumin1001) |
[production] |
| 15:28 |
<volans> |
upgraded spicerack to v0.0.15 on cumin[12]001 |
[production] |
| 15:26 |
<volans> |
uploaded spicerack_0.0.15-1_amd64.deb to apt.wikimedia.org stretch-wikimedia |
[production] |
| 15:12 |
<marostegui> |
Clear idrac logs from db2085 - T214840 |
[production] |
| 14:45 |
<godog> |
depool and stop logstash1009 for stretch reimage - T213898 |
[production] |
| 14:20 |
<marostegui> |
Stop MySQL on db2085 for on-site maintenance - T214840 |
[production] |
| 14:12 |
<jijiki> |
Enabling puppet on thumbor* servers - T214597 |
[production] |
| 13:39 |
<arturo> |
T215892 icinga downtime cloudvirt1024 for 2 weeks |
[production] |
| 12:22 |
<zeljkof> |
EU SWAT finished |
[production] |
| 12:21 |
<zfilipin@deploy1001> |
Synchronized php-1.33.0-wmf.17/extensions/ExternalGuidance/: SWAT: [[gerrit:490523|Fix the eventlogging schema definition as per manifest_version=2]] (duration: 00m 55s) |
[production] |
| 11:43 |
<_joe_> |
restarting hhvm on mw1338, hot tc exhausted T216084 |
[production] |
| 11:04 |
<_joe_> |
upgrading python3-etcd on stretch T209136 |
[production] |
| 11:03 |
<jbond42> |
rolling security updates for curl |
[production] |
| 11:02 |
<jijiki> |
Disabling puppet on thumbor* servers - T214597 |
[production] |
| 10:59 |
<moritzm> |
installing python3.4 security updates |
[production] |
| 10:53 |
<godog> |
bounce prometheus instances on prometheus2004 to take a snapshot |
[production] |
| 08:10 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool db1106 T214840 (duration: 00m 52s) |
[production] |
| 07:57 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool db1087 T210713 (duration: 00m 54s) |
[production] |
| 07:36 |
<marostegui> |
Stop MySQL on db1106 for reboot - T214840 |
[production] |
| 06:10 |
<marostegui> |
Deploy schema change on db1087 with replication, lag will be generated on labsdb:s8 T210713 |
[production] |
| 06:10 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Depool db1087 T210713 (duration: 00m 55s) |
[production] |
| 01:52 |
<mutante> |
scandium - removing parsoid deploy dir and letting puppet re-clone it after merging gerrit fix 484602 - replace manual clone with proper puppetization (T201366) |
[production] |
| 01:52 |
<mutante> |
scandium - removing parsoid deploy dir and letting puppet re-clone it after merging gerrit fix 484602 - replace manual hack with proper puppet |
[production] |
| 01:15 |
<mutante> |
phab1001 - phabricator mail config converted to cluster.mailers to adjust to upstream change (T212989) |
[production] |
| 00:36 |
<bd808@deploy1001> |
Finished deploy [scholarships/scholarships@1d89fe2]: Live hack PHPMailer namespace T215302 (duration: 00m 02s) |
[production] |
| 00:36 |
<bd808@deploy1001> |
Started deploy [scholarships/scholarships@1d89fe2]: Live hack PHPMailer namespace T215302 |
[production] |
| 00:32 |
<catrope@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Enable ORES (damaging only) on itwiki (T211032) (duration: 00m 53s) |
[production] |
| 00:24 |
<catrope@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Enable help panel search on cswiki and kowiki (T209301) (duration: 00m 55s) |
[production] |
|
2019-02-13
§
|
| 23:42 |
<niharika29@deploy1001> |
Finished deploy [scholarships/scholarships@1d89fe2]: Update scholarships app for 2019 cycle T215302 (duration: 00m 02s) |
[production] |
| 23:42 |
<niharika29@deploy1001> |
Started deploy [scholarships/scholarships@1d89fe2]: Update scholarships app for 2019 cycle T215302 |
[production] |
| 21:31 |
<jijiki> |
Restarting nutcracker on scb100*.eqiad.wmnet |
[production] |
| 20:54 |
<mutante> |
ruthenium - shell access for parsoid-testers revoked by puppet, please use scandium.eqiad.wmnet (T201366) |
[production] |
| 20:44 |
<otto@deploy1001> |
Started restart [eventstreams/deploy@07033d4]: bouncing eventstreams to apply page-links-change stream config |
[production] |
| 20:43 |
<mutante> |
ms-be2021 - powercycling |
[production] |
| 20:09 |
<thcipriani@deploy1001> |
Synchronized php: group1 wikis to 1.33.0-wmf.17 (duration: 00m 53s) |
[production] |
| 20:08 |
<thcipriani@deploy1001> |
rebuilt and synchronized wikiversions files: group1 wikis to 1.33.0-wmf.17 |
[production] |
| 19:55 |
<mforns@deploy1001> |
Finished deploy [analytics/refinery@5f1461e]: Deploying analytics refinery with refinery-source v0.0.85 jars (duration: 07m 36s) |
[production] |
| 19:48 |
<mforns@deploy1001> |
Started deploy [analytics/refinery@5f1461e]: Deploying analytics refinery with refinery-source v0.0.85 jars |
[production] |
| 18:13 |
<jynus@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool es1014 (duration: 00m 52s) |
[production] |
| 18:06 |
<godog> |
reimage prometheus2003 - T187987 |
[production] |
| 18:00 |
<krinkle@deploy1001> |
Synchronized php-1.33.0-wmf.17/includes/libs/rdbms/loadbalancer/LoadBalancer.php: Id70fdfa62ef / T215611 (duration: 00m 55s) |
[production] |
| 17:49 |
<marostegui> |
Stop MYSQL on db1114 for onsite maintenance - T214720 |
[production] |
| 17:25 |
<jijiki> |
Pooling mw1299 back - T215569 |
[production] |
| 17:06 |
<cmjohnson1> |
db1106, troubleshooting idrac issue and updating f/w |
[production] |