|
2020-01-17
§
|
| 07:39 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Slowly repool db1082', diff saved to https://phabricator.wikimedia.org/P10198 and previous config saved to /var/cache/conftool/dbconfig/20200117-073917-marostegui.json |
[production] |
| 07:25 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Slowly repool db1082', diff saved to https://phabricator.wikimedia.org/P10197 and previous config saved to /var/cache/conftool/dbconfig/20200117-072544-marostegui.json |
[production] |
| 07:10 |
<marostegui> |
Stop and upgrade db1082 |
[production] |
| 07:06 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool es2019', diff saved to https://phabricator.wikimedia.org/P10193 and previous config saved to /var/cache/conftool/dbconfig/20200117-070636-marostegui.json |
[production] |
| 07:05 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repool es2012', diff saved to https://phabricator.wikimedia.org/P10192 and previous config saved to /var/cache/conftool/dbconfig/20200117-070516-marostegui.json |
[production] |
| 07:03 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool es2012', diff saved to https://phabricator.wikimedia.org/P10191 and previous config saved to /var/cache/conftool/dbconfig/20200117-070320-marostegui.json |
[production] |
| 06:35 |
<marostegui> |
Compress db1125:3314 tables - this will create lag on s4 labs hosts |
[production] |
| 06:28 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Slowly repool db1081', diff saved to https://phabricator.wikimedia.org/P10190 and previous config saved to /var/cache/conftool/dbconfig/20200117-062838-marostegui.json |
[production] |
| 06:16 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Slowly repool db1081', diff saved to https://phabricator.wikimedia.org/P10189 and previous config saved to /var/cache/conftool/dbconfig/20200117-061602-marostegui.json |
[production] |
| 06:03 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Slowly repool db1081', diff saved to https://phabricator.wikimedia.org/P10188 and previous config saved to /var/cache/conftool/dbconfig/20200117-060259-marostegui.json |
[production] |
| 02:45 |
<urandom> |
bootstrapping restbase2022-b — T243000 |
[production] |
| 00:45 |
<mutante> |
urldownloaders - rm /etc/logrotate.d/squid3 ; systemctl start logrotate (this fixes failed logrotate because of squid3 vs squid file = duplicate entry, but puppet will recreate it) |
[production] |
| 00:33 |
<urandom> |
bootstrapping restbase2022-a — T243000 |
[production] |
|
2020-01-16
§
|
| 22:38 |
<mutante> |
ganeti1003 - deleting VM gerrit-test (T239151) |
[production] |
| 22:37 |
<dzahn@cumin1001> |
END (ERROR) - Cookbook sre.hosts.decommission (exit_code=97) |
[production] |
| 22:37 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.decommission |
[production] |
| 22:34 |
<dzahn@cumin1001> |
END (FAIL) - Cookbook sre.hosts.decommission (exit_code=99) |
[production] |
| 22:34 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.decommission |
[production] |
| 22:21 |
<urandom> |
bootstrapping restbase2021-c — T243000 |
[production] |
| 20:40 |
<mforns@deploy1001> |
Finished deploy [analytics/refinery@26a587a] (thin): deploying analytics-refinery to accompany refinery-source v0.0.112 (duration: 00m 07s) |
[production] |
| 20:40 |
<mforns@deploy1001> |
Started deploy [analytics/refinery@26a587a] (thin): deploying analytics-refinery to accompany refinery-source v0.0.112 |
[production] |
| 20:37 |
<mforns@deploy1001> |
Finished deploy [analytics/refinery@26a587a]: deploying analytics-refinery to accompany refinery-source v0.0.112 (duration: 14m 06s) |
[production] |
| 20:29 |
<jforrester@deploy1001> |
Synchronized php-1.35.0-wmf.15/extensions/CentralAuth/includes/GlobalRename/GlobalRenameBlacklist.php: Special:GlobalRenameRequest: Initialize blacklist even if empty T242974 (duration: 00m 57s) |
[production] |
| 20:23 |
<mforns@deploy1001> |
Started deploy [analytics/refinery@26a587a]: deploying analytics-refinery to accompany refinery-source v0.0.112 |
[production] |
| 20:13 |
<urandom> |
bootstrapping restbase2021-b — T243000 |
[production] |
| 20:01 |
<Urbanecm> |
Purge 12 logos URLs (T150618) |
[production] |
| 20:00 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: 5a32bde: Add logos to IS.php (T150618) (duration: 00m 56s) |
[production] |
| 19:58 |
<urbanecm@deploy1001> |
Synchronized static/images/project-logos/: SWAT: b558eea: Fix mistakes in HD logos (T150618) (duration: 00m 56s) |
[production] |
| 19:33 |
<catrope@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: GrowthExperiments: Enable topic search, behind a hidden preference (T242698) (duration: 00m 56s) |
[production] |
| 19:15 |
<arlolra@deploy1001> |
Finished deploy [parsoid/deploy@7bf9819]: (no justification provided) (duration: 07m 13s) |
[production] |
| 19:08 |
<arlolra@deploy1001> |
Started deploy [parsoid/deploy@7bf9819]: (no justification provided) |
[production] |
| 19:08 |
<catrope@deploy1001> |
Synchronized wmf-config/CommonSettings.php: Remove kask-echoseen-transition definition, now unused (T234963) (duration: 01m 35s) |
[production] |
| 19:05 |
<catrope@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Echo: switch entirely to Kask, remove Redis fallback (T234963) (duration: 00m 56s) |
[production] |
| 19:02 |
<arlolra@deploy1001> |
Finished deploy [parsoid/deploy@7bf9819]: Updating Parsoid to 02f0066 (duration: 08m 30s) |
[production] |
| 18:54 |
<arlolra@deploy1001> |
Started deploy [parsoid/deploy@7bf9819]: Updating Parsoid to 02f0066 |
[production] |
| 18:01 |
<urandom> |
bootstrapping restbase2021-a — T243000 |
[production] |
| 17:48 |
<James_F> |
Manually purged the trwiki logos from Varnish as part of updating them to reflect unblocking, T242977 |
[production] |
| 17:48 |
<jforrester@deploy1001> |
Synchronized static/images/project-logos/trwiki-2x.png: [trwiki] Change logo to reflect unblocking, 2x T242977 (duration: 00m 56s) |
[production] |
| 17:47 |
<jforrester@deploy1001> |
Synchronized static/images/project-logos/trwiki-1.5x.png: [trwiki] Change logo to reflect unblocking, 1.5x T242977 (duration: 00m 55s) |
[production] |
| 17:46 |
<jforrester@deploy1001> |
Synchronized static/images/project-logos/trwiki.png: [trwiki] Change logo to reflect unblocking, 1x T242977 (duration: 00m 56s) |
[production] |
| 17:39 |
<effie> |
Updgrade parsoid to to php 7.2.26 and restart - T241222 |
[production] |
| 17:05 |
<dcausse> |
restarting blazegraph@wdqs1007 (T242453) |
[production] |
| 17:02 |
<jakob@deploy1001> |
helmfile [EQIAD] Ran 'apply' command on namespace 'termbox' for release 'production' . |
[production] |
| 16:53 |
<jakob@deploy1001> |
helmfile [CODFW] Ran 'apply' command on namespace 'termbox' for release 'production' . |
[production] |
| 16:51 |
<jakob@deploy1001> |
helmfile [STAGING] Ran 'apply' command on namespace 'termbox' for release 'staging' . |
[production] |
| 16:31 |
<dcausse> |
depooling wdqs1007, blazegraph stuck (T242453) |
[production] |
| 16:30 |
<jakob@deploy1001> |
helmfile [STAGING] Ran 'apply' command on namespace 'termbox' for release 'test' . |
[production] |
| 15:59 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 15:59 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 15:59 |
<effie> |
Updgrade appservers and api to php 7.2.26 and restart - T241222 |
[production] |