|
2019-12-02
§
|
| 23:37 |
<dzahn@cumin1001> |
conftool action : set/pooled=yes; selector: name=mw2251.codfw.wmnet |
[production] |
| 23:06 |
<dzahn@cumin1001> |
conftool action : set/pooled=yes; selector: name=mw2249.codfw.wmnet |
[production] |
| 23:06 |
<dzahn@cumin1001> |
conftool action : set/pooled=yes; selector: name=mw2248.codfw.wmnet |
[production] |
| 23:05 |
<mutante> |
mw2248 - restart nginx (for some reason unit was running but not listening on 443 after reimage..now it does) |
[production] |
| 23:05 |
<bblack@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 23:02 |
<bblack@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 22:46 |
<ejegg> |
updated payments-wiki from 06a8c3cdff to f61c9f0692 |
[production] |
| 22:44 |
<bblack> |
reimaging dns4002 to buster - T239667 |
[production] |
| 22:07 |
<mholloway-shell@deploy1001> |
Synchronized php-1.35.0-wmf.8/extensions/MachineVision: Update text for no personal uploads message (T238873) (duration: 01m 03s) |
[production] |
| 22:05 |
<dzahn@cumin1001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 22:03 |
<dzahn@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 22:01 |
<dzahn@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 22:00 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 22:00 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 21:59 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 21:32 |
<dzahn@cumin1001> |
conftool action : set/pooled=yes; selector: name=mw2247.codfw.wmnet |
[production] |
| 21:31 |
<dzahn@cumin1001> |
conftool action : set/pooled=yes; selector: name=mw2246.codfw.wmnet |
[production] |
| 21:25 |
<dzahn@cumin1001> |
conftool action : set/pooled=yes; selector: name=mw2230.codfw.wmnet |
[production] |
| 21:25 |
<mholloway-shell@deploy1001> |
helmfile [CODFW] Ran 'apply' command on namespace 'wikifeeds' for release 'production' . |
[production] |
| 21:23 |
<mholloway-shell@deploy1001> |
helmfile [EQIAD] Ran 'apply' command on namespace 'wikifeeds' for release 'production' . |
[production] |
| 21:22 |
<mholloway-shell@deploy1001> |
helmfile [STAGING] Ran 'apply' command on namespace 'wikifeeds' for release 'staging' . |
[production] |
| 20:59 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repool db1078 after schema change', diff saved to https://phabricator.wikimedia.org/P9796 and previous config saved to /var/cache/conftool/dbconfig/20191202-205904-marostegui.json |
[production] |
| 20:47 |
<ariel@cumin1001> |
conftool action : set/pooled=yes; selector: cluster=appserver,name=mw2232.codfw.wmnet,service=nginx,dc=codfw |
[production] |
| 20:47 |
<ariel@cumin1001> |
conftool action : set/pooled=yes; selector: cluster=appserver,name=mw2232.codfw.wmnet,service=apache2,dc=codfw |
[production] |
| 20:47 |
<ariel@cumin1001> |
conftool action : set/pooled=yes; selector: name=mw2233.codfw.wmnet,dc=codfw,service=nginx,cluster=appserver |
[production] |
| 20:46 |
<ariel@cumin1001> |
conftool action : set/pooled=yes; selector: name=mw2233.codfw.wmnet,dc=codfw,service=apache2,cluster=appserver |
[production] |
| 20:46 |
<ariel@cumin1001> |
conftool action : set/pooled=yes; selector: name=mw2234.codfw.wmnet,service=nginx,cluster=appserver,dc=codfw |
[production] |
| 20:46 |
<ariel@cumin1001> |
conftool action : set/pooled=yes; selector: name=mw2234.codfw.wmnet,service=apache2,cluster=appserver,dc=codfw |
[production] |
| 20:46 |
<ariel@cumin1001> |
conftool action : set/pooled=yes; selector: cluster=appserver,name=mw2231.codfw.wmnet,service=nginx,dc=codfw |
[production] |
| 20:46 |
<ariel@cumin1001> |
conftool action : set/pooled=yes; selector: cluster=appserver,name=mw2231.codfw.wmnet,service=apache2,dc=codfw |
[production] |
| 20:36 |
<mobrovac@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Switch Flow on all wikis to Parsoid/PHP - T229015 (duration: 00m 59s) |
[production] |
| 20:35 |
<otto@deploy1001> |
helmfile [STAGING] Ran 'apply' command on namespace 'eventgate-logging-external' for release 'logging-external' . |
[production] |
| 20:26 |
<otto@deploy1001> |
helmfile [STAGING] Ran 'apply' command on namespace 'eventgate-logging-external' for release 'logging-external' . |
[production] |
| 20:21 |
<mobrovac@deploy1001> |
Finished deploy [restbase/deploy@92acf1e]: Switch everything to Parsoid/PHP - T229015 (duration: 14m 59s) |
[production] |
| 20:12 |
<joal@deploy1001> |
Finished deploy [analytics/refinery@9cd234a] (thin): Analytics deploy - Fixes for today deploy (2) (duration: 00m 05s) |
[production] |
| 20:12 |
<joal@deploy1001> |
Started deploy [analytics/refinery@9cd234a] (thin): Analytics deploy - Fixes for today deploy (2) |
[production] |
| 20:08 |
<joal@deploy1001> |
Finished deploy [analytics/refinery@9cd234a]: Analytics deploy - Fixes for today deploy (2) (duration: 08m 08s) |
[production] |
| 20:06 |
<mobrovac@deploy1001> |
Started deploy [restbase/deploy@92acf1e]: Switch everything to Parsoid/PHP - T229015 |
[production] |
| 20:05 |
<reedy@deploy1001> |
Synchronized wmf-config/LabsServices.php: labslabslabs (duration: 01m 08s) |
[production] |
| 20:05 |
<mobrovac@deploy1001> |
Finished deploy [restbase/deploy@92acf1e] (dev-cluster): Switch everything to Parsoid/PHP (duration: 02m 48s) |
[production] |
| 20:02 |
<mobrovac@deploy1001> |
Started deploy [restbase/deploy@92acf1e] (dev-cluster): Switch everything to Parsoid/PHP |
[production] |
| 20:01 |
<dzahn@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 19:59 |
<joal@deploy1001> |
Started deploy [analytics/refinery@9cd234a]: Analytics deploy - Fixes for today deploy (2) |
[production] |
| 19:59 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 19:56 |
<ariel@cumin1001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 19:55 |
<ariel@cumin1001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |