|
2022-01-13
ยง
|
| 20:31 |
<dduvall@deploy1002> |
rebuilt and synchronized wikiversions files: Revert "group1 wikis to 1.38.0-wmf.17" |
[production] |
| 20:31 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] START helmfile.d/services/mwdebug: apply on pinkunicorn |
[production] |
| 20:28 |
<dduvall> |
rolling back wmf.17 from group1 due to a large increase in "Parser state cleared while parsing" across commons and group1 wikipedias (T293958, T299149) |
[production] |
| 20:26 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/mwdebug: sync on pinkunicorn |
[production] |
| 20:24 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] START helmfile.d/services/mwdebug: apply on pinkunicorn |
[production] |
| 20:24 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/mwdebug: sync on pinkunicorn |
[production] |
| 20:20 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] START helmfile.d/services/mwdebug: apply on pinkunicorn |
[production] |
| 20:17 |
<dduvall@deploy1002> |
Synchronized php: group1 wikis to 1.38.0-wmf.17 refs T293958 (duration: 01m 06s) |
[production] |
| 20:16 |
<dduvall@deploy1002> |
rebuilt and synchronized wikiversions files: group1 wikis to 1.38.0-wmf.17 refs T293958 |
[production] |
| 20:10 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/mwdebug: sync on pinkunicorn |
[production] |
| 20:08 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] START helmfile.d/services/mwdebug: apply on pinkunicorn |
[production] |
| 20:08 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/mwdebug: sync on pinkunicorn |
[production] |
| 20:07 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] START helmfile.d/services/mwdebug: apply on pinkunicorn |
[production] |
| 20:07 |
<dduvall@deploy1002> |
rebuilt and synchronized wikiversions files: group0 wikis to 1.38.0-wmf.17 refs T293958 |
[production] |
| 20:02 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/mwdebug: sync on pinkunicorn |
[production] |
| 20:01 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] START helmfile.d/services/mwdebug: apply on pinkunicorn |
[production] |
| 20:01 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/mwdebug: sync on pinkunicorn |
[production] |
| 20:00 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] START helmfile.d/services/mwdebug: apply on pinkunicorn |
[production] |
| 19:46 |
<cmjohnson@cumin1001> |
END (PASS) - Cookbook sre.dns.netbox (exit_code=0) |
[production] |
| 19:43 |
<cmjohnson@cumin1001> |
START - Cookbook sre.dns.netbox |
[production] |
| 19:43 |
<bking@cumin2002> |
START - Cookbook sre.hosts.reimage for host elastic2051.codfw.wmnet with OS stretch |
[production] |
| 19:42 |
<bking@cumin2002> |
END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host elastic2051.codfw.wmnet with OS stretch |
[production] |
| 19:40 |
<dzahn@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/miscweb: sync on main |
[production] |
| 19:40 |
<taavi@deploy1002> |
Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:753634|Enable ArticlePlaceholder on dagwiki (T298349)]] (duration: 01m 13s) |
[production] |
| 19:37 |
<dzahn@deploy1002> |
helmfile [eqiad] START helmfile.d/services/miscweb: apply on main |
[production] |
| 19:30 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/mwdebug: sync on pinkunicorn |
[production] |
| 19:29 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] START helmfile.d/services/mwdebug: apply on pinkunicorn |
[production] |
| 19:29 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/mwdebug: sync on pinkunicorn |
[production] |
| 19:27 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] START helmfile.d/services/mwdebug: apply on pinkunicorn |
[production] |
| 19:25 |
<dzahn@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/miscweb: sync on main |
[production] |
| 19:23 |
<dzahn@deploy1002> |
helmfile [codfw] START helmfile.d/services/miscweb: apply on main |
[production] |
| 19:22 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/mwdebug: sync on pinkunicorn |
[production] |
| 19:21 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] START helmfile.d/services/mwdebug: apply on pinkunicorn |
[production] |
| 19:21 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/mwdebug: sync on pinkunicorn |
[production] |
| 19:19 |
<taavi@deploy1002> |
Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:747993|Add event stream config for ios.notification_interaction (T290920)]] (duration: 01m 13s) |
[production] |
| 19:17 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] START helmfile.d/services/mwdebug: apply on pinkunicorn |
[production] |
| 19:15 |
<taavi@deploy1002> |
Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:747991|Add event stream config for android.customize_toolbar_interaction (T297818)]] (duration: 01m 12s) |
[production] |
| 19:12 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/mwdebug: sync on pinkunicorn |
[production] |
| 19:11 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] START helmfile.d/services/mwdebug: apply on pinkunicorn |
[production] |
| 19:10 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/mwdebug: sync on pinkunicorn |
[production] |
| 19:09 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] START helmfile.d/services/mwdebug: apply on pinkunicorn |
[production] |
| 19:07 |
<taavi@deploy1002> |
Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:753793|Enable skin migration mode on the beta cluster]] (duration: 01m 14s) |
[production] |
| 18:59 |
<cmjohnson@cumin1001> |
END (PASS) - Cookbook sre.dns.netbox (exit_code=0) |
[production] |
| 18:42 |
<cmjohnson@cumin1001> |
START - Cookbook sre.dns.netbox |
[production] |
| 17:52 |
<bking@cumin2002> |
START - Cookbook sre.hosts.reimage for host elastic2051.codfw.wmnet with OS stretch |
[production] |
| 17:49 |
<bking@cumin2002> |
END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host elastic2051.codfw.wmnet with OS stretch |
[production] |
| 17:45 |
<hnowlan@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on maps1005.eqiad.wmnet with reason: requires resync after planet sync |
[production] |
| 17:45 |
<hnowlan@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on maps1005.eqiad.wmnet with reason: requires resync after planet sync |
[production] |
| 17:37 |
<hnowlan@cumin1001> |
END (FAIL) - Cookbook sre.postgresql.postgres-init (exit_code=99) |
[production] |
| 17:34 |
<hnowlan@cumin1001> |
START - Cookbook sre.postgresql.postgres-init |
[production] |