|
2020-07-29
§
|
| 06:16 |
<XioNoX> |
standardize mr1-codfw interfaces |
[production] |
| 06:14 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Slowly repool db1078', diff saved to https://phabricator.wikimedia.org/P12104 and previous config saved to /var/cache/conftool/dbconfig/20200729-061450-marostegui.json |
[production] |
| 06:05 |
<XioNoX> |
standardize mr1-ulsfo interfaces |
[production] |
| 06:01 |
<legoktm> |
ssh doc1001.eqiad.wmnet sudo -u doc-uploader git -C /srv/docroot pull |
[production] |
| 05:52 |
<XioNoX> |
standardize mr1-eqsin interfaces |
[production] |
| 05:45 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 05:43 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 05:03 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1078', diff saved to https://phabricator.wikimedia.org/P12103 and previous config saved to /var/cache/conftool/dbconfig/20200729-050346-marostegui.json |
[production] |
| 05:02 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1141', diff saved to https://phabricator.wikimedia.org/P12102 and previous config saved to /var/cache/conftool/dbconfig/20200729-050247-marostegui.json |
[production] |
| 05:02 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Fully repool db1142', diff saved to https://phabricator.wikimedia.org/P12101 and previous config saved to /var/cache/conftool/dbconfig/20200729-050204-marostegui.json |
[production] |
| 04:58 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Slowly repool db1142', diff saved to https://phabricator.wikimedia.org/P12100 and previous config saved to /var/cache/conftool/dbconfig/20200729-045859-marostegui.json |
[production] |
| 02:19 |
<tstarling@deploy1001> |
Synchronized wmf-config/CommonSettings.php: re-enable lilypond in safe mode (duration: 01m 09s) |
[production] |
| 01:47 |
<tstarling@deploy1001> |
Synchronized php-1.36.0-wmf.2/extensions/Score/includes/Score.php: work around firejail bug (duration: 01m 07s) |
[production] |
| 01:45 |
<tstarling@deploy1001> |
Synchronized php-1.36.0-wmf.1/extensions/Score/includes/Score.php: work around firejail bug (duration: 01m 08s) |
[production] |
| 01:19 |
<dzahn@cumin1001> |
conftool action : set/pooled=yes; selector: name=wtp1048.eqiad.wmnet |
[production] |
| 01:15 |
<dzahn@cumin1001> |
conftool action : set/pooled=yes; selector: name=wtp1047.eqiad.wmnet |
[production] |
| 00:53 |
<dzahn@cumin1001> |
conftool action : set/pooled=yes; selector: name=wtp1046.eqiad.wmnet |
[production] |
| 00:48 |
<ryankemper> |
sudo -E cumin -b 10 'A:wdqs-all' 'sudo run-puppet-agent' |
[production] |
| 00:18 |
<dzahn@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 00:14 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
|
2020-07-28
§
|
| 23:38 |
<dzahn@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 23:37 |
<ebernhardson@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: cirrus: reduce mlr window size on enwiki (duration: 01m 05s) |
[production] |
| 23:36 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 23:34 |
<ebernhardson@deploy1001> |
Synchronized wmf-config/CirrusSearch-common.php: cirrus: reduce mlr window size on enwiki (duration: 01m 06s) |
[production] |
| 23:31 |
<dzahn@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 23:29 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 23:04 |
<catrope@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Remove unused setting $wgGEHomepageSuggestedEditsNewAccountInitiatedPercentage (no-op) (duration: 01m 06s) |
[production] |
| 22:55 |
<dzahn@cumin1001> |
conftool action : set/pooled=no; selector: name=wtp1046.eqiad.wmnet |
[production] |
| 22:19 |
<jayme@cumin1001> |
conftool action : set/pooled=yes; selector: name=wtp1044.eqiad.wmnet |
[production] |
| 21:27 |
<dancy@deploy1001> |
helmfile [eqiad] Ran 'sync' command on namespace 'blubberoid' for release 'production' . |
[production] |
| 21:24 |
<dancy@deploy1001> |
helmfile [codfw] Ran 'sync' command on namespace 'blubberoid' for release 'production' . |
[production] |
| 21:17 |
<dancy@deploy1001> |
helmfile [staging] Ran 'sync' command on namespace 'blubberoid' for release 'staging' . |
[production] |
| 20:38 |
<andrew@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 20:36 |
<andrew@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 20:02 |
<eileen> |
process-control config revision is b6ece03513 |
[production] |
| 19:50 |
<andrew@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 19:48 |
<andrew@cumin1001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 19:48 |
<andrew@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 19:48 |
<andrew@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 19:25 |
<andrew@cumin1001> |
END (FAIL) - Cookbook sre.hosts.decommission (exit_code=1) |
[production] |
| 19:25 |
<andrew@cumin1001> |
START - Cookbook sre.hosts.decommission |
[production] |
| 19:24 |
<andrew@cumin1001> |
END (FAIL) - Cookbook sre.hosts.decommission (exit_code=1) |
[production] |
| 19:24 |
<andrew@cumin1001> |
START - Cookbook sre.hosts.decommission |
[production] |
| 19:24 |
<andrew@cumin1001> |
END (FAIL) - Cookbook sre.hosts.decommission (exit_code=1) |
[production] |
| 19:23 |
<andrew@cumin1001> |
START - Cookbook sre.hosts.decommission |
[production] |
| 19:19 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1142', diff saved to https://phabricator.wikimedia.org/P12097 and previous config saved to /var/cache/conftool/dbconfig/20200728-191926-marostegui.json |
[production] |
| 19:12 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Fully repool db1147', diff saved to https://phabricator.wikimedia.org/P12096 and previous config saved to /var/cache/conftool/dbconfig/20200728-191237-marostegui.json |
[production] |
| 19:11 |
<ebernhardson@deploy1001> |
Finished deploy [wikimedia/discovery/analytics@69bbbbb]: airflow: drop_old_data_daily: top_queries table renamed to fulltext_head_queries (duration: 00m 53s) |
[production] |
| 19:11 |
<ebernhardson@deploy1001> |
Started deploy [wikimedia/discovery/analytics@69bbbbb]: airflow: drop_old_data_daily: top_queries table renamed to fulltext_head_queries |
[production] |
| 19:09 |
<jhuneidi@deploy1001> |
helmfile [eqiad] Ran 'sync' command on namespace 'blubberoid' for release 'production' . |
[production] |