|
2019-11-07
ยง
|
| 23:44 |
<twentyafterfour@deploy1001> |
Started deploy [releng/phatality@62e2870]: revert phatalaty again |
[production] |
| 23:41 |
<twentyafterfour@deploy1001> |
Finished deploy [releng/phatality@11d4ad8]: one more time (duration: 03m 00s) |
[production] |
| 23:38 |
<twentyafterfour@deploy1001> |
Started deploy [releng/phatality@11d4ad8]: one more time |
[production] |
| 23:31 |
<twentyafterfour@deploy1001> |
Finished deploy [releng/phatality@11d4ad8]: trying again with a longer scap timeout (duration: 03m 02s) |
[production] |
| 23:28 |
<twentyafterfour@deploy1001> |
Started deploy [releng/phatality@11d4ad8]: trying again with a longer scap timeout |
[production] |
| 23:23 |
<twentyafterfour@deploy1001> |
Finished deploy [releng/phatality@62e2870]: revert to previous phatality plugin version (duration: 02m 55s) |
[production] |
| 23:20 |
<twentyafterfour@deploy1001> |
Started deploy [releng/phatality@62e2870]: revert to previous phatality plugin version |
[production] |
| 23:09 |
<twentyafterfour@deploy1001> |
Finished deploy [releng/phatality@11d4ad8]: (no justification provided) (duration: 00m 06s) |
[production] |
| 23:09 |
<twentyafterfour@deploy1001> |
Started deploy [releng/phatality@11d4ad8]: (no justification provided) |
[production] |
| 23:04 |
<twentyafterfour@deploy1001> |
Finished deploy [releng/phatality@11d4ad8]: (no justification provided) (duration: 06m 48s) |
[production] |
| 23:00 |
<XenoRyet> |
updated payments-wiki from aac3d93f70 to bd907656fb |
[production] |
| 22:57 |
<twentyafterfour@deploy1001> |
Started deploy [releng/phatality@11d4ad8]: (no justification provided) |
[production] |
| 22:53 |
<volker-e@deploy1001> |
Finished deploy [design/style-guide@4abbc70]: Update responsive Illustrations styles changes (duration: 00m 05s) |
[production] |
| 22:53 |
<volker-e@deploy1001> |
Started deploy [design/style-guide@4abbc70]: Update responsive Illustrations styles changes |
[production] |
| 22:32 |
<mholloway-shell@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: MachineVision: Remove annotation job delay (duration: 00m 53s) |
[production] |
| 22:03 |
<volker-e@deploy1001> |
Finished deploy [design/style-guide@4abbc70]: Update to latest master with components overview additions (duration: 00m 06s) |
[production] |
| 22:03 |
<volker-e@deploy1001> |
Started deploy [design/style-guide@4abbc70]: Update to latest master with components overview additions |
[production] |
| 21:54 |
<andrewbogott> |
rebuilding labtestpuppetmaster2001 w/Stretch |
[production] |
| 21:43 |
<dzahn@cumin1001> |
conftool action : set/pooled=yes; selector: name=wtp2020.codfw.wmnet |
[production] |
| 21:28 |
<mutante> |
boron apt-get clean (saved 9G on /) (T237649) |
[production] |
| 20:42 |
<twentyafterfour@deploy1001> |
rebuilt and synchronized wikiversions files: group2 wikis to 1.35.0-wmf.5 refs T233853 |
[production] |
| 20:24 |
<catrope@deploy1001> |
Synchronized php-1.35.0-wmf.5/extensions/VisualEditor/modules/ve-mw/init/targets/ve.init.mw.ArticleTarget.js: Fix error handling (duration: 01m 00s) |
[production] |
| 20:21 |
<herron> |
performing rolling reboots of kafka-main hosts for security updates |
[production] |
| 20:17 |
<onimisionipe> |
cluster restart for cloudelastic to pick JVM upgrade |
[production] |
| 20:08 |
<eileen> |
civicrm revision changed from f1ce5c86f7 to bfa53ee611, config revision is 72d2692743 |
[production] |
| 19:54 |
<mholloway-shell@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: MachineVision: Enqueue annotation job on upload complete (duration: 05m 19s) |
[production] |
| 18:31 |
<mholloway-shell@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: MachineVision: Disable retrying annotation requests (duration: 05m 17s) |
[production] |
| 18:25 |
<ebernhardson> |
restart mjolnir-kafka-bulk-daemon and mjolnir-kafka-msearch-daemon across `cirrus` dsh group |
[production] |
| 18:20 |
<ebernhardson@deploy1001> |
Finished deploy [search/mjolnir/deploy@afd41d7]: bulk_daemon: Adjust glent configuration (duration: 05m 49s) |
[production] |
| 18:14 |
<ebernhardson@deploy1001> |
Started deploy [search/mjolnir/deploy@afd41d7]: bulk_daemon: Adjust glent configuration |
[production] |
| 17:44 |
<elukey@cumin1001> |
END (PASS) - Cookbook sre.zookeeper.roll-restart-zookeeper (exit_code=0) |
[production] |
| 17:39 |
<elukey@cumin1001> |
END (PASS) - Cookbook sre.hadoop.roll-restart-workers (exit_code=0) |
[production] |
| 17:38 |
<elukey@cumin1001> |
START - Cookbook sre.zookeeper.roll-restart-zookeeper |
[production] |
| 17:37 |
<elukey@cumin1001> |
END (PASS) - Cookbook sre.zookeeper.roll-restart-zookeeper (exit_code=0) |
[production] |
| 17:30 |
<elukey@cumin1001> |
START - Cookbook sre.zookeeper.roll-restart-zookeeper |
[production] |
| 17:25 |
<mholloway-shell@deploy1001> |
Synchronized php-1.35.0-wmf.5/extensions/MachineVision: Drop currently unsupported external dependencies (T227349) (duration: 05m 19s) |
[production] |
| 17:10 |
<XioNoX> |
Homer push - forwarding-options - to all cr |
[production] |
| 17:09 |
<elukey@cumin1001> |
START - Cookbook sre.hadoop.roll-restart-workers |
[production] |
| 17:08 |
<XioNoX> |
add sampling stanza (disabled) to cr2-esams |
[production] |
| 17:00 |
<mutante> |
wtp2020 - 2 hours downtime - shut down (T205712) - go ahead @papaul |
[production] |
| 17:00 |
<elukey@cumin1001> |
END (PASS) - Cookbook sre.hadoop.roll-restart-workers (exit_code=0) |
[production] |
| 16:58 |
<mutante> |
wtp2020 - depooled for T205712 |
[production] |
| 16:58 |
<dzahn@cumin1001> |
conftool action : set/pooled=no; selector: name=wtp2020.codfw.wmnet |
[production] |
| 16:42 |
<reedy@deploy1001> |
Synchronized wmf-config/InitialiseSettings-labs.php: some alphasorted config (duration: 01m 00s) |
[production] |
| 16:34 |
<XioNoX> |
Homer push on cr2-knams: Sampling (disabled), enhanced-hash-key, ospf interfaces re-ordering (noop), policy-statement BGP_from_LVS (unused), lo0 term allow_vmhost |
[production] |
| 16:32 |
<jynus@cumin1001> |
dbctl commit (dc=all): 'Repool db1080 at 100%', diff saved to https://phabricator.wikimedia.org/P9553 and previous config saved to /var/cache/conftool/dbconfig/20191107-163235-jynus.json |
[production] |
| 16:20 |
<XioNoX> |
add BGP sessions to AS64050 in eqiad |
[production] |
| 16:15 |
<XioNoX> |
add BGP sessions to AS57695 in esams and eqiad |
[production] |
| 16:12 |
<XioNoX> |
clear v4 BGP sessions to AS7713 in eqsin (hit max prefix limit) |
[production] |
| 16:02 |
<mutante> |
mw2225 restart cron (T236799) |
[production] |