|
2020-04-17
§
|
| 07:45 |
<gehel> |
restart wdqs-updater on all nodes after deployment |
[production] |
| 06:31 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Fully repool db1092 after compression', diff saved to https://phabricator.wikimedia.org/P11005 and previous config saved to /var/cache/conftool/dbconfig/20200417-063138-marostegui.json |
[production] |
| 06:30 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Remove db1111 from API', diff saved to https://phabricator.wikimedia.org/P11004 and previous config saved to /var/cache/conftool/dbconfig/20200417-063038-marostegui.json |
[production] |
| 06:26 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Slowly repool db1092 after compression', diff saved to https://phabricator.wikimedia.org/P11003 and previous config saved to /var/cache/conftool/dbconfig/20200417-062642-marostegui.json |
[production] |
| 06:19 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Slowly repool db1092 after compression', diff saved to https://phabricator.wikimedia.org/P11002 and previous config saved to /var/cache/conftool/dbconfig/20200417-061907-marostegui.json |
[production] |
| 06:04 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Slowly repool db1092 after compression', diff saved to https://phabricator.wikimedia.org/P11001 and previous config saved to /var/cache/conftool/dbconfig/20200417-060419-marostegui.json |
[production] |
|
2020-04-16
§
|
| 22:34 |
<maryum> |
reindexing wikis that failed from previous reindex on mwmain1002 |
[production] |
| 22:10 |
<jforrester@deploy1001> |
Pruned MediaWiki: 1.35.0-wmf.26 (duration: 05m 26s) |
[production] |
| 21:59 |
<jforrester@deploy1001> |
Synchronized php-1.35.0-wmf.28/extensions/FlaggedRevs/: T250439 Don't try to create a Revision with null (duration: 01m 02s) |
[production] |
| 21:54 |
<bsitzmann@deploy1001> |
helmfile [CODFW] Ran 'apply' command on namespace 'wikifeeds' for release 'production' . |
[production] |
| 21:51 |
<bsitzmann@deploy1001> |
helmfile [EQIAD] Ran 'apply' command on namespace 'wikifeeds' for release 'production' . |
[production] |
| 21:48 |
<mholloway-shell@deploy1001> |
helmfile [STAGING] Ran 'apply' command on namespace 'wikifeeds' for release 'staging' . |
[production] |
| 20:42 |
<mstyles@deploy1001> |
Finished deploy [wdqs/wdqs@1fb52b3]: WDQS version 0.3.22 (duration: 11m 43s) |
[production] |
| 20:30 |
<mstyles@deploy1001> |
Started deploy [wdqs/wdqs@1fb52b3]: WDQS version 0.3.22 |
[production] |
| 20:01 |
<maryum> |
"beginning deploy of WDQS 0.3.22" |
[production] |
| 19:06 |
<jforrester@deploy1001> |
rebuilt and synchronized wikiversions files: all wikis to 1.35.0-wmf.28 |
[production] |
| 18:57 |
<krinkle@deploy1001> |
Synchronized errorpages/404.php: I9fd5c99130c64 (duration: 01m 07s) |
[production] |
| 17:52 |
<XioNoX> |
rename/format asw-ulsfo interfaces to match future homer driven format |
[production] |
| 16:51 |
<herron> |
kafka-logging eqiad set retention.bytes=500000000000 on topic udp_localhost-warning T250133 |
[production] |
| 16:45 |
<herron> |
kafka-logging eqiad set retention.bytes=500000000000 on topic udp_localhost-info T250133 |
[production] |
| 16:30 |
<hnowlan@deploy1001> |
helmfile [STAGING] Ran 'sync' command on namespace 'changeprop' for release 'staging' . |
[production] |
| 15:54 |
<elukey> |
restart chi on cloudelastic1001 with -XX:NewRatio=3 - T231517 |
[production] |
| 15:26 |
<akosiaris> |
truncate /var/log/ganeti/monitoring-daemon-error.log on ganeti1003, start again all ganeti daemons |
[production] |
| 15:20 |
<akosiaris> |
stop ganeti daemons on ganeti1003 |
[production] |
| 15:02 |
<Urbanecm> |
mwscript extensions/CentralAuth/maintenance/fixStuckGlobalRename.php --wiki=commonswiki --logwiki=metawiki 'Petri Gyula' '23eki' (T250387) |
[production] |
| 14:51 |
<hknust> |
holger@mwmaint1002 END (Fail) uppercaseTitlesForUnicodeTransition.php as part of T219279 |
[production] |
| 14:30 |
<hknust> |
holger@mwmaint1002 Starting uppercaseTitlesForUnicodeTransition.php as part of T219279 |
[production] |
| 14:21 |
<gehel@cumin1001> |
END (FAIL) - Cookbook sre.wdqs.data-reload (exit_code=99) |
[production] |
| 14:17 |
<hnowlan@deploy1001> |
Finished deploy [changeprop/deploy@354ae2d]: Enabling rules on k8s, disabling on scb (duration: 01m 12s) |
[production] |
| 14:16 |
<hnowlan@deploy1001> |
Started deploy [changeprop/deploy@354ae2d]: Enabling rules on k8s, disabling on scb |
[production] |
| 14:14 |
<dcausse> |
elastic (search cluster) reindexing commonswiki_content in codfw and ediad (T246882) |
[production] |
| 14:13 |
<ema> |
cache: upgrade varnish to 5.1.3-1wm14 and rolling restart T249810 |
[production] |
| 13:40 |
<XioNoX> |
rename/format asw2-esams interfaces to match future homer driven format |
[production] |
| 13:36 |
<kormat> |
Optimizing all tables on pc1010 T247787 |
[production] |
| 13:32 |
<hashar> |
Restarting CI Jenkins for plugin upgrade T250377 |
[production] |
| 13:04 |
<hnowlan@deploy1001> |
Finished deploy [changeprop/deploy@baf0a4b]: Rollback removing k8s rules, again (duration: 00m 30s) |
[production] |
| 13:04 |
<hnowlan@deploy1001> |
Started deploy [changeprop/deploy@baf0a4b]: Rollback removing k8s rules, again |
[production] |
| 13:03 |
<gehel@cumin1001> |
START - Cookbook sre.wdqs.data-reload |
[production] |
| 12:54 |
<gehel@cumin1001> |
END (ERROR) - Cookbook sre.wdqs.data-reload (exit_code=97) |
[production] |
| 12:54 |
<gehel@cumin1001> |
START - Cookbook sre.wdqs.data-reload |
[production] |
| 12:48 |
<vgutierrez> |
pool cp1087 |
[production] |
| 12:44 |
<jynus> |
test sal again |
[production] |
| 12:37 |
<vgutierrez> |
depool & reboot cp1087 |
[production] |
| 11:29 |
<elukey> |
restart atskafka on cp3050 after maintenance |
[production] |
| 11:22 |
<XioNoX> |
rename/format asw1-eqsin interfaces to match future homer driven format |
[production] |
| 11:17 |
<elukey> |
stop atskafka on cp3050 to re-create the topic atskafka_test_webrequest_text on Kafka Jumbo - T250347 |
[production] |
| 11:16 |
<Urbanecm> |
EU SWAT done |
[production] |
| 11:14 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: a105f38: Remove broken groupOverrides from amwikimedia (T249585) (duration: 01m 05s) |
[production] |
| 11:12 |
<urbanecm@deploy1001> |
Synchronized wmf-config/CommonSettings.php: SWAT: 70ee5f6: Remove grants for tboverride and tboverride-account (T241114) (duration: 01m 06s) |
[production] |
| 11:05 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: 74ad793: Turn off direct account creations at Testwikidata (T250348; take II) (duration: 01m 04s) |
[production] |