|
2017-09-06
ยง
|
| 23:17 |
<twentyafterfour@tin> |
scap failed: average error rate on 9/11 canaries increased by 10x (rerun with --force to override this check, see https://logstash.wikimedia.org/goto/2cc7028226a539553178454fc2f14459 for details) |
[production] |
| 23:12 |
<twentyafterfour@tin> |
Synchronized wmf-config/: roll-back due to huge error rate spike (duration: 00m 51s) |
[production] |
| 23:10 |
<twentyafterfour@tin> |
Synchronized wmf-config/: deploy config for CirrusSearch human relevancy survey. Change-Id: I272c69e5a3bb6e833fca59282142d6b237fd9e60 Bug: T174106 (duration: 00m 52s) |
[production] |
| 22:18 |
<no_justification> |
prior deploy was no-op |
[production] |
| 22:18 |
<demon@tin> |
Finished deploy [gerrit/gerrit@d4f9a77]: (no justification provided) (duration: 00m 07s) |
[production] |
| 22:18 |
<demon@tin> |
Started deploy [gerrit/gerrit@d4f9a77]: (no justification provided) |
[production] |
| 22:11 |
<bsitzmann@tin> |
Finished deploy [mobileapps/deploy@507a479]: Update mobileapps to 2cb6281 (T168848 T169277 T169274 T162179 T164033 T167921 T174698 T168848 T174808) (duration: 04m 53s) |
[production] |
| 22:06 |
<bsitzmann@tin> |
Started deploy [mobileapps/deploy@507a479]: Update mobileapps to 2cb6281 (T168848 T169277 T169274 T162179 T164033 T167921 T174698 T168848 T174808) |
[production] |
| 21:39 |
<niharika29@tin> |
Synchronized wmf-config/CommonSettings-labs.php: Config for ArticleCreationWorkflow T175054 (duration: 00m 50s) |
[production] |
| 21:28 |
<ejegg> |
updated payments-wiki from bf97cd236bdec421082db8a8a782965de9c98358 to ed2e4811067ad3d74433e8d7f4944ae838af6e28 |
[production] |
| 21:24 |
<ejegg> |
updated CiviCRM from f9c373ec42f0a7b078a3eb7b31385183dab79bdf to 764bfe1f92cc8bbf8d7536744fb47b34ca7a4365 |
[production] |
| 21:17 |
<ottomata> |
rebooting kafka-jumbo1004 |
[production] |
| 20:34 |
<arlolra> |
Updated Parsoid to f9d367ea (T169342) |
[production] |
| 20:27 |
<arlolra@tin> |
Finished deploy [parsoid/deploy@f07ac8c]: Updating Parsoid to f9d367ea (duration: 08m 27s) |
[production] |
| 20:18 |
<arlolra@tin> |
Started deploy [parsoid/deploy@f07ac8c]: Updating Parsoid to f9d367ea |
[production] |
| 19:53 |
<ottomata> |
reimaging kafka-jumbo* with stretch |
[production] |
| 19:48 |
<chasemp> |
reboot labvirt1018 |
[production] |
| 18:50 |
<maxsem@tin> |
Synchronized wmf-config/: just to make sure... (duration: 00m 50s) |
[production] |
| 18:48 |
<maxsem@tin> |
Synchronized wmf-config/CommonSettings.php: revert (duration: 00m 49s) |
[production] |
| 18:33 |
<maxsem@tin> |
Synchronized wmf-config/CommonSettings.php: https://gerrit.wikimedia.org/r/#/c/370292/3 (duration: 00m 49s) |
[production] |
| 18:25 |
<maxsem@tin> |
Synchronized wmf-config/CommonSettings-labs.php: https://gerrit.wikimedia.org/r/#/c/370291/2 (duration: 00m 49s) |
[production] |
| 18:19 |
<maxsem@tin> |
Synchronized wmf-config/Wikibase.php: https://gerrit.wikimedia.org/r/#/c/376317/2 (duration: 00m 48s) |
[production] |
| 18:18 |
<maxsem@tin> |
Synchronized wmf-config/Wikibase-production.php: https://gerrit.wikimedia.org/r/#/c/376317/2 (duration: 00m 48s) |
[production] |
| 18:09 |
<maxsem@tin> |
Synchronized portals: (no justification provided) (duration: 00m 49s) |
[production] |
| 18:09 |
<ejegg> |
rolled payments-wiki back to bf97cd236bdec421082db8a8a782965de9c98358 |
[production] |
| 18:08 |
<maxsem@tin> |
Synchronized portals/prod/wikipedia.org/assets: (no justification provided) (duration: 00m 50s) |
[production] |
| 18:05 |
<ejegg> |
updated payments-wiki from bf97cd236bdec421082db8a8a782965de9c98358 to 691115837b3bbd07a53248ca46a7bd80d2a3970e |
[production] |
| 17:53 |
<bblack> |
cp1048 - varnish backend restart for mailbox lag... |
[production] |
| 17:23 |
<demon@tin> |
Synchronized wmf-config/FeaturedFeedsWMF.php: code cleanup (duration: 00m 49s) |
[production] |
| 17:20 |
<mutante> |
added LDAP user schoenbaechler to WMF group (T175156) |
[production] |
| 17:16 |
<mutante> |
modify-ldap-group on terbium is broken |
[production] |
| 17:02 |
<ejegg> |
disabled creating CiviMail records for Thank You emails |
[production] |
| 16:51 |
<demon@tin> |
Synchronized php-1.30.0-wmf.17/extensions/Flow/includes/: I284b5aa (duration: 01m 01s) |
[production] |
| 16:41 |
<chasemp> |
disable puppet for cloud things to test changes |
[production] |
| 16:40 |
<bblack> |
cp1072 - varnish backend restart (mailbox lag) |
[production] |
| 16:40 |
<bblack> |
cp1062 - varnish backend restart (mailbox lag) |
[production] |
| 15:55 |
<ejegg> |
re-enabled IPN notification at PayPal console (48 minutes ago) |
[production] |
| 15:53 |
<akosiaris> |
powercycling lvs3001 |
[production] |
| 15:28 |
<papaul> |
firmware upgrade on mw2256 |
[production] |
| 15:11 |
<bblack> |
cp1049 - restart varnish backend (mailbox lag) |
[production] |
| 15:07 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Depool es1019 (duration: 00m 49s) |
[production] |
| 15:06 |
<jynus@tin> |
Synchronized wmf-config/db-codfw.php: Repool db2040 (duration: 00m 49s) |
[production] |
| 14:17 |
<gehel> |
wdqs1005 is coming up after a few reimaging issues, expect some icinga noise... |
[production] |
| 13:19 |
<jynus> |
restarting and upgrading db2040 |
[production] |
| 13:13 |
<reedy@tin> |
Synchronized wmf-config/Wikibase-production.php: Move some wikidata config T174962 (duration: 00m 48s) |
[production] |
| 13:12 |
<reedy@tin> |
Synchronized wmf-config/Wikibase-labs.php: Move some wikidata config T174962 (duration: 00m 49s) |
[production] |
| 13:07 |
<reedy@tin> |
Synchronized wmf-config/throttle.php: Throttle exception T175113 (duration: 00m 49s) |
[production] |
| 12:08 |
<moritzm> |
installing libapache2-mod-perl update from jessie 8.9 update |
[production] |
| 11:41 |
<moritzm> |
installing gtk+2.0 update from jessie 8.9 update |
[production] |
| 11:24 |
<elukey> |
temporarily raise kafka log4j authorizer verbosity to DEBUG on kafka1012 - T173493 |
[production] |