|
2018-05-15
§
|
| 14:00 |
<andrewbogott> |
rebooting labnet1001 |
[production] |
| 13:50 |
<elukey> |
roll restart of kafka main codfw (kafka200[1-3]) to pick up group.initial.rebalance.delay.ms = 10s |
[production] |
| 13:31 |
<jynus> |
stop db2055 for reimage |
[production] |
| 13:09 |
<chasemp> |
disable puppet for all openstack things in eqiad |
[production] |
| 13:07 |
<andrewbogott> |
stopping nodepool and puppet on labnodepool1001 for T193579 |
[production] |
| 12:59 |
<andrewbogott> |
stopping puppet on labnet1001 and 1002, silencing icinga for T193579 |
[production] |
| 12:42 |
<jynus> |
stop db2060 for reimage |
[production] |
| 12:14 |
<moritzm> |
uploaded intel-microcode 20180425 for jessie-wikimedia/stretch-wikimedia |
[production] |
| 10:57 |
<jynus> |
stop db2067 for reimage |
[production] |
| 10:49 |
<joal@tin> |
Finished deploy [analytics/refinery@25abeec]: Fix for regular weekly deploy (duration: 06m 45s) |
[production] |
| 10:46 |
<jynus> |
stop db2066 for reimage |
[production] |
| 10:43 |
<joal@tin> |
Started deploy [analytics/refinery@25abeec]: Fix for regular weekly deploy |
[production] |
| 10:38 |
<joal> |
Kill-Restart mediawiki-history-reduced ooie coordinator to pick up deployed changes |
[analytics] |
| 10:16 |
<jynus> |
stop db2065 for reimage |
[production] |
| 10:15 |
<moritzm> |
installing uwsgi security update on graphite servers in eqiad |
[production] |
| 10:07 |
<moritzm> |
installing php5 security updates on trusty |
[production] |
| 09:51 |
<moritzm> |
upgrading API server canaries to HHVM 3,18.5+dfsg-1+wmf8+deb9u1 |
[production] |
| 09:47 |
<jynus> |
stop and restart db2091 for upgrade |
[production] |
| 09:37 |
<joal> |
Deploy refinery onto HDFS |
[analytics] |
| 09:36 |
<joal> |
Deployed refinery using scap |
[analytics] |
| 09:36 |
<joal@tin> |
Finished deploy [analytics/refinery@b2f4c3c]: Regular weekly deploy (duration: 05m 38s) |
[production] |
| 09:30 |
<joal@tin> |
Started deploy [analytics/refinery@b2f4c3c]: Regular weekly deploy |
[production] |
| 09:26 |
<jynus> |
stop and restart db2088 for upgrade |
[production] |
| 09:21 |
<moritzm> |
upgrading app server canaries to HHVM 3,18.5+dfsg-1+wmf8+deb9u1 |
[production] |
| 09:03 |
<jynus> |
stop db2061 for reimage |
[production] |
| 08:42 |
<jynus> |
stop db2068 for reimage |
[production] |
| 07:26 |
<zhuyifei1999_> |
applied 5324236 via toolsbeta-puppetmaster-01 T190893 |
[toolsbeta] |
| 05:29 |
<legoktm> |
deployed https://gerrit.wikimedia.org/r/432947 https://gerrit.wikimedia.org/r/433017 https://gerrit.wikimedia.org/r/433020 |
[releng] |
| 05:28 |
<zhuyifei1999_> |
Making project puppetmaster at toolsbeta-puppetmaster-01 |
[toolsbeta] |
| 04:28 |
<andrewbogott> |
depooling, rebooting, re-pooling tools-exec-1414. It's hanging for unknown reasons. |
[tools] |
| 04:07 |
<zhuyifei1999_> |
Draining unresponsive tools-exec-1414 following Portal:Toolforge/Admin#Draining_a_node_of_Jobs |
[tools] |
| 04:05 |
<zhuyifei1999_> |
Force deletion of grid job 5221417 (tools.giftbot sga), host tools-exec-1414 not responding |
[tools] |
| 03:10 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Tue May 15 03:10:59 UTC 2018 (duration 7m 11s) |
[production] |
| 03:03 |
<l10nupdate@tin> |
scap sync-l10n completed (1.32.0-wmf.3) (duration: 06m 32s) |
[production] |
| 02:33 |
<bawolff@tin> |
Finished scap: Backport https://gerrit.wikimedia.org/r/#/c/433096/ - log js loads of unregistered user js subpages (duration: 56m 27s) |
[production] |
| 01:37 |
<bawolff@tin> |
Started scap: Backport https://gerrit.wikimedia.org/r/#/c/433096/ - log js loads of unregistered user js subpages |
[production] |
| 01:17 |
<bawolff@tin> |
Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/433095/ log security channel (duration: 01m 02s) |
[production] |
|
2018-05-14
§
|
| 23:54 |
<ottomata> |
bouncing main -> jumbo MirrorMaker with larger max.request.size |
[analytics] |
| 23:39 |
<bawolff@tin> |
Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/433089/ - re-enable sending csp logs to logstash (duration: 01m 01s) |
[production] |
| 23:36 |
<bawolff@tin> |
Synchronized wmf-config/CommonSettings.php: https://gerrit.wikimedia.org/r/#/c/432334/ Use english messages in badpass log (duration: 01m 16s) |
[production] |
| 22:39 |
<ottomata> |
bouncing main-eqiad -> jumbo mirror maker after committing new offset for eqiad.mediawiki.job.RecordLintJob |
[analytics] |
| 22:24 |
<eileen> |
update process-control to 97fd42b3ab disabling dedupe catch up |
[production] |
| 21:54 |
<mutante> |
mwdebug1001 - temp modifying apache 08-wikimedia.conf to test gerrit:429863 |
[production] |
| 21:03 |
<arlolra> |
Updated Parsoid to 945ed23 (T194082, T194083, T194084) |
[production] |
| 20:54 |
<arlolra@tin> |
Finished deploy [parsoid/deploy@28fcc4e]: Updating Parsoid to 945ed23 (duration: 11m 29s) |
[production] |
| 20:42 |
<arlolra@tin> |
Started deploy [parsoid/deploy@28fcc4e]: Updating Parsoid to 945ed23 |
[production] |
| 20:09 |
<bsitzmann@tin> |
Finished deploy [mobileapps/deploy@ccffa6b]: Update mobileapps to 39c16e4 (T193440 T193439 T194065) (duration: 07m 49s) |
[production] |
| 20:01 |
<bsitzmann@tin> |
Started deploy [mobileapps/deploy@ccffa6b]: Update mobileapps to 39c16e4 (T193440 T193439 T194065) |
[production] |
| 19:41 |
<bearND> |
BC: Update mobileapps to 39c16e4 |
[releng] |
| 19:11 |
<urandom> |
rolling cassandra restart, restbase dev environment |
[production] |