|
2017-06-21
ยง
|
| 15:45 |
<moritzm> |
upgrade ms-be2013/ms-be2014 to final stretch release and reboot for kernel update |
[production] |
| 15:41 |
<andrewbogott> |
switching the proxy ip back to tools-proxy-02 |
[tools] |
| 15:35 |
<hashar> |
deployment-prep changing Varnish director for citoid from citoid.wmflabs.org to citoid-beta.wmflabs.org ( via https://horizon.wikimedia.org/project/prefixpuppet/ ) - T168519 |
[releng] |
| 15:34 |
<ema> |
reboot lvs[4003-4004] (ulsfo secondaries) for kernel update |
[production] |
| 15:32 |
<moritzm> |
reboot image scalers in codfw for kernel update |
[production] |
| 15:32 |
<andrewbogott> |
rebooting labvirt1007 |
[production] |
| 15:31 |
<andrewbogott> |
temporarily pointing the tools-proxy IP to tools-proxy-01 |
[tools] |
| 15:26 |
<andrewbogott> |
depooling tools-exec-1422, tools-webgrid-lighttpd-1413 |
[tools] |
| 15:13 |
<andrewbogott> |
rebooting labvirt1006 |
[production] |
| 15:12 |
<andrewbogott> |
depooling tools-exec-1404, tools-exec-1434, tools-worker-1026 |
[tools] |
| 15:10 |
<andrewbogott> |
repooling tools-exec-1402, 1426, 1429, 1433, tools-webgrid-lighttpd-1408, 1414, 1424 |
[tools] |
| 15:04 |
<moritzm> |
rebooting ruthenium for kernel update |
[production] |
| 15:01 |
<moritzm> |
reboot job runners in codfw for kernel update |
[production] |
| 15:01 |
<elukey> |
reboot kafka200[23] for kernel updates (eventbus codfw) |
[production] |
| 14:53 |
<andrewbogott> |
rebooting labvirt1005 |
[production] |
| 14:53 |
<andrewbogott> |
depooling tools-exec-1402, 1426, 1429, 1433, tools-webgrid-lighttpd-1408, 1414, 1424 |
[tools] |
| 14:52 |
<andrewbogott> |
repooling tools-exec-1403, tools-exec-gift-trusty-01, tools-webgrid-generic-1402, tools-webgrid-lighttpd-1403 |
[tools] |
| 14:41 |
<hashar> |
deployment-tmh01 is down for some reason |
[releng] |
| 14:40 |
<moritzm> |
reboot remaining scb* hosts for kernel update |
[production] |
| 14:38 |
<andrewbogott> |
rebooting labvirt1004 |
[production] |
| 14:37 |
<andrewbogott> |
depooling tools-exec-1403, tools-exec-gift-trusty-01, tools-webgrid-generic-1402, tools-webgrid-lighttpd-1403 |
[tools] |
| 14:32 |
<ema> |
reboot lvs[3001-3002] (esams primaries) for kernel update |
[production] |
| 14:32 |
<andrewbogott> |
repooling tools-exec-1405, tools-exec-1425, tools-webgrid-generic-1403, tools-webgrid-lighttpd-1401, tools-webgrid-lighttpd-1405 |
[tools] |
| 14:25 |
<andrewbogott> |
rebooting labvirt1003 |
[production] |
| 14:21 |
<andrewbogott> |
rebooting labvirt1002 |
[production] |
| 14:21 |
<hashar> |
deployment-prep: force running puppet on all instances |
[releng] |
| 14:20 |
<andrewbogott> |
depooling tools-exec-1405, tools-exec-1425, tools-webgrid-generic-1403, tools-webgrid-lighttpd-1401, tools-webgrid-lighttpd-1405 |
[tools] |
| 14:19 |
<andrewbogott> |
repooling tools-exec-1401, 1407, 1408, 1430, 1431, 1432, 1435, 1438, 1439, 1440, 1441, tools-webgrid-lighttpd-1402, tools-webgrid-lighttpd-1407 |
[tools] |
| 14:18 |
<herron> |
rebooting mx1001 for kernel upgrade |
[production] |
| 14:17 |
<hashar> |
finally fixed puppet on deployment-prep ! |
[releng] |
| 14:08 |
<ema> |
reboot lvs[3003-3004] (esams secondaries) for kernel update |
[production] |
| 14:03 |
<elukey> |
reboot eventlog2001 for kernel update |
[production] |
| 14:02 |
<hashar> |
deployment-puppmaster (cd /etc/puppet && ln -s /var/lib/git/operations/puppet/manifests && ln -s /var/lib/git/operations/puppet/modules) |
[releng] |
| 14:02 |
<andrewbogott> |
rebooting labvirt1001 |
[production] |
| 14:01 |
<gehel> |
restarting wdqs1001 for kernel upgrade |
[production] |
| 14:01 |
<godog> |
reimage ms-be1020 / ms-be1021 with stretch |
[production] |
| 13:59 |
<elukey> |
eventlogging restarted after reboot |
[analytics] |
| 13:56 |
<andrewbogott> |
depooling tools-exec-1401, 1407, 1408, 1430, 1431, 1432, 1435, 1438, 1439, 1440, 1441, tools-webgrid-lighttpd-1402, tools-webgrid-lighttpd-1407 |
[tools] |
| 13:54 |
<elukey> |
stop eventlogging and reboot eventlog1001 |
[analytics] |
| 13:52 |
<gehel> |
install analysis-kuromoji plugin on relforge |
[production] |
| 13:52 |
<herron> |
install exim security updates on fermium (lists) |
[production] |
| 13:51 |
<elukey> |
rebooting eventlog1001 for kernel update (eventlogging host) |
[production] |
| 13:50 |
<moritzm> |
pruning old kernels on prometheus* |
[production] |
| 13:48 |
<addshore@tin> |
Synchronized php-1.30.0-wmf.6/extensions/RevisionSlider/modules/ext.RevisionSlider.SliderView.js: SWAT: [[gerrit:360648|Fix errors leading to wrong slider scroll postions]] T168299 (duration: 00m 44s) |
[production] |
| 13:47 |
<addshore@tin> |
Synchronized php-1.30.0-wmf.5/extensions/RevisionSlider/modules/ext.RevisionSlider.SliderView.js: SWAT: [[gerrit:360650|Fix errors leading to wrong slider scroll postions]] T168299 (duration: 00m 46s) |
[production] |
| 13:44 |
<elukey> |
reboot aqs100[89] for kernel updates |
[production] |
| 13:39 |
<ema> |
reboot lvs[2001-2003] (codfw primaries) for kernel update |
[production] |
| 13:29 |
<elukey> |
reboot aqs1007 for kernel update |
[production] |
| 13:26 |
<hashar> |
deployment-prep: puppet master got erroneously upgrade to puppet* 4.8. Roll it back to 3.8 which fail, and then back to 3.7! |
[releng] |
| 13:22 |
<marostegui> |
Deploy alter table on s7 - directly on codfw master (db2029) - this will generate lag on codfw - T166208 |
[production] |