|
2017-06-21
ยง
|
| 14:25 |
<andrewbogott> |
rebooting labvirt1003 |
[production] |
| 14:21 |
<andrewbogott> |
rebooting labvirt1002 |
[production] |
| 14:21 |
<hashar> |
deployment-prep: force running puppet on all instances |
[releng] |
| 14:20 |
<andrewbogott> |
depooling tools-exec-1405, tools-exec-1425, tools-webgrid-generic-1403, tools-webgrid-lighttpd-1401, tools-webgrid-lighttpd-1405 |
[tools] |
| 14:19 |
<andrewbogott> |
repooling tools-exec-1401, 1407, 1408, 1430, 1431, 1432, 1435, 1438, 1439, 1440, 1441, tools-webgrid-lighttpd-1402, tools-webgrid-lighttpd-1407 |
[tools] |
| 14:18 |
<herron> |
rebooting mx1001 for kernel upgrade |
[production] |
| 14:17 |
<hashar> |
finally fixed puppet on deployment-prep ! |
[releng] |
| 14:08 |
<ema> |
reboot lvs[3003-3004] (esams secondaries) for kernel update |
[production] |
| 14:03 |
<elukey> |
reboot eventlog2001 for kernel update |
[production] |
| 14:02 |
<hashar> |
deployment-puppmaster (cd /etc/puppet && ln -s /var/lib/git/operations/puppet/manifests && ln -s /var/lib/git/operations/puppet/modules) |
[releng] |
| 14:02 |
<andrewbogott> |
rebooting labvirt1001 |
[production] |
| 14:01 |
<gehel> |
restarting wdqs1001 for kernel upgrade |
[production] |
| 14:01 |
<godog> |
reimage ms-be1020 / ms-be1021 with stretch |
[production] |
| 13:59 |
<elukey> |
eventlogging restarted after reboot |
[analytics] |
| 13:56 |
<andrewbogott> |
depooling tools-exec-1401, 1407, 1408, 1430, 1431, 1432, 1435, 1438, 1439, 1440, 1441, tools-webgrid-lighttpd-1402, tools-webgrid-lighttpd-1407 |
[tools] |
| 13:54 |
<elukey> |
stop eventlogging and reboot eventlog1001 |
[analytics] |
| 13:52 |
<gehel> |
install analysis-kuromoji plugin on relforge |
[production] |
| 13:52 |
<herron> |
install exim security updates on fermium (lists) |
[production] |
| 13:51 |
<elukey> |
rebooting eventlog1001 for kernel update (eventlogging host) |
[production] |
| 13:50 |
<moritzm> |
pruning old kernels on prometheus* |
[production] |
| 13:48 |
<addshore@tin> |
Synchronized php-1.30.0-wmf.6/extensions/RevisionSlider/modules/ext.RevisionSlider.SliderView.js: SWAT: [[gerrit:360648|Fix errors leading to wrong slider scroll postions]] T168299 (duration: 00m 44s) |
[production] |
| 13:47 |
<addshore@tin> |
Synchronized php-1.30.0-wmf.5/extensions/RevisionSlider/modules/ext.RevisionSlider.SliderView.js: SWAT: [[gerrit:360650|Fix errors leading to wrong slider scroll postions]] T168299 (duration: 00m 46s) |
[production] |
| 13:44 |
<elukey> |
reboot aqs100[89] for kernel updates |
[production] |
| 13:39 |
<ema> |
reboot lvs[2001-2003] (codfw primaries) for kernel update |
[production] |
| 13:29 |
<elukey> |
reboot aqs1007 for kernel update |
[production] |
| 13:26 |
<hashar> |
deployment-prep: puppet master got erroneously upgrade to puppet* 4.8. Roll it back to 3.8 which fail, and then back to 3.7! |
[releng] |
| 13:22 |
<marostegui> |
Deploy alter table on s7 - directly on codfw master (db2029) - this will generate lag on codfw - T166208 |
[production] |
| 13:21 |
<elukey> |
reboot kafka1013 for kernel updates |
[production] |
| 13:16 |
<marostegui> |
Deploy alter table s5 - labsdb1001 - T166207 |
[production] |
| 13:15 |
<elukey> |
reboot analytics1003 for kernel update |
[analytics] |
| 13:15 |
<marostegui> |
Deploy alter table s5 - db1045 - T166207 |
[production] |
| 13:14 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Depool db1045 - T166207 (duration: 00m 44s) |
[production] |
| 13:08 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1070 - T166207 (duration: 00m 46s) |
[production] |
| 13:05 |
<elukey> |
reboot analytics1003 (Hue, Camus, Oozie, Hive master) for kernel upgrade |
[production] |
| 12:47 |
<hashar> |
broke deployment-prep puppet master while upgrading it :( |
[releng] |
| 12:32 |
<gehel> |
deploying T167871 and restarting kartotherian / tilerator on maps eqiad |
[production] |
| 12:32 |
<moritzm> |
rebooting mw1189-mw1199 for kernel update |
[production] |
| 12:28 |
<hashar> |
deployment-imagescaler01 removed puppetmaster and puppetmaster-common packages |
[releng] |
| 12:10 |
<akosiaris@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=sca1004.eqiad.wmnet |
[production] |
| 12:09 |
<akosiaris@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=mwdebug1002.eqiad.wmnet |
[production] |
| 12:04 |
<hashar> |
apt-get dist-upgrade on deployment-mediawiki hosts |
[releng] |
| 11:59 |
<moritzm> |
rebooting mw1209-mw1220 for kernel update |
[production] |
| 11:59 |
<hashar> |
armed keyholder on deployment-tin and deployment-mira |
[releng] |
| 11:45 |
<moritzm> |
rebooting mediawiki api servers in codfw for kernel update |
[production] |
| 11:42 |
<akosiaris> |
rollback change in asw-a-eqiad for ganeti interface range due to alerts |
[production] |
| 11:23 |
<akosiaris> |
reboot ganeti1007 for insertion into ganeti cluster |
[production] |
| 11:15 |
<hashar> |
deployment-cache-text04 : apt-get dist-upgrade |
[releng] |
| 11:14 |
<elukey> |
reboot aqs1006 for kernel update |
[production] |
| 11:12 |
<hashar> |
varnish fails on deployment-cache-text04 |
[releng] |
| 11:08 |
<hashar> |
deployment-prep : rebooting deployment-tin deployment-mira deployment-cache-text04 deployment-cache-upload04 |
[releng] |