|
2016-02-03
§
|
| 15:32 |
<jynus> |
restart and reconfigure mysql in db1060 |
[production] |
| 15:30 |
<hashar> |
MediaWiki 1.27.0-wmf.12, from 1.27.0-wmf.12, successfully checked out. |
[production] |
| 15:23 |
<jynus@mira> |
Synchronized wmf-config/db-eqiad.php: Depool db1060 (duration: 00m 43s) |
[production] |
| 15:21 |
<hashar> |
mira: cloning 1.27.0-wmf.12 (no link updates) |
[production] |
| 15:15 |
<bblack> |
rebooting cp1060 (depooled/downtimed) |
[production] |
| 15:12 |
<hashar> |
apt-get upgrade deployment-sentry2 |
[releng] |
| 15:11 |
<bblack> |
depooling cp1060 temporarily from cache_mobile varnish backends |
[production] |
| 15:03 |
<hashar> |
redeployed rcstream/rcstream on deployment-stream by using git-deploy on deployment-bastion |
[releng] |
| 14:55 |
<jynus@mira> |
Synchronized wmf-config/db-eqiad.php: Repool db1054 with low weight, repool db1067 with original weight (duration: 01m 22s) |
[production] |
| 14:55 |
<hashar> |
upgrading deployment-stream |
[releng] |
| 14:50 |
<bblack> |
rebooting cp1008 for kernel |
[production] |
| 14:42 |
<hashar> |
pooled back integration-slave-trusty-1015 Seems ok |
[releng] |
| 14:35 |
<hashar> |
manually triggered a bunch of browser tests jobs |
[releng] |
| 14:28 |
<godog> |
investigating uwsgi processes for graphite-web not coming up after reboot |
[production] |
| 14:10 |
<moritzm> |
rebooting graphite1001 for kernel update |
[production] |
| 13:41 |
<godog> |
powercycle ms-be2015 |
[production] |
| 13:39 |
<jynus> |
restarting and reconfiguring mysql at db1054 |
[production] |
| 13:27 |
<jynus@mira> |
Synchronized wmf-config/db-eqiad.php: Repool db1067 at low weight; depool db1054 (duration: 01m 16s) |
[production] |
| 11:45 |
<jynus> |
restarting and reconfiguring mysql at db1067 |
[production] |
| 11:40 |
<hashar> |
apt-get upgrade deployment-ms-be01 and deployment-ms-be02 |
[releng] |
| 11:32 |
<hashar> |
fixing puppet.conf on deployment-memc04 |
[releng] |
| 11:11 |
<moritzm> |
repooling restbase1001 |
[production] |
| 11:08 |
<hashar> |
restarting beta cluster puppetmaster just in case |
[releng] |
| 11:07 |
<hashar> |
beta: apt-get upgrade on delpoyment-cache* hosts and checking puppet |
[releng] |
| 11:04 |
<akosiaris> |
OTRS database upgraded to 3.3, moving on with 4.0 |
[production] |
| 11:00 |
<jynus@mira> |
Synchronized wmf-config/db-eqiad.php: Repool db1063 at 100% load; depool db1067 for maintenance (duration: 01m 16s) |
[production] |
| 10:59 |
<hashar> |
integration/beta: deleting /etc/apt/apt.conf.d/*proxy files. There is no need for them, in fact web proxy is not reachable from labs |
[releng] |
| 10:53 |
<hashar> |
integration: switched puppet repo back to 'production' branch, rebased. |
[releng] |
| 10:49 |
<hashar> |
various beta cluster have puppet errors .. |
[releng] |
| 10:48 |
<moritzm> |
depooling restbase1001 for kernel/Java update |
[production] |
| 10:46 |
<hashar> |
integration-slave-trusty-1013 heading to out of disk space on /mnt ... |
[releng] |
| 10:42 |
<hashar> |
integration-slave-trusty-1016 out of disk space on /mnt ... |
[releng] |
| 10:37 |
<_joe_> |
ending the load test on the eqiad apaches |
[production] |
| 10:11 |
<moritzm> |
reboot francium for kernel update |
[production] |
| 09:53 |
<jynus> |
m2 backup finished on /srv/backups/2016-02-03_08-51-06, filename 'db1020-bin.000842', position 220103947 |
[production] |
| 09:50 |
<moritzm> |
restarting neodymium for kernel update |
[production] |
| 09:49 |
<_joe_> |
doing some basic load test on appservers in eqiad |
[production] |
| 08:52 |
<akosiaris> |
stop otrs-daemon on mendelevium |
[production] |
| 08:51 |
<jynus> |
starting mysql backup on db1020 (/srv/backups) |
[production] |
| 08:44 |
<akosiaris> |
stop slave on db2011, db1020's (m2-master) slave, for OTRS migration. DO NOT ENABLE |
[production] |
| 08:40 |
<akosiaris> |
stop exim4, cron, apache2 on iodine, mendelevium |
[production] |
| 08:39 |
<akosiaris> |
disabling puppet on iodine, mendelevium, OTRS migration |
[production] |
| 08:24 |
<jynus@mira> |
Synchronized wmf-config/db-eqiad.php: Repool db1063 with low weight (duration: 01m 20s) |
[production] |
| 03:45 |
<bd808> |
Puppet failing on deployment-fluorine with "Error: Could not set uid on user[datasets]: Execution of '/usr/sbin/usermod -u 10003 datasets' returned 4: usermod: UID '10003' already exists" |
[releng] |
| 03:44 |
<bd808> |
Freed 28G by deleting deployment-fluorine:/srv/mw-log/archive/*2015* |
[releng] |
| 03:41 |
<bd808> |
Ran deployment-bastion.deployment-prep:/home/bd808/cleanup-var-crap.sh and freed 565M |
[releng] |
| 03:00 |
<YuviPanda> |
upgraded flannel on all hosts running it |
[tools] |