|
2014-10-07
§
|
| 11:54 |
<Krinkle> |
The new integration-slave1009 must remain unpooled because Setup failed (puppet unable to mount /mnt) - see also [[Nova Resource:Integration/Setup]] |
[releng] |
| 11:53 |
<Krinkle> |
Deleted integration-slave1004 because {{bug|71741}} |
[releng] |
| 10:16 |
<hashar> |
beta: apt-get upgraded all instances beside the lucid one. |
[releng] |
| 09:57 |
<hashar> |
beta: deleting old occurrences of /etc/apt/preferences.d/puppet_base_2.7 |
[releng] |
| 09:53 |
<hashar> |
apt-get upgrade on all beta cluster instances |
[releng] |
| 09:34 |
<Krinkle> |
Rebase integration-puppetmaster on latest operations-puppet (patches: I7163fd38bcd082a1, If2e96bfa9a1c46) |
[releng] |
| 09:32 |
<Krinkle> |
Apply I44d33af1ce85 instead of Ib95c292190d on integration-puppetmaster (remove php5-parsekit package) |
[releng] |
| 09:28 |
<hashar> |
upgrading php5-fss on both beta-cluster and integration instances. {{bug|66092}} https://rt.wikimedia.org/Ticket/Display.html?id=7213 |
[releng] |
|
2014-09-30
§
|
| 23:47 |
<bd808> |
jobrunner using outdated ip address for redis01. Testing patch to use hostname rather than hardcoded ip |
[releng] |
| 22:00 |
<bd808> |
Cleaned deleted instances out of salt and trebuchet redis |
[releng] |
| 21:45 |
<bd808> |
jobrunner not running. ebernhardson is debugging. |
[releng] |
| 21:38 |
<bd808> |
/srv on rsync01 now has 3.2G of free space and should be fine fro quite a while again. |
[releng] |
| 21:15 |
<bd808> |
local l10nupdate users on bastion, mediawiki01 and rsync01 |
[releng] |
| 21:06 |
<bd808> |
Local mwdeploy user on deployment-bastion making things sad |
[releng] |
| 20:36 |
<bd808> |
lots and lots of "file has vanished" errors from rsync. Not sure why |
[releng] |
| 20:35 |
<bd808> |
Initial puppet run with role::beta::rsync_slave applied on rsync02 failed spectacularly in /Stage[main]/Mediawiki::Scap/Exec[fetch_mediawiki] stage |
[releng] |
| 20:26 |
<bd808> |
Converted deployment-rsync02 to use local puppet & salt masters |
[releng] |
| 20:02 |
<bd808> |
Started building deployment-rsync02 to replace deployment-rsync01 |
[releng] |
| 19:59 |
<bd808|LUNCH> |
/srv partition on deployment-rsync01 full again. We need a new rsync server with more space |
[releng] |
| 17:44 |
<bd808> |
Updated scap to 064425b (Remove restart-nutcracker and restart-twemproxy scripts) |
[releng] |
| 16:08 |
<bd808> |
Occasional memecached-serious errors in beta for something trying to connect to the default memcached port (11211) rather than the nutcracker port (11212). |
[releng] |
| 15:58 |
<bd808> |
scap happy again after fixing rogue group/user on rsync01 \\o/ Not sure why they were created but likely an ldap hiccup during a puppet run |
[releng] |
| 15:56 |
<bd808> |
removed local group/user mwdeploy on deployment-rsync01 |
[releng] |
| 15:54 |
<bd808> |
Local mwdeploy (gid=996) shadowing ldap group gid=603(mwdeploy) on deployment-rsync01 |
[releng] |
| 15:49 |
<bd808> |
apt-get dist-upgrade fixed hhvm on deployment-mediawiki03 |
[releng] |
| 15:45 |
<hashar> |
Updating our Jenkins job builder fork 686265a..ee80dbc (no job changed) |
[releng] |
| 15:44 |
<bd808> |
scap failing in beta due to "Permission denied (publickey)" talking to deployment-rsync01.eqiad.wmflabs |
[releng] |
| 15:39 |
<bd808> |
hhvm not starting after puppet run on deployment-mediawiki03. Investigating. |
[releng] |
| 15:36 |
<bd808> |
enabling puppet and forcing run on deployment-mediawiki03 |
[releng] |
| 15:36 |
<bd808> |
enabling puppet and forcing run on deployment-mediawiki03 |
[releng] |
| 15:34 |
<bd808> |
enabling puppet and forcing run on deployment-mediawiki02 |
[releng] |