1001-1050 of 2928 results (9ms)
2014-10-21 §
21:10 <hashar> contint: refreshed slave-scripts 0b85d48..8c3f228 sqlite files will be cleared out after 20 minutes (instead of 60 minutes) {{bug|71128}} [releng]
20:51 <cscott> deployment-prep _joe_ promises to fix this properly tomorrow am [releng]
20:51 <cscott> deployment-prep turned off puppet on deployment-pdf01, manually fixed broken /etc/ocg/mw-ocg-service.js [releng]
20:50 <cscott> deployment-prep updated OCG to version 523c8123cd826c75240837c42aff6301032d8ff1 [releng]
10:55 <hashar> deleted salt master key on deployment-elastic{06,07}, restarted salt-minion and reran puppet. It is now passing on both instances \\O/ [releng]
10:48 <hashar> rerunning puppet manually on deployment-elastic{06,07} [releng]
10:48 <hashar> beta: signing puppet cert for deployment-elastic{06,07}. On deployment-salt ran: puppet ca sign i-000006b6.eqiad.wmflabs; puppet ca sign i-000006b7.eqiad.wmflabs [releng]
09:29 <hashar> forget me deployment-logstash1 has a puppet agent error but it is simply because the agent is disabled "'debugging logstash config'" [releng]
09:28 <hashar> deployment-logstash1 disk full [releng]
2014-10-20 §
17:41 <bd808> Disabled redis input plugin and restarted logstash on deployment-logstash1 [releng]
17:39 <bd808> Disabled puppet on deployment-logstash1 for some live hacking of logstash config [releng]
15:27 <apergos> upgrded salt-master on virt1000 (master for labs) [releng]
2014-10-17 §
22:34 <subbu> live fixed bad logger config in /srv/deployment/parsoid/deploy/conf/wmf/betalabs.localsettings.js and verified that parsoid doesn't crash anymore -- fix now on gerrit and being merged [releng]
20:48 <hashar> qa-morebots is back [releng]
2014-10-15 §
01:08 <Krinkle> Pooled integration-slave1009 [releng]
01:08 <Krinkle> Setting up integration-slave1009 ({{bug|72014}} fixed}}) [releng]
01:00 <Krinkle> integration-publisher and integration-zuul-server were rebooted by me yesterday. Seems they only show up in graphite now. Maybe they were shutdown or had puppet stuck. [releng]
2014-10-14 §
21:00 <JohnLewis> icinga says deployment-sca01 is good (yay) [releng]
20:42 <JohnLewis> deleted and recreated deployment-sca01 (still needs puppet set up) [releng]
20:24 <JohnLewis> rebooted deployment-sca01 [releng]
09:26 <hashar> renamed deployment-cxserver02 node slaves to 03 and updated the ip address [releng]
06:49 <Krinkle> Did a slow-rotating graceful depool/reboot/repool of all integration-slave's over the past hour to debug problems whilst waiting for puppet to unblock and set up new slaves. [releng]
06:43 <Krinkle> Keeping the new integration-slave1009 unpooled because setup could not be completed due to {{bug|72014}}. [releng]
06:43 <Krinkle> Pooled integration-slave1004 [releng]
05:40 <Krinkle> Setting up integration-slave1004 and integration-slave1009 ({{bug|71873}} fixed) [releng]
2014-10-10 §
20:53 <Krinkle> Deleted integration-slave1004 and integration-slave1009. When {{bug|71873}} is fixed, they'll need to be re-created. [releng]
19:11 <Krinkle> integration-slave1004 (new instance, not set up yet) was broken ({{bug|71741}}). The bug seems fixed for new instances so, I deleted and re-created it. Will be setting up as a Precise instance and pool it. [releng]
19:09 <Krinkle> integration-slave1009 (new instance) remains unpooled as it is not yet fully set up ({{bug|71874}}). See [[Nova_Resource:Integration/Setup]] [releng]
2014-10-09 §
20:17 <bd808> rebooted deployment-sca01 via wikitech ui [releng]
20:16 <bd808> deployment-sca01 dead -- Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000100 [releng]
19:44 <bd808> added role::deployment::test to deployment-rsync01 and deployment-mediawiki03 for trebuchet testing [releng]
19:07 <bd808> updated scap to include 8183d94 (Fix "TypeError bufsize must be an integer") [releng]
09:35 <hashar> migrating deployment-cxserver02 to beta cluster puppet and salt masters [releng]
09:22 <hashar> Renamed Jenkins slave deployment-cxserver01 to deployment-cxserver02 and updated IP. It is marked offline until the instance is ready and has the relevant puppet classes applied. [releng]
09:19 <hashar> deleting deployment-cxserver01 (borked since virt1005 outage) creating deployment-cxserver02 to replace it {{bug|71783}} [releng]
2014-10-08 §
22:04 <subbu> updated OCG to version def24eca [releng]
2014-10-07 §
22:50 <cscott> updated OCG to version c778ea8b898f8ad8c2b7ad9de78a75469e7ed061 [releng]
19:19 <bd808> ^d deleted all files/directories in gallium:/var/lib/jenkins-slave/tmpfs [releng]
18:24 <bd808> /var/lib/jenkins-slave/tmpfs full (100%) on gallium [releng]
11:54 <Krinkle> The new integration-slave1009 must remain unpooled because Setup failed (puppet unable to mount /mnt) - see also [[Nova Resource:Integration/Setup]] [releng]
11:53 <Krinkle> Deleted integration-slave1004 because {{bug|71741}} [releng]
10:16 <hashar> beta: apt-get upgraded all instances beside the lucid one. [releng]
09:57 <hashar> beta: deleting old occurrences of /etc/apt/preferences.d/puppet_base_2.7 [releng]
09:53 <hashar> apt-get upgrade on all beta cluster instances [releng]
09:34 <Krinkle> Rebase integration-puppetmaster on latest operations-puppet (patches: I7163fd38bcd082a1, If2e96bfa9a1c46) [releng]
09:32 <Krinkle> Apply I44d33af1ce85 instead of Ib95c292190d on integration-puppetmaster (remove php5-parsekit package) [releng]
09:28 <hashar> upgrading php5-fss on both beta-cluster and integration instances. {{bug|66092}} https://rt.wikimedia.org/Ticket/Display.html?id=7213 [releng]
2014-10-06 §
23:13 <YuviPanda> killed extra log files in deployment-bastion [releng]
21:44 <cscott> updated OCG to version bbdf4c6400cfbbc6030114ad16e1a6f7025eab2c [releng]
15:36 <cscott> updated OCG to version aee3712b352f51f96569de0bcccf3facf654e688 [releng]