3351-3400 of 7896 results (18ms)
2016-04-26 §
08:12 <hashar> hard rebooting castor instance | T133652 [releng]
08:10 <hashar> soft rebooting castor instance | T133652 [releng]
08:06 <hashar> CI jobs deadlocked due to castor being unavailable | https://phabricator.wikimedia.org/T133652 [releng]
00:46 <thcipriani> temporary keyholder fix in place in beta [releng]
00:18 <thcipriani> beta-scap-eqiad failure due to bad keyholder-auth.d fingerprints [releng]
2016-04-25 §
20:58 <cscott> updated OCG to version 58a720508deb368abfb7652e6a8c7225f95402d2 [releng]
19:46 <hashar> Nodepool now has a couple trusty instances intended to experiment with Zend 5.5 / HHVM migration . https://phabricator.wikimedia.org/T133203#2236625 [releng]
13:34 <hashar> Nodepool is attempting to create a Trusty snapshot with name ci-trusty-wikimedia-1461591203 | T133203 [releng]
13:15 <hashar> openstack image create --file /home/hashar/image-trusty-20160425T124552Z.qcow2 ci-trusty-wikimedia --disk-format qcow2 --property show=true # T133203 [releng]
10:38 <hashar> Refreshing Nodepool Jessie snapshot based on new image [releng]
10:35 <hashar> Refreshed Nodepool Jessie image ( image-jessie-20160425T100035Z ) [releng]
09:24 <hashar> beta / scap failure filled as T133521 [releng]
09:20 <hashar> Keyholder / mwdeploy ssh keys have been messed up on beta cluster somehow :-( [releng]
08:47 <hashar> mwdeploy@deployment-tin has lost ssh host keys file :( [releng]
2016-04-24 §
17:14 <jzerebecki> reloading e06f1fe..672fc84 [releng]
2016-04-22 §
18:13 <legoktm> deploying https://gerrit.wikimedia.org/r/284841 [releng]
08:13 <legoktm> deploying https://gerrit.wikimedia.org/r/284860 [releng]
2016-04-21 §
19:07 <thcipriani> scap version testing should be done, puppet should no longer be disabled on hosts [releng]
18:02 <thcipriani> disabling puppet on scap targets to test scap_3.1.0-1+0~20160421173204.70~1.gbp6706e0_all.deb [releng]
2016-04-20 §
22:28 <thcipriani> rolling back scap version in beta, legit failure :( [releng]
21:52 <thcipriani> testing new scap version in beta on deployment-tin [releng]
17:54 <thcipriani> Reloading Zuul to deploy [[gerrit:284494]] [releng]
13:58 <hashar> Stopping HHVM on CI slaves by cherry picking a couple puppet patches | T126594 [releng]
13:33 <hashar> salt -v '*trusty*' cmd.run 'rm /usr/lib/x86_64-linux-gnu/hhvm/extensions/current' # Cleanup on CI slaves for T126658 [releng]
13:27 <hashar> Restarted integration puppet master service (out of memory / mem leak) [releng]
2016-04-19 §
14:56 <hashar> integration / beta cluster : fixed salt on bunch of instances and upgraded some packages [releng]
2016-04-18 §
18:53 <bd808> Ran ~bd808/cleanup-var-crap.sh to free a bit of space on deployment-mediawiki01; this instance really needs a bigger disk [releng]
18:47 <bd808> deployment-mediawiki01: rm /var/log/apache2/*.gz [releng]
14:51 <jzerebecki> reloading zuul for d9e80bd..113ea21 [releng]
12:59 <jzerebecki> integration-slave-trusty-1014:~$ sudo -i rm -f /mnt/jenkins-workspace/workspace/mediawiki-extensions-hhvm/src/extensions/GlobalCssJs/.git/refs/heads/wmf/1.27.0-wmf.11.lock [releng]
12:57 <jzerebecki> integration-slave-trusty-1014:~$ sudo -i 'rm -f /mnt/jenkins-workspace/workspace/mediawiki-extensions-hhvm/src/extensions/GlobalCssJs/.git/refs/heads/wmf/1.27.0-wmf.11.lock' [releng]
2016-04-17 §
01:01 <legoktm> deploying https://gerrit.wikimedia.org/r/283837 [releng]
2016-04-16 §
14:20 <Krenair> restarted qa-morebots per request [releng]
14:18 <Krenair> <jzerebecki> !log reloading zuul for 3f64dbd..c6411a1 [releng]
13:58 <jzerebecki> reloading zuul for 3f64dbd..c6411a1 [releng]
2016-04-14 §
21:51 <csteipp> enabled oath on beta for Staff global group [releng]
16:36 <csteipp> does some mucking around on deployment-mediawiki03 [releng]
2016-04-13 §
01:48 <legoktm> deploying https://gerrit.wikimedia.org/r/282952 [releng]
2016-04-12 §
19:47 <bd808> Cleaned up large hhbc cache file on deployment-medaiwiki03 via `sudo service hhvm stop; sudo rm /var/cache/hhvm/fcgi.hhbc.sq3; sudo service hhvm start` [releng]
19:47 <bd808> Cleaned up large hhbc cache file on deployment-medaiwiki02 via `sudo service hhvm stop; sudo rm /var/cache/hhvm/fcgi.hhbc.sq3; sudo service hhvm start` [releng]
19:46 <bd808> Cleaned up large hhbc cache file on deployment-medaiwiki01 via `sudo service hhvm stop; sudo rm /var/cache/hhvm/fcgi.hhbc.sq3; sudo service hhvm start` [releng]
19:10 <Amir1> manually rebooted deployment-ores-web [releng]
19:08 <Amir1> manually cherry-picked 282992/2 into to puppetmaster [releng]
17:05 <Amir1> ran puppet agen in sca01 manually in /srv directory [releng]
11:34 <hashar> Jenkins upgrading "Script Security Plugin" from 1.17 to 1.18.1 https://wiki.jenkins-ci.org/display/SECURITY/Jenkins+Security+Advisory+2016-04-11 [releng]
2016-04-11 §
21:23 <csteipp> deployed and reverted oath [releng]
20:30 <thcipriani> relaunched slave-agent on integration-slave-trusty-1025, back online [releng]
20:18 <thcipriani> integration-slave-trusty-1025 horizon console filled with INFO: task jbd2/vda1-8:170 blocked for more than 120 seconds. rebooting [releng]
20:13 <thcipriani> killing stuck jobs, marking integration-slave-trusty-1025 as offline temporarily [releng]
14:42 <thcipriani> deployment-mediawiki01 disk full :( [releng]