5101-5150 of 6287 results (15ms)
2014-02-06 §
14:43 <hashar> restarting udp2log-mw on deployment-bastion. Logstash.wmflabs.org no more receiving fatals logs since Jan 31st [releng]
2014-02-04 §
17:22 <hashar> fixed up beta-parsoid-update job so Parsoid should be up to date again. The issue is that the multigit job pointed to a wrong host (ZUUL_URL should be zuul.eqiad.wmnet) [releng]
13:33 <hashar> removing role::memcached from both apache servers [releng]
09:58 <hashar> rebooting all varnish caches [releng]
09:57 <hashar> Upgrading all varnish [releng]
2014-02-03 §
16:59 <hashar> upgrading varnish on deployment-parsoidcache3 [releng]
2014-01-30 §
19:35 <hashar> deployment-cache-bits03 restarted gmond, leaked memory. Upgrading varnish [releng]
19:32 <hashar> Canceled varnish package upgrade on deployment-cache-mobile01 , it runs a specific version ( 3.0.5plus~wmftest-wm1 ) instead of 3.0.3plus~rc1-wm29 [releng]
19:30 <hashar> upgrading varnish on deployment-cache-mobile01 [releng]
19:29 <hashar> upgrading varnish on deployment-cache-bits03 [releng]
19:29 <hashar> upgrading varnish on deployment-staging-cache-mobile02 [releng]
19:28 <hashar> upgrading varnish on deployment-cache-upload04 [releng]
19:27 <hashar> reenabling puppet on deployment-cache-mobile01 [releng]
17:10 <manybubbles> done reindexing beta. everything looks good [releng]
16:54 <manybubbles> reindexing beta like we're going to do in production when the release train departs later today [releng]
2014-01-28 §
17:10 <hashar> added addshore and jhall to project so they can grep logs [releng]
2014-01-27 §
15:17 <hashar> applying role::beta::fatalmonitor puppet class on deployment-bastion {{bug|60046}} [releng]
2014-01-23 §
19:38 <hashar> VisualEditor was not being updated properly because some files belonged to root instead of mwdeploy. Ran chown -R mwdeploy:mwdeploy /data/project/apache/common-local/php-master/extensions/VisualEditor [releng]
2014-01-16 §
20:54 <manybubbles> turning elasticsearch's disk space aware allocator [releng]
2014-01-15 §
21:14 <manybubbles> finished updating to elasticsearch 0.90.10 [releng]
08:48 <andrewbogott> rebooted deployment-cache-text1 [releng]
2014-01-02 §
15:32 <hashar> Migrated parsoid on deployment-parsoid2 to use mediawiki/services/parsoid out of a checkouts made in /srv/deployment/parsoid/{parsoid,deploy}. No job self updating it yet [releng]
15:00 <manybubbles> finished upgrading Elasticsearch in beta. We're on 0.90.9 now. [releng]
14:07 <hashar> running mw-update-l10n , it was broken because of https://gerrit.wikimedia.org/r/#/c/104741/ fixed up by https://gerrit.wikimedia.org/r/#/c/104953/ [releng]
13:54 <manybubbles> upgrading Elasticsearch servers in beta [releng]
2013-12-26 §
18:54 <manybubbles> performing in place index rebuild for wikis in beta after recent cirrus update [releng]
2013-12-23 §
20:40 <anomie> Restarting mw-job-runner service on deployment-jobrunner08, since jobs don't seem to be being run [releng]
20:03 <anomie> Restarting apache on deployment-apache33 to see if that clears the odd errors going on [releng]
2013-12-18 §
10:56 <hashar> reenabling puppet on parsoid2 and deploying the new Parsoid upstart configuration {{gerrit|99656}} [releng]
2013-12-13 §
15:15 <hashar> Beta apaches port 80 were firewalled out. Got fixed up :-} [releng]
14:38 <hashar> applied role::beta::natfixup to several instances, that is meant to rewrite IP packet trying to access the public IP of beta instances. See {{bug|45868}} [releng]
10:51 <hashar> update operations/puppet repository on deployment-staging-cache-mobile02 [releng]
10:51 <hashar> apt upgrade of deployment-staging-cache-mobile02 [releng]
2013-12-11 §
12:36 <hashar> Migrated old SAL entries to subpage [[Nova_Resource:Deployment-prep/SAL/Archive1]] [releng]
12:29 <hashar> test [releng]
10:44 <hashar> deleted deployment-search01 and deployment-searchidx01 , Beta cluster has been migrated to ElasticSearch over the summer. [releng]
2013-12-10 §
23:19 <bd808> Freed 4.2G on deployment-jobrunner08.pmtpa.wmflab by deleting files in /tmp [releng]
23:17 <bd808> deployment-jobrunner08.pmtpa.wmflabs is out of disk on / [releng]
21:11 <hashar> used git fetch && git reset --hard on Flow extension. Just to be sure [releng]
2013-12-09 §
09:52 <hashar> added dan-nl so he can look at MediaWiki log files when playing with glamtoolset [releng]
09:44 <hashar> deleted old jobs from commonswiki job queue (up to timestamp 20130315031930) [releng]
2013-12-08 §
20:55 <bd808> chmod -R a+w deployment-bastion:/data/project/upload7/private/gwtoolset [releng]
2013-12-06 §
22:22 <hashar> rebooting deployment-cache-text1 (aka text varnish) [releng]
22:11 <hashar> upgrading packages on text cache / running puppet and rebooting it. [releng]
21:15 <MaxSem> Debugging apache on deployment-apache33, may look hung [releng]
11:24 <hashar> upgrading varnish on deployment-parsoidcache [releng]
11:18 <hashar> made sure puppet agent is enabled on varnish caches and reran it manually [releng]
11:13 <hashar> shut downing deployment-search01 and deployment-searchidx01. They were for Lucene search. We use Elastic search now. [releng]
11:12 <hashar> upgrading varnish on deployment-cache-mobile01 [releng]
11:12 <hashar> upgrading varnish on deployment-cache-bits03 [releng]