751-800 of 10000 results (14ms)
2011-11-28 §
18:43 <hashar:> jenkins backup. The host was missing the SSL cert and apache did not start up. Fixed by Daniel Zahn https://gerrit.wikimedia.org/r/1156 [production]
17:43 <mutante> gallium: installing new kernel and rebooting [production]
17:42 <hashar:> wrongly stopped jenkins while mutante was doing an upgrade. Jenkins process dead for some minutes :( [production]
17:40 <hashar:> restarting jenkins [production]
17:15 <mutante> gallium - installing some upgrades: apache2..,apt,bind9/dnsutils,,php5..,openjdk,puppet..RT 2039 [production]
16:38 <reedy> synchronizing Wikimedia installation... : Pushing [[rev:104448|r104448]] [production]
16:37 <mutante> fenari: re-installed parsekit via pecl, added extension=parsekit.so to php5/cli/php.ini [production]
15:22 <RoanKattouw> Started logmsgbot on fenari, now running as me. Need to puppetize it with ircecho now that fenari has been upgraded to lucid [production]
15:21 <reedy> synchronized php-1.18/includes/api/ApiHelp.php '[[rev:104439|r104439]]' [production]
15:21 <Reedy> sync-file php-1.18/includes/api/ApiMain.php [[rev:104439|r104439]] [production]
2011-11-27 §
23:24 <Reedy> Disabled ContributionReportings extension (about 15 minutes ago) [production]
22:47 <mark> Restarted ganglia on spence [production]
22:41 <notpeter> restarting nagios on spence [production]
22:24 <mark> Shutdown mysql on db1025 [production]
2011-11-26 §
22:21 <mark> Upgraded puppet client to 2.7.7rc2 on formey (expecting lint breakage) [production]
18:25 <mark> Let puppet upgrade puppet master and puppet agent to 2.7.7rc2 on sockpuppet [production]
2011-11-25 §
18:10 <mutante> secure.wm is up again, thanks Rob [production]
16:59 <mutante> singer down :/ - http://wikitech.wikimedia.org/view/Singer#fail - secure.wm and others affected [production]
16:07 <mutante> power cycling singer [production]
15:48 <mutante> restarting singer to finish upgrade [production]
15:21 <mutante> starting dist-upgrade on singer [production]
15:12 <mutante> removing rpm and alien package from fenari [production]
14:47 <mutante> hooper - done, Blog and Etherpad accessible again [production]
14:44 <mutante> hooper - rebooting for upgrade [production]
14:41 <mutante> hooper - installing new kernel,apache2 upgrades,dist-upgrade (within lucid) [production]
13:41 <mutante> rebooting hume to finish upgrade [production]
12:57 <mutante> starting to dist-upgrade hume [production]
12:33 <mutante> stopping nagios and rebooting spence to finish kernel upgrade [production]
12:26 <mutante> reinstalled wikimedia-task-appserver on fenari - fixed dpkg conflict php5-apc vs php-apc by manually installing php-apc first [production]
12:23 <mutante> dist-upgrading spence (within same release) [production]
12:23 <mutante> removing php5-apc from spence [production]
12:09 <mark> Fixed puppet in labs [production]
11:36 <mutante> fenari back up as lucid :) feel free to use again [production]
11:32 <mutante> side-note: sh: /home/wikipedia/bin/start-ircbot: not found [production]
11:29 <mutante> rebooting fenari to finish upgrade [production]
10:59 <mutante> starting fenari dist-upgrade [production]
09:38 <mark> whipped aluminium back into submission by killing rfaulk's python processes [production]
09:27 <apergos> email notification was off momentarily for the second and last batch of bugzilla cleanup. now back on. [production]
07:50 <apergos> disabled user account creation again on bugzilla for the moment [production]
01:57 <LocalisationUpdate> failed [production]
01:52 <Tim> created arbcom 2011 vote on enwiki [production]
01:47 <Tim> on hume: running makeArbcomList.php in tstarling screen [production]
00:06 <rainman-sr> set the "last indexed" dates back by 2 days for all wikis so that the wrongly indexed pages get reindexes properly [production]
2011-11-24 §
20:44 <mark> Puppet is broken in labs [production]
19:32 <mark> Letting puppet upgrade itself on virt1 [production]
19:04 <robla> new account creation on BZ reenabled [production]
18:59 <apergos> added manually the editbugs right for all users on bugzilla; automatic right via regexp removed by robla [production]
18:12 <mark> Temporarily disabled puppet on sockpuppet and virt1 [production]
18:02 <mark> Installed new Puppet 2.7.7rc2 packages into the Wikimedia APT repository (lucid-wikimedia); hardy packages will follow if needed [production]
13:50 <catrope> synchronized php-1.18/extensions/GlobalUsage/GlobalUsage_body.php '[[rev:104163|r104163]]' [production]