7901-7950 of 10000 results (28ms)
2010-11-10 §
04:59 <apergos> shot all dump processes on dataset1; note a kernel panic in logs from within __destroy_inode, going to reboot and leave rsync of pagecounts and dumps off [production]
00:41 <tfinc> synchronized php-1.5/wmf-config/CommonSettings.php 'Taking outage on cc cluster' [production]
2010-11-09 §
22:42 <RobH> running puppet on spence to remove all the old apaches that are no longer in any kind of service [production]
22:39 <jeluf> synchronized php-1.5/wmf-config/InitialiseSettings.php '24539 - Transwiki import source for ml.wikisource.org' [production]
22:36 <RobH> didnt log my change that I ran about 25 minutes ago to change test.w.o from srv124 to srv193 in squid settings and deployed [production]
22:32 <jeluf> ran sync-common-all [production]
22:27 <jeluf> synchronized closed.dblist [production]
21:32 <robh> synchronized php-1.5/wmf-config/mc.php 'removed srv193 from potential memcached pool as it will shortly become the new test.w.o server' [production]
20:53 <catrope> synchronized php-1.5/wmf-config/InitialiseSettings.php 'Set tenwiki logo to local Wiki.png' [production]
17:26 <robh> synchronized php-1.5/wmf-config/abusefilter.php 'bugzilla#24394' [production]
17:17 <robh> synchronized php-1.5/wmf-config/abusefilter.php [production]
17:15 <RobH> that actually ran 15 minutes ago and was stuck on a broken server at the end, all other hosts had synced [production]
17:15 <robh> synchronized php-1.5/wmf-config/mc.php 'srv230 unresponsive to ssh, needs to reboot, swapped it out for working spare' [production]
15:51 <RobH> removed srv* under srv151 from pybal, left entry for srv124 as its test.w.o, even though its set to false [production]
15:46 <RobH> srv229 puppet was hanging, manually ran apt-get update and reran puppet, now its happy [production]
15:22 <RobH> srv229 rebooted, wouldnt let me ssh in, coming back up now with puppet run [production]
15:21 <robh> synchronized php-1.5/wmf-config/mc.php 'srv281 is not behaving, swapped it out' [production]
15:17 <robh> synchronized php-1.5/wmf-config/mc.php 'replacing a server that i am working on' [production]
15:01 <mark> Reduced CARP weight of new amssq squids from 30 to 20, until they get SSDs [production]
12:16 <mark> Started copy of ms2:/a/ex-fedora data to tridge [production]
12:15 <mark> Included class base for server tridge in Puppet [production]
12:09 <mark> Cleaned up temporary files on image scalers [production]
06:33 <apergos> cleared out some old bin log files on db27 to get back some space [production]
05:20 <apergos> reboot of ms4 successful, we need to monitor performance over the next several days. If people are still seeing 503's for thumbs that's a problem [production]
05:09 <apergos> er, ms4!! [production]
05:09 <apergos> rebooting ms5 into alternate boot environ, now with new improved patches (:-P), let's see if it works [production]
01:00 <tfinc> synchronized php-1.5/wmf-config/CommonSettings.php 'Bumping style version for meta udp2log fix' [production]
2010-11-08 §
23:50 <robh> synchronized php-1.5/wmf-config/InitialiseSettings.php 'updating tenwiki logo' [production]
23:18 <tstarling> ran sync-common-all [production]
22:32 <jeluf> synchronized php-1.5/wmf-config/InitialiseSettings.php '25801 - New logo for et.wikimedia.org' [production]
22:27 <jeluf> synchronized php-1.5/wmf-config/InitialiseSettings.php '25742 Please set the Buryat Wikipedia logo' [production]
22:25 <jeluf> synchronized php-1.5/wmf-config/InitialiseSettings.php '25742 Please set the Buryat Wikipedia logo' [production]
22:20 <jeluf> synchronized php-1.5/wmf-config/InitialiseSettings.php '25836 - Commons' [production]
22:13 <jeluf> synchronized php-1.5/wmf-config/InitialiseSettings.php '25779 - Create new namespace "Institution"/"Museum" for Commons' [production]
21:55 <JeLuF> Fixed typo: Replaced the second "srv290" in node group "apaches" by srv296. [production]
21:10 <jeluf> synchronizing Wikimedia installation... Revision: 76208 [production]
21:10 <JeLuF> srv154 didn't receive any updates in the last few days, it was missing in the mediawiki-installation nodegroup [production]
20:19 <mark> Moved uplink of asw-b-sdtpa from temporary GigE link csw1-sdtpa:3/48 to 2x 10G (LACP) links csw1-sdtpa:16/2 and 16/3, shutdown 3/48 [production]
19:44 <nimishg> synchronized php-1.5/extensions/LandingCheck/SpecialLandingCheck.php 'r76156' [production]
19:44 <nimishg> synchronized php-1.5/extensions/LandingCheck/LandingCheck.php 'r76156' [production]
19:22 <RobH> updated dns, seems it borked, but pdns is running on nescio so it should clear up [production]
17:47 <robh> synchronized php-1.5/wmf-config/abusefilter.php 'bug' [production]
16:38 <RobH> esams squid flapping on text squids is due to disk i/o use, they will be replaced with SSD soon [production]
06:04 <apergos> brought ms4 back up in primary boot environment, testing concluded for tonight, results being sent to Oracle [production]
05:50 <apergos> doing reboot of ms4 into alternate boot environ for testing [production]
02:34 <apergos> restarted torrus, it was out to lunch again [production]
2010-11-07 §
17:39 <RoanKattouw> Starting makeArbcomList.php in a screen on fenari. Tim says this'll take about a day [production]
12:43 <catrope> synchronized php-1.5/includes/Profiler.php 'r76243' [production]
12:04 <catrope> synchronized php-1.5/wmf-config/abusefilter.php 'More perm changes for frwiktionary' [production]
11:59 <catrope> synchronized php-1.5/wmf-config/abusefilter.php 'More perm changes for frwiktionary' [production]