151-200 of 10000 results (10ms)
2012-02-12 §
12:09 <mark> Killed lsearchd processes on search8, restarted [production]
12:07 <mark> Rebalanced mw API app servers from load 120 to 150 in pybal list [production]
10:08 <mark> Increased MaxClients to 100 on API apaches in Puppet [production]
09:45 <mark> Restricted only opensearch API requests to the API squids [production]
09:43 <mark> Restricted only opensearch API requests to the API backend apaches, other API requests now hit the main mediawiki cluster [production]
08:44 <mark> maximum_forwards change deployed to all squids [production]
08:42 <mark> Set maximum_forwards 2 in squid.conf, deployed to the API squids only so far, rest is pending [production]
07:52 <binasher> restarted lsearchd on search{3,4,9} [production]
02:19 <LocalisationUpdate> completed (1.18) at Sun Feb 12 02:19:17 UTC 2012 [production]
2012-02-11 §
20:31 <apergos> restarted lightty on dataset2 [production]
17:28 <RobH> manual test of each affected service complete, db9 fully online. [production]
17:26 <RobH> db9 moved, all systems online [production]
17:08 <RobH> db9 shutting down to move racks, offline during this includes: blogs, bugzilla, racktables, rt, survey, etherpad, observium [production]
02:18 <LocalisationUpdate> completed (1.18) at Sat Feb 11 02:18:36 UTC 2012 [production]
00:17 <reedy> synchronizing Wikimedia installation... : [production]
2012-02-10 §
22:17 <LeslieCarr> fixing the labs apache2 puppet groups [production]
21:48 <RobH> memory in cp1017 wasnt properly seated as far as i can tell, if it doesnt mess up again it should be ok. [production]
21:41 <RobH> cp1017 being tested for bad memory [production]
21:36 <RobH> powercycling msw-a2-eqiad resolves all mgmt issues in rack [production]
21:34 <RobH> powercycling msw-a1-eqiad. [production]
21:29 <RobH> db1001 rebooting, locked up [production]
20:53 <RobH> updating dns for new db hosts [production]
19:59 <Reedy> Checking out 1.19wmf1 to /tmp on fenari [production]
19:12 <RobH> oxygen setup and installed per rt2343, still needs puppet runs and full deployment per rt 2430 [production]
17:58 <RobH> updating dns for oxygen internal ip [production]
17:21 <mutante> labs logging is broken [production]
17:14 <RobH> oxygen offline for hard disk upgrade to replace locke [production]
16:50 <mutante> running sync-apache, trying to redirect office.wm to https [production]
16:07 <mark> Rebalanced appserver load balancing by giving the new mw* pmtpa app servers weight 150 in the pybal server list [production]
15:17 <mark> Turned on KeepAlive on apaches for better miss service times from eqiad [production]
13:42 <mark> Configured cp1001 and cp1020 to contact backend servers directly instead of via pmtpa squids [production]
12:02 <mark> Decommissioning sq38, sq46 and sq47 in squid configurator [production]
11:50 <mark> Making cp1001-1005 API squids [production]
05:08 <maplebed> deployed squid config to uploads to send 100% of thumbnail traffic to swift [production]
02:49 <maplebed> deploying fix for & bug with swift (files with an & in the name wouldn't load properly) [production]
02:18 <LocalisationUpdate> completed (1.18) at Fri Feb 10 02:18:37 UTC 2012 [production]
00:22 <LeslieCarr> increased nagios max concurrent checks on spence and lowered the interval between processing them [production]
00:20 <maplebed> deployed squid config to upload squids rolling thumbnails back to 75% handled by swift to test the & bug [production]
2012-02-09 §
22:22 <Jeff_Green> adding community-analytics.wikimedia.org to DNS [production]
21:36 <reedy> synchronized wmf-config/CommonSettings.php 'Increase wgCodeReviewMaxDiffPaths' [production]
20:04 <binasher> started 1.19 schema migrations on seconadry db's (mwscript upgrade-1.19wmf1-1.php --secondary) [production]
19:32 <reedy> synchronized wmf-config/CommonSettings.php 'Increase wgCodeReviewMaxDiffPaths' [production]
17:31 <maplebed> deployed squid config to upload squids sending 100% of all thumbnail traffic to swift [production]
16:31 <mark> Sending ALL non-european wikipedia traffic to eqiad text squids [production]
16:17 <mark> Added Brazil traffic to eqiad text squids [production]
15:49 <mark> Sending some Asian wikipedia traffic to wikipedia-lb.eqiad [production]
15:13 <mark> Sending Canadian wikipedia traffic to wikipedia-lb.eqiad [production]
14:27 <reedy> synchronized wmf-config/InitialiseSettings.php 'Bug 34183 - add alias redirection in ml wikisource' [production]
14:23 <mark> Redirecting {wikiquote,wikisource,wikiversity,wiktionary}-lb.pmtpa traffic to .eqiad (geodns) [production]
14:17 <reedy> synchronized wmf-config/InitialiseSettings.php 'Bug 34253 - Change site logo for lb.wiktionary' [production]