751-800 of 10000 results (18ms)
2012-02-10 §
17:58 <RobH> updating dns for oxygen internal ip [production]
17:21 <mutante> labs logging is broken [production]
17:14 <RobH> oxygen offline for hard disk upgrade to replace locke [production]
16:50 <mutante> running sync-apache, trying to redirect office.wm to https [production]
16:07 <mark> Rebalanced appserver load balancing by giving the new mw* pmtpa app servers weight 150 in the pybal server list [production]
15:17 <mark> Turned on KeepAlive on apaches for better miss service times from eqiad [production]
13:42 <mark> Configured cp1001 and cp1020 to contact backend servers directly instead of via pmtpa squids [production]
12:02 <mark> Decommissioning sq38, sq46 and sq47 in squid configurator [production]
11:50 <mark> Making cp1001-1005 API squids [production]
05:08 <maplebed> deployed squid config to uploads to send 100% of thumbnail traffic to swift [production]
02:49 <maplebed> deploying fix for & bug with swift (files with an & in the name wouldn't load properly) [production]
02:18 <LocalisationUpdate> completed (1.18) at Fri Feb 10 02:18:37 UTC 2012 [production]
00:22 <LeslieCarr> increased nagios max concurrent checks on spence and lowered the interval between processing them [production]
00:20 <maplebed> deployed squid config to upload squids rolling thumbnails back to 75% handled by swift to test the & bug [production]
2012-02-09 §
22:22 <Jeff_Green> adding community-analytics.wikimedia.org to DNS [production]
21:36 <reedy> synchronized wmf-config/CommonSettings.php 'Increase wgCodeReviewMaxDiffPaths' [production]
20:04 <binasher> started 1.19 schema migrations on seconadry db's (mwscript upgrade-1.19wmf1-1.php --secondary) [production]
19:32 <reedy> synchronized wmf-config/CommonSettings.php 'Increase wgCodeReviewMaxDiffPaths' [production]
17:31 <maplebed> deployed squid config to upload squids sending 100% of all thumbnail traffic to swift [production]
16:31 <mark> Sending ALL non-european wikipedia traffic to eqiad text squids [production]
16:17 <mark> Added Brazil traffic to eqiad text squids [production]
15:49 <mark> Sending some Asian wikipedia traffic to wikipedia-lb.eqiad [production]
15:13 <mark> Sending Canadian wikipedia traffic to wikipedia-lb.eqiad [production]
14:27 <reedy> synchronized wmf-config/InitialiseSettings.php 'Bug 34183 - add alias redirection in ml wikisource' [production]
14:23 <mark> Redirecting {wikiquote,wikisource,wikiversity,wiktionary}-lb.pmtpa traffic to .eqiad (geodns) [production]
14:17 <reedy> synchronized wmf-config/InitialiseSettings.php 'Bug 34253 - Change site logo for lb.wiktionary' [production]
14:11 <mark> Redirecting {foundation,wikibooks,wikinews}-lb.pmtpa traffic to .eqiad (geodns) [production]
14:10 <hashar> synchronized docroot/mediawiki/xml 'update html pages for 0.1 and 0.2 schemes' [production]
14:06 <hashar> synchronized docroot/mediawiki/xml/index.html 'Give a home page to our XML namespaces homepage' [production]
13:52 <hashar> synchronized docroot/mediawiki/xml [production]
13:45 <mark> Redirecting wikimedia-lb.pmtpa traffic to wikimedia-lb.eqiad (geodns) [production]
13:28 <mark> Redirecting mediawiki-lb.pmtpa traffic to mediawiki-lb.eqiad (geodns) [production]
12:49 <apergos> around 11:05 UTC, increased /proc/sys/vm/min_free_kbytes from 16259 to 262144 to check impact on network alloc issues, on dataset1001 [production]
11:56 <mark_> Power cycled cp1017 [production]
11:50 <mark_> Changed topology of eqiad text squids to request from pmtpa (similar to esams) [production]
09:29 <mark_> Reactivated term selected-paths in policy-statement BGP_transit_in on cr2-eqiad, making path 14907 3257 1299 43821 active again [production]
02:26 <LocalisationUpdate> completed (1.18) at Thu Feb 9 02:26:49 UTC 2012 [production]
00:35 <asher> synchronized wmf-config/db.php 'returning db38 to s1 post schema migration testing' [production]
2012-02-08 §
23:52 <asher> synchronized wmf-config/db.php 'returning db11' [production]
23:49 <asher> synchronized wmf-config/db.php 'pulling db11' [production]
22:01 <maplebed> deployed updated to upload squid.conf to send 75% of all thumbnail traffic to swift [production]
17:37 <maplebed> deployed new squid config to upload to direct 50% of thumbnail traffic to swift [production]
17:21 <apergos> reboot of dataset1001 for testing. [production]
14:21 <mutante> srv189 - shut down again, PCI Express Error, -> RT 2413 created [production]
14:15 <mutante> powercycling srv189 [production]
13:09 <RoanKattouw> Batch-deleting 9,760 redirects on cswiktionary, requested by Danny B [production]
02:26 <LocalisationUpdate> completed (1.18) at Wed Feb 8 02:26:55 UTC 2012 [production]
2012-02-07 §
22:11 <maplebed> increasing traffic to swift from 12.5% to 25% [production]
21:19 <apergos> reboot dataset1001, new kernel 2.6.38, should deal with kswap bug we just ran into under heavy i/o load, need to test [production]
20:24 <mark> Uncommented sq63 in squid text-settings.php, please DO NOT COMMENT DOWN SQUIDS UNLESS DECOMMISSIONING THEM PERMANENTLY [production]