651-700 of 10000 results (16ms)
2012-03-12 §
05:53 <mutante> dunno, copper was stuck (no mgmt output after reboot) but powercycling it and back [production]
05:43 <mutante> rebooting copper to make sure grub update didnt break it and asked for restart anyways [production]
05:37 <mutante> copper - installing (security) updates (apt,grub,openssl,ruby,libc6..) [production]
03:40 <mutante> same (and nscd) on fenari [production]
03:35 <mutante> upgrading libc6 and related packages on spence [production]
02:17 <LocalisationUpdate> completed (1.19) at Mon Mar 12 02:17:28 UTC 2012 [production]
2012-03-11 §
08:14 <apergos> restarted lighttp on dataset2 [production]
07:49 <apergos> removed current htcp log file, restarted purger, it seems to be logging normallynow [production]
07:35 <apergos> current ls shows 17416851456 2012-03-11 07:34 HTCPpurger.log while current du -sh shows 175M for /var/log. Sparse file that gets rotated badly? lots of leading nulls (many gb worth), why? [production]
07:33 <apergos> on ms1004 the HTCPpurger.log file after rotation was 17 gb, filling the disk. Removed it. [production]
02:17 <LocalisationUpdate> completed (1.19) at Sun Mar 11 02:17:35 UTC 2012 [production]
2012-03-10 §
22:09 <Reedy> Make that wikimania2012, not wikimediawiki [production]
22:08 <reedy> synchronized wmf-config/InitialiseSettings.php 'Disable anon page creation for wikimediawiki' [production]
19:28 <binasher> set sync_binlog = 1 on all current masters and eqiad dbs [production]
19:22 <binasher> reslaved db1033 [production]
07:03 <mutante> ran puppet on db1022, another one that works fine manually but somehow did not by itself [production]
05:11 <mutante> doing more (cp*, db*, msbe-* ,mw*) by hand / for loop [production]
05:01 <mutante> starting nagios-nrpe-server on all via dsh (fail to restart on config change issue) [production]
02:16 <LocalisationUpdate> completed (1.19) at Sat Mar 10 02:16:57 UTC 2012 [production]
01:07 <maplebed> started swiftcleaner on owa1 looking for (and purging) bad objects [production]
01:06 <maplebed> rebalanced the swift rings to finish decreasing traffic sent to ms1 and ms2 [production]
00:18 <Ryan_Lane> powercycling ssl1003 [production]
00:18 <Ryan_Lane> powercycling ssl1001 [production]
2012-03-09 §
20:34 <notpeter> stopping search indexer on searchidx2 for fresh rsync to searchidx1001 [production]
19:58 <preilly> pushed change to remove description from landing page [production]
19:57 <preilly> synchronized php-1.19/extensions/ZeroRatedMobileAccess/ZeroRatedMobileAccess.body.php 'changes for zero' [production]
18:59 <Ryan_Lane> sending test.m.wikipedia.org to the same place as test.wikipedia.org via squid [production]
18:58 <awjrichards> synchronized wmf-config/InitialiseSettings.php 'Fixing wgMobileUrlTemplate settings for domains that do not have .m. domains configured' [production]
18:48 <reedy> synchronized php-1.19/extensions/WikiLove/modules/ext.wikiLove/ext.wikiLove.css '[[rev:113497|r113497]]' [production]
18:40 <awjrichards> synchronized wmf-config/CommonSettings.php 'Changing the way in which wgMobileUrlTemplate is configurable by InitialiseSettings.php' [production]
18:39 <awjrichards> synchronized wmf-config/InitialiseSettings.php 'Disabling wgMobileUrlTemplate for testwiki - hopefully for real this time' [production]
18:34 <awjrichards> synchronized wmf-config/CommonSettings.php 'Making wgMobileUrlTemplate configurable by InitialiseSettings.php' [production]
18:34 <awjrichards> synchronized wmf-config/InitialiseSettings.php 'Disabling wgMobileUrlTemplate for testwiki' [production]
17:40 <awjrichards> synchronized php/extensions/MobileFrontend/templates/ApplicationTemplate.php '[[rev:113489|r113489]]' [production]
17:32 <maplebed> set swift storage device weight on ms2 to 0 and pushed out rings [production]
15:52 <apergos> cleared up a little bit of space on root partition of snapshot2, but that's about it. I hope we never have 3 versions of mw in test at the same time, the tmp caches will kill us [production]
15:52 <mark> Turned off vcc_err_unref on all varnish servers, so varnish doesn't complain when ACLs/probes/backends are unused [production]
15:44 <Jeff_Green> hume apt upgrades, puppetd --test, switch to mysql 5.1.53-fb3753-wm1 [production]
06:38 <Ryan_Lane> reloading autofs on all labs instances [production]
06:13 <Tim> running svn cleanup on extdist trunk [production]
04:18 <Tim> switched php and wmf-deployment symlinks over to php-1.19 instead of php-1.18 [production]
04:18 <Tim> restarted morebots [production]
00:57 <pp-pdf2> updated pyfribidi to 0.11.0 fixing https://github.com/pediapress/pyfribidi/issues/2 [production]
00:57 <pp-pdf3> updated pyfribidi to 0.11.0 fixing https://github.com/pediapress/pyfribidi/issues/2 [production]
00:57 <pp-pdf1> updated pyfribidi to 0.11.0 fixing https://github.com/pediapress/pyfribidi/issues/2 [production]
00:38 <preilly> synchronized wmf-config/InitialiseSettings.php 'add ZeroRatedMobileAccess extension to mywiki' [production]
00:32 <preilly> synchronized php-1.19/extensions/MobileFrontend/javascripts/beta_opensearch.js 'fixes to code push' [production]
00:32 <preilly> synchronized php-1.19/extensions/MobileFrontend/javascripts/beta_opensearch.min.js 'fixes to code push' [production]
00:27 <preilly> synchronized php-1.19/extensions/MobileFrontend/templates/ApplicationTemplate.php 'fixes to code push' [production]
00:27 <preilly> synchronized php-1.19/extensions/MobileFrontend/javascripts/opensearch.js 'fixes to code push' [production]