151-200 of 7194 results (4ms)
2010-06-03 §
19:39 <mark> Moved mobile1 switchport from vlan 101 to 100 [production]
19:36 <mark> Reverted DNS change of mobile1, back to .157 [production]
17:21 <Fred> mobile1 going to be unreacheable while re-ip'ing [production]
14:05 <midom> synchronized php-1.5/wmf-config/InitialiseSettings.php 'timezone change for bat-smg' [production]
11:53 <mark> Made m.wikipedia.org CNAME m.wikipedia.org, m.wikipedia.org A to mobile1/2 in RR [production]
10:57 <hcatlin> mobile2 has been rebuilt and is featuring the new apache/mobile stack taking 40% of all mobile traffic. pls help monitor on ganglia. [production]
09:04 <Tim> cleaning COSS on sq45, resynced its configuration, will start squid when done [production]
08:58 <Tim> kernel reports degraded RAID on sq33, sq34, sq35, sq37, sq38, sq40 [production]
08:39 <Tim> checked all serial consoles, all nonresponsive, rebooted all [production]
08:23 <Tim> sq33, sq34, sq35, sq37, sq38, sq40, sq45 have been down for 16-28 days, apparently for no good reason, can't find any log or DT entries. Will try restarts. [production]
07:56 <Tim> added new squids to nagios [production]
06:36 <Tim> cleaning cache directories on sq56 to avoid resurrection of expired content [production]
06:35 <Tim> adding monitoring for rather important service IPs: upload.esams and text.esams [production]
06:22 <Tim> sq56 not responding to ping or serial console (for 4 days), nothing in racadm getsel, rebooting [production]
06:07 <tstarling> synchronized php-1.5/wmf-config/InitialiseSettings.php 'disabling ClickTracking due to CR r58099' [production]
05:24 <Tim> started apache on srv216, was stopped for some reason [production]
03:57 <Fred> shutting down mailman on list for a few minutes while exim and spamd catch up [production]
01:42 <Tim> adding forward and reverse DNS for mobile.tesla.usability.wikimedia.org, 208.80.152.245 [production]
2010-06-02 §
23:48 <Fred> re-imaging mobile2 again ;p [production]
21:12 <robh> ran sync-common-all [production]
20:57 <Rob> srv134 has temp errors and multiple bad fans, out of warranty, decommissioning and removing from nagios/lvs/dsh [production]
20:52 <jeluf> synchronized php-1.5/wmf-config/InitialiseSettings.php '23756 - Enable "Rollbacker" group on itwiki' [production]
20:44 <Rob> how many ops does it take to bring srv281 back online? one + a vounteer to point out he did it wrong. [production]
20:42 <robh> ran sync-common-all [production]
20:36 <robh> ran sync-common-all [production]
20:26 <jeluf> synchronizing Wikimedia installation... Revision: 66620 [production]
20:15 <Rob> srv281 back online and apache files updated [production]
20:08 <Rob> lomaria is down, because it was throwing crap around and generally broken, so its shutdown now. [production]
19:59 <Rob> had to swap power cable on msw-b5-sdtpa [production]
14:02 <Rob> stay off db28, fixing its firmware. [production]
13:44 <andrew> synchronized php-1.5/extensions/StrategyWiki/ActiveStrategy/ActiveStrategy_body.php [production]
13:39 <andrew> synchronized php-1.5/extensions/StrategyWiki/ActiveStrategy/ActiveStrategy_body.php [production]
13:38 <Tim> ArchRunner on lily was stuck in a busy loop of some kind, had to restart it with kill -9 [production]
13:36 <andrew> synchronized php-1.5/extensions/StrategyWiki/ActiveStrategy/ActiveStrategy_body.php [production]
13:35 <andrew> synchronized php-1.5/extensions/StrategyWiki/ActiveStrategy/ActiveStrategy_body.php [production]
13:32 <andrew> synchronized php-1.5/extensions/StrategyWiki/ActiveStrategy/ActiveStrategy_body.php [production]
13:32 <andrew> synchronized php-1.5/extensions/StrategyWiki/ActiveStrategy/ParserFunctions.php [production]
13:31 <andrew> synchronized php-1.5/extensions/StrategyWiki/ActiveStrategy/ActiveStrategy.i18n.php [production]
13:31 <andrew> synchronized php-1.5/extensions/StrategyWiki/ActiveStrategy/ActiveStrategy.php [production]
06:49 <tstarling> synchronized php-1.5/extensions/CodeReview/backend/Subversion.php 'r67213' [production]
06:45 <Tim> updated codereview-proxy on isidore to r67211 [production]
06:13 <Tim> attempting to set up CodeReview for pywikipedia [production]
05:48 <jeluf> synchronized php-1.5/wmf-config/InitialiseSettings.php '23705 - Change namespaces in ja.wikipedia' [production]
05:33 <jeluf> synchronized php-1.5/wmf-config/flaggedrevs.php '23742 - make category namespace flaggable on en.wikinews' [production]
05:29 <jeluf> synchronized php-1.5/wmf-config/InitialiseSettings.php '23572 - Imagemap cannot accept dynamic links.' [production]
2010-06-01 §
23:10 <Fred> mobile2 re-installed and puppetized [production]
22:44 <Fred> re-imaging mobile2 for total lucid puppetization. [production]
20:17 <Rob> mobile1 decided no more ssh server, its reinstalled and defined in puppet so it is ok now. [production]
19:42 <catrope> synchronized php-1.5/languages/messages/MessagesCkb_arab.php 'r67198' [production]
14:26 <catrope> synchronized php-1.5/extensions/CategoryTree/CategoryTreeFunctions.php 'r67183' [production]