1-50 of 4345 results (4ms)
2009-11-03 §
23:47 <tstarling> synchronized wmf-deployment/cache/trusted-xff.cdb [production]
23:23 <Tim> deploying trusted XFF update from r58506 [production]
23:12 <midom> synchronized php-1.5/wmf-config/db.php 'removing db28' [production]
23:10 <Tim> restarted udp2log on locke, sampled-1000.log was empty [production]
19:32 <andrew> synchronized php-1.5/extensions/LiquidThreads/classes/NewMessagesController.php 'Deploy r58494' [production]
16:12 <mark> Recovered torrus from its deadlocked state [production]
15:24 <mark> Setup APT repository and preseeding repository on brewster, using puppet [production]
15:21 <mark> Moved apt.wikimedia.org DNS CNAME from khaldun to brewster [production]
15:17 <mark> Shutdown BGP session to AS 16150; no connectivity to wikitech.wikimedia.org [production]
15:14 <mark> Added --timeout=30 to the wget invocations in the post-commit script on svn.wikimedia.org [production]
14:17 <Andrew> Scapping [production]
14:09 <fvassard> synchronized php-1.5/wmf-config/mc.php 'swapped dead memcache from srv217 to srv156' [production]
14:06 <Andrew> Will scap to update LiquidThreads in a few minutes [production]
2009-11-02 §
19:30 <aaron> synchronized php-1.5/includes/FileDeleteForm.php 'deployed r58429' [production]
14:12 <Fred> We are running out of memcache spare boxes... Need some cleanup of the "downned memcached" [production]
14:12 <fvassard> synchronized php-1.5/wmf-config/mc.php 'swapped dead memcache from srv146 to srv217' [production]
2009-11-01 §
19:41 <rainman-sr> started search indexer on searchidx1 [production]
19:22 <mark> Renamed db6 to locke in DNS and racktables [production]
19:04 <mark> Powercycled frozen server searchidx1 [production]
18:59 <mark> synchronized php-1.5/wmf-config/InitialiseSettings.php 'Changed proxy host for file-by-url testing on test.wikipedia.org from khaldun to brewster' [production]
18:59 <mark> Stopped oprofile on srv218 and removed all samples; its disk was full [production]
18:54 <mark> synchronized php-1.5/wmf-config/InitialiseSettings.php [production]
18:53 <mark> Setup Squid proxy on brewster, and setup selective proxying for security.ubuntu.com through it on all hosts, using puppet [production]
17:44 <mark> Set up tftp server on brewster (using puppet) [production]
16:54 <mark> Added ubuntu.wikimedia.org DNS entry [production]
16:45 <mark> Set up Ubuntu mirror on brewster using puppet [production]
16:45 <mark> Fixed Domas's IPC cleanup cron job to not run if there are no stale ipc semaphores [production]
12:48 <river> allocated vlan 301 to the toolserver, 10.23.1.0/24 (we're out of public IPs) [production]
2009-10-31 §
11:18 <catrope> synchronized php-1.5/includes/MagicWord.php 'Removing MagicWordArray live hack; problem was found and fixed yesterday' [production]
09:47 <Raymond_afk> ParserTests for CodeReview does not run. Last run for r58231 2009-10-27 [production]
00:48 <aaron> synchronized php-1.5/extensions/FlaggedRevs/FlaggedRevs.hooks.php 'deployed r58380' [production]
2009-10-30 §
23:11 <RoanKattouw> Rebuilt l10ncache on srv194, seems to have fixed it. Presumably caused by an incomplete run of a sync script caused by permissions errors (see private-l) [production]
23:00 <Fred> depooled srv194 while testing / fixing [production]
22:08 <Fred> actually stopped apache on srv194 since the restart did not help. Investigating.... [production]
22:07 <Fred> restarted apache on srv194 since it was throwing a lot of ""is not a valid magic thingie for xxxx" [production]
20:05 <Fred> scaped as root to resolve cache problems on some boxes. [production]
20:02 <fvassard> ran sync-common-all [production]
19:50 <RoanKattouw> Scapping in the hopes of resolving the MagicWordsArray exception; looks like it could be a problem with localisation cache [production]
19:37 <catrope> synchronized php-1.5/includes/MagicWord.php 'Live hack for debugging: make MagicWordArray::parseMatch parameter not found error more verbose' [production]
18:36 <catrope> synchronized php-1.5/extensions/LocalisationUpdate/LocalisationUpdate.class.php 'Deploy r58359 (LocalisationUpdate fix for languages with a '-' in their name)' [production]
18:33 <catrope> synchronized php-1.5/wmf-config/InitialiseSettings.php 'Per bug 19865, enable the enhanced toolbar by default on enwikinews and disable the OptIn extension' [production]
16:39 <RoanKattouw> sync-common-all threw lots of errors like srv175: rsync: failed to set times on "/usr/local/apache/common-local/wmf-deployment/includes/.svn": Operation not permitted (1) [production]
16:38 <catrope> ran sync-common-all [production]
16:36 <RoanKattouw> Scapping so srv194 will receive updates [production]
15:34 <apergos> cleared out some files in /tmp and ran apt-get clean on srv194 to get back about 1gb space on / [production]
14:08 <RoanKattouw> @wikimediatech feed on identi.ca has been broken for the past 3 days [production]
14:01 <RoanKattouw> srv194 has full disk, causing syncs to fail; throws PHP exception "MagicWordArray::parseMatch: parameter not found" [production]
11:47 <rainman-sr> searchidx1 ssh dead, shows 100% I/O, disk dead? [production]
2009-10-29 §
22:49 <catrope> synchronized php-1.5/includes/upload/UploadFromUrl.php 'Re-sync for srv226' [production]
22:48 <RoanKattouw> Scapping again for previous deployment because srv226 wouldn't let me log in at that time [production]