3101-3150 of 5991 results (9ms)
2009-07-16 ยง
22:51 <atglenn> sym link back in place, let's see what happens [production]
22:47 <atglenn> reverting temporarily while we resolve mount issues for the ms4 share [production]
22:40 <atglenn> ...whether the image scalers will fall over if we force them to do (some) regeneration. [production]
22:37 <atglenn> on ms1, /export/upload/wikipedia/en/thumb/0/00 symlinked to (shared from ms4) /mnt/thumbs/wikipedia/en/thumb/0/00 to test [production]
21:29 <brion> robots.php for robots.txt generation now also working. yay! [production]
21:28 <brion> synchronized live-1.5/robots.php [production]
21:28 <brion> extract2.php now fixed up for new deployment; portal pages ok (www.wikipedia.org) [production]
21:27 <brion> synchronized live-1.5/robots.php [production]
21:26 <brion> synchronized extract2.php [production]
21:26 <brion> synchronized extract2.php [production]
21:22 <brion> synchronized extract2.php [production]
21:18 <brion> ran sync-common-all [production]
21:18 <brion> rsync messed up the php-1.5 directory to symlink translation. retrying as root [production]
21:14 <brion> synchronized extract2.php [production]
21:13 <brion> synchronized extract2.php [production]
21:13 <atglenn> started copy of thumbnails to ms4, symlinks going in on ms1 (but no data removal yet) [production]
21:11 <brion> synchronized live-1.5/extract2.php [production]
21:10 <brion> synchronized live-1.5/robots.php [production]
21:09 <brion> ran sync-common-all [production]
21:08 <brion> attempting to replace the old php-1.5 dir with wmf-deployment symlink [production]
21:02 <brion> synchronized wmf-deployment/wmf-config/InitialiseSettings.php 'I think touching the new master InitialiseSettings will fix it' [production]
21:01 <brion> synchronized wmf-deployment/includes/GlobalFunctions.php 'mkdir error trackdown hack' [production]
20:54 <brion> synchronized wmf-deployment/wmf-config/missing.php [production]
20:52 <brion> synchronized wmf-deployment/wmf-config/CommonSettings.php [production]
20:52 <brion> synchronized wmf-deployment/wmf-config/reporting-setup.php [production]
20:48 <brion> switching all sites to wmf-deployment branch [production]
20:48 <brion> synchronized live-1.5/MWVersion.php [production]
19:06 <Tim> copying ExtensionDistributor stuff to ms4:/export/ext-dist, from root screen on ms1 [production]
19:01 <brion> Now running test.wikipedia.org, www.mediawiki.org, and meta.wikimedia.org on new deployment checkout [production]
19:01 <brion> synchronized live-1.5/MWVersion.php [production]
18:58 <brion> ran sync-common-all [production]
18:39 <Tim> restarted xinetd on zwinger [production]
18:24 <tstarling> synchronized php-1.5/CommonSettings.php [production]
17:57 <brion> also restarted 186, 196 which had some funkiness in php err log [production]
17:56 <brion> srv186 also bad sudo [production]
17:55 <brion> srv171 has some borkage; sudo config is broken can't run apache-restart as user [production]
17:52 <brion> ran sync-common-all [production]
17:51 <brion> running updated sync-common-all friendly to non-NFS boxes [production]
17:49 <brion> swapped private SVN-managed /home/wikipedia/bin into place [production]
15:09 <apergos> removing the last of our snapshots on ms1 :-( getting us a little more space [production]
14:47 <apergos> disabled snapshots on ms1 in preparation for move of thumbnails to ms4 [production]
14:38 <brion> updated wikibugs-l list config to allow bugzilla-daemon@wikimedia.org to post [production]
14:34 <brion> restarted wikibugs bot [production]
14:27 <brion> ms1 performance seems to be sucking again [production]
14:17 <brion> synchronized php-1.5/InitialiseSettings.php 'adjusting throttle temporarily for outreach event' [production]
11:55 <RoanKattouw> ExtensionDistributor repeatedly reported broken in the past 48 hrs [production]
07:08 <Fred> traffic profile switched back to normal. Esams is back to normal. [production]
06:11 <hcatlin> Mobile1 has returned to normal function. [production]
05:58 <hcatlin> Error after restarting mobile1 stopped stats logging from working. Stats will be low for July 15th and higher for July 16th. Parsing of the 6 hour log file (about 1GB) might slow server for next few minutes until caught up. [production]
04:24 <Rob> outage for esams servers started at approx 3:20 gmt [production]