51-100 of 2688 results (4ms)
2009-06-30 §
23:20 <Andrew> 23:19 <@brion> !log s1 switched to db16, enwiki back online! (thanks tim!) tomasz rebooted wikitech linode which also borked. :P [production]
22:58 <brion> synchronized php-1.5/CommonSettings.php 'put non-enwiki back to read/write' [production]
22:57 <brion> db14 (s1 master) is in some kind of borked state. slaves seem up to date; gonna try a master switch. [production]
22:49 <brion> synchronized php-1.5/CommonSettings.php [production]
22:49 <brion> putting site to read-only [production]
22:44 <Andrew> Rolling back updates, first vector skin, then config changes. If that doesn't help, will roll back UsabilityInitiative changes [production]
22:43 <andrew> synchronized php-1.5/skins/Vector.php 'Rolling back updates to r52581' [production]
22:29 <Andrew> db14 seems to be overloaded [production]
22:27 <Andrew> reports of (Cannot contact the database server: Unknown error (10.0.6.24)) [production]
22:24 <andrew> synchronized php-1.5/InitialiseSettings.php 'Remove vector from wgSkipSkins on usabilitywiki' [production]
22:21 <Andrew> Updated Vector and UsabilityInitiative, deployed UsabilityInitiative to usabilitywiki. Scapping to apply. [production]
21:06 <tomaszf> restarting lighttpd on storage2 due to large i/o wait [production]
20:01 <andrew> synchronized php-1.5/extensions/WikimediaMessages/WikimediaMessages.i18n.php [production]
18:44 <tomaszf> dropping -grosley dns entries in favor of dev. and civicrm. [production]
16:07 <Fred> restarted mwserve on pdf1 as it was not processing pdfs anymore. [production]
15:56 <brion> activity on pdf1 seems to have croaked around 15:30. pdf daemon might need a restart? [production]
04:28 <tstarling> synchronized php-1.5/extensions/timeline/Timeline.php [production]
04:28 <tstarling> synchronized php-1.5/extensions/timeline/EasyTimeline.pl [production]
04:27 <Tim> updated EasyTimeline to r52591 [production]
03:34 <Rob> seeing good results with the firewall plugin on the corporate blog, copying it over to the techblog and setting it to active. [production]
02:57 <Rob> It was net gremlins! [production]
02:56 <brion> appears to have been temporary networking issue (external?) affecting access to some clients [production]
02:48 <brion> some sort of downtime reported for a few minutes. no apparent current problems persisting, all looks well [production]
02:11 <hcatlin> Mobile site is having some encoding issues with UTF-8 characters, therefore the redirects have been disabled from common.js [production]
2009-06-29 §
23:03 <andrew> synchronized php-1.5/extensions/WikimediaMessages/WikimediaMessages.i18n.php 'Updated l10n' [production]
23:00 <Fred> updated DNS for *.m.wikipedia.org to point to mobile1 instead of eiximenis [production]
22:05 <andrew> synchronized php-1.5/extensions/UsabilityInitiative/EditToolbar/EditToolbar.js 'Trevor told me to!' [production]
22:05 <andrew> synchronized php-1.5/extensions/UsabilityInitiative/EditToolbar/EditToolbar.css 'Trevor told me to!' [production]
21:47 <Fred> apache stuck on srv[210-214] - restarting. [production]
21:36 <Andrew> scapping for vector updates [production]
21:35 <Andrew> updated Vector skin to r52581 [production]
21:18 <Andrew> Lost connection to zwinger mid-scap. Running scap again to make sure everything's fine. [production]
21:10 <Andrew> Scapping to deploy UsabilityInitiative CSS/JS changes, and license updates done by Fred. [production]
21:09 <Andrew> updated WikimediaMessages to push latest changes pushed to svn by siebrand. [production]
20:58 <Andrew> not scapping yet, deploying license change stuff too, waiting for Fred to do the rest of the updates for that [production]
20:52 <Andrew> scapping [production]
20:49 <Andrew> Activating UsabilityInitiative extension and removing vector from $wgSkipSkins on testwiki [production]
20:38 <RobH_A90> racked and installed mobile1 [production]
20:38 <RobH_A90> pulled nehlam test server [production]
19:01 <Andrew> pdated UsabilityInitiative extension on test as prep for deployment on testwiki. [production]
2009-06-28 §
19:39 <RobH_away> ran authdns-update [production]
19:39 <RobH_away> changing bayes mgmt IP (to make it sane and in line with the rest of the mgmt IP ranges) and adding the new IP to dns and reverse template files [production]
18:58 <brion> bayes back up [production]
18:52 <midom> synchronized php-1.5/db.php 'db28 live' [production]
18:49 <brion> poking LOM on bayes; box is down, EZ requested reboot. [production]
08:54 <domas> manually reset db28 position, based on innodb internal info, got some filesystem relaylog corruption [production]
08:52 <midom> synchronized php-1.5/includes/GlobalFunctions.php 'removing messages profiling hook' [production]
06:43 <Tim> rebooted db28 with /proc/sysrq-trigger [production]
06:29 <Tim> depooled db28, locked up in kswapd, needs reboot [production]
2009-06-27 §
07:42 <domas> changed log expiration on db9 to 100 days [production]