751-800 of 3344 results (5ms)
2009-06-28 §
18:49 <brion> poking LOM on bayes; box is down, EZ requested reboot. [production]
08:54 <domas> manually reset db28 position, based on innodb internal info, got some filesystem relaylog corruption [production]
08:52 <midom> synchronized php-1.5/includes/GlobalFunctions.php 'removing messages profiling hook' [production]
06:43 <Tim> rebooted db28 with /proc/sysrq-trigger [production]
06:29 <Tim> depooled db28, locked up in kswapd, needs reboot [production]
2009-06-27 §
07:42 <domas> changed log expiration on db9 to 100 days [production]
2009-06-26 §
20:33 <Rob> blog.wikimedia.org back online, very restrictive settings and improved security (i hope) [production]
19:09 <Rob> singer https services back online for survey. ocs, wm09schols [production]
18:20 <Rob> Singer Restore: survey.wikimedia.org - UP, ocs.wikimania2009.wikimedia.org - UP [production]
17:11 <Rob> pulled blog.wikimedia.org out of squid via changing its dns to point directly at singer. just to make it easier to secure and fix on the fly later today [production]
15:03 <Rob> firing singer back up for reinstall and restoration from the haxor [production]
14:42 <midom> synchronized php-1.5/includes/parser/ParserCache.php 'removing the pcache hack for now' [production]
11:44 <domas> halted singer, don't start it without contacting me. [production]
07:27 <brion> synchronized php-1.5/includes/parser/ParserCache.php 'Putting live hack for MJ article back. Total traffic spike down but article traffic still spiking.' [production]
05:53 <Tim> rebooting srv182, swap death since ~00:10 UTC [production]
05:50 <Tim> killed broken mysqld_safe instances on srv157, srv161, srv167, srv173, srv174, srv175, srv181, srv185, srv186 [production]
05:17 <jeluf> synchronized php-1.5/mc-pmtpa.php 'replace srv113 by srv142' [production]
05:14 <jeluf> synchronized php-1.5/mc-pmtpa.php 'replace srv182 by srv92' [production]
2009-06-25 §
23:36 <brion> synchronized php-1.5/includes/parser/ParserCache.php 'live hack to extend caching of Michael Jackson' [production]
23:17 <brion> synchronized php-1.5/includes/db/Database.php 'tweak the db err msg' [production]
22:46 <brion> synchronized php-1.5/mc-pmtpa.php 'putting 156 mc back' [production]
22:44 <brion> synchronized php-1.5/mc-pmtpa.php 'swapping in 145 in place of temp down 156' [production]
22:43 <tomaszf> rebooting srv156 due to hard down [production]
22:42 <midom> synchronized php-1.5/db.php '156 down' [production]
18:26 <RobH_A90> srv92 back up with replaced fans [production]
18:16 <RobH_A90> srv92 down due to bad fan, going to replace it with another dead server fan, yay for frankenstien servers [production]
18:09 <RobH_A90> db17 has had a cold reset, service processors are now responsive, as well as system [production]
09:21 <tstarling> synchronized php-1.5/languages/Language.php [production]
2009-06-24 §
16:27 <robh> synchronized php-1.5/InitialiseSettings.php 'bug 19364' [production]
15:56 <Fred> modified /home/wikipedia/conf/nagios/sync to include new nagios server in the sync/restart process. [production]
15:55 <Fred> cleaned up /home/wikipedia/conf/nagios/conf.php to remove unused server / conf parameters. [production]
15:54 <Fred> Cleaned up dsh node_group and resynched Nagios. [production]
10:26 <midom> synchronized php-1.5/includes/specials/SpecialRecentchanges.php 'livehack upper limit, as mediawiki devs are lazy.. oh wait.' [production]
08:29 <domas> compressed databases on db3-5 using lzip, freed up the space. :) [production]
08:16 <midom> synchronized php-1.5/db.php 'db24 back to life' [production]
06:28 <Tim> updated GeSHi to 1.0.8.4 [production]
05:54 <Tim> removed all SVN externals from the MW working copy. Updated extensions/SyntaxHighlight_GeSHi to r52346. Scapping. [production]
2009-06-23 §
22:00 <Fred> make && deploy of new squid configuration with added acl for spence.w.o [production]
21:07 <robh> synchronized php-1.5/InitialiseSettings.php '19364 Set project namespace of Portuguese Wikibooks' [production]
20:49 <robh> synchronized php-1.5/InitialiseSettings.php 'Fixing issues with arwiki namespacing and formatting' [production]
19:33 <Rob> checked up on the updateTitles running against enwiki on hume. On 11864380 / 23266362 [production]
19:31 <robh> synchronized php-1.5/InitialiseSettings.php '19357 Create new namespace on arwiki' [production]
19:22 <robh> synchronized php-1.5/InitialiseSettings.php [production]
19:14 <robh> synchronized php-1.5/InitialiseSettings.php 'Bug 19116 adding namespace aliases for itwikibooks' [production]
15:39 <robh> synchronized php-1.5/InitialiseSettings.php '19328 set for fywikibooks - typosssss' [production]
15:36 <robh> synchronized php-1.5/InitialiseSettings.php '19328 set for fywikibooks' [production]
15:24 <robh> synchronized php-1.5/abusefilter.php 'Updated with Andrew' [production]
15:15 <Rob> wrong reason listed in sync, opps, was for bug 19272 [production]
15:15 <robh> synchronized php-1.5/InitialiseSettings.php 'extensions/AbuseFilter/abusefilter.tables.sql' [production]
15:14 <robh> synchronized php-1.5/abusefilter.php '19274 Enable AbuseFilter on Lithuanian Wikipedia.' [production]