551-600 of 1089 results (6ms)
2009-01-06 §
16:01 <mark> Changed 'knams' into 'esams' in DNS, kept a lot of old names in place [production]
15:26 <Tim> cleaned up binlogs on db1 [production]
13:09 <mark> Did some Traffic Engineering on the Amsterdam network [production]
11:58 <Tim> installed NRPE on new ES servers [production]
11:47 <domas> added db29 to s3 duty [production]
11:32 <Tim> locked clusters 18 and 19, updated nagios [production]
11:27 <Tim> fixed lack of schema on srv161 [production]
11:21 <Tim> retired cluster18 from the write list, added cluster20 and cluster21 [production]
11:18 <domas> test!!! [production]
11:18 <domas> test [production]
2009-01-03 §
23:57 <mark> Restored AAAA record on upload.wikimedia.org [production]
12:04 <domas> db17, db18 had OS/firmware updates, rebooted [production]
10:50 <domas> db19 RAID complaining about temperature, check-raid/kswapd/mysqld deadlock. upgrading RAID firmware, rebooting, etc [production]
01:23 <Tim> removed db3 and db4 from rotation again, to allow recompressTracked to go faster [production]
00:36 <Tim> depooled db19, is down [production]
00:32 <Tim> restarting recompressTracked with an extra wfWaitForSlaves() [production]
00:08 <Tim> repooled db3 and db4 [production]
2009-01-02 §
22:35 <Tim> depooled db3 and db4 temporarily [production]
20:54 <mark> Depooled db4 [production]
00:42 <Tim> restarting recompressTracked.php on hume [production]
2009-01-01 §
20:34 <brion> live-merging file delete fatal error fix from r45278 [production]
19:47 <brion> bumped meter image to 7 [production]
01:59 <brion> scapping! [production]
01:40 <brion> svn up'ing test.wiki to r45274 [production]
00:55 <brion> svn up'ing on test.wikipedia [production]
2008-12-31 §
18:40 <brion> fixed old whygive.wikimedia.org blog by copying de-conflicted WordPress source files out of the active blog where we fixed it after the 2.7 upgrade [production]
2008-12-30 §
23:02 <RobH> is leaving on a jet plane, weeeeeeeee.. in 8 hours. [production]
23:01 <RobH> all knams squids are now online. [production]
22:49 <RobH> knsq23-26 back in rotation, 3 more to go. [production]
22:33 <RobH> enabled knsq16-knsq22 in lvs, almost time to go back to hotel and die. [production]
22:22 <brion> attempting to purge affected pages on dawiktionary, dawiki [production]
22:21 <brion> taking dawiki, dawiktionary out of read-only because the rest of the fixes won't work until it's disabled :P [production]
22:14 <brion> poking diff version in live DifferenceEngine.php to eliminate bogus cache entries for dawiki/dawiktionary [production]
22:11 <RobH> stopping and clearing the cache on knsq16-knsq30. [production]
22:06 <brion> trying it again, but this time with the right variable names [production]
22:02 <brion> attempting to clear revision text loading cache entries for dawiktionary, dawiki [production]
21:47 <brion> live-merging r45206 so [[bugzilla:16841]] corrupted entries will be loaded properly on dawiki/dawiktionary. need to clear revision, diff, parser caches... [production]
21:15 <brion> locking dawiki, dawiktionary ($wgReadOnly) pending encoding fix [production]
20:07 <brion> killed recompressTracked.php processes on hume pending investigation of encoding breakage [production]
20:02 <brion> commenting ariel out of pmtpa also [production]
19:58 <brion> trying to clear no-longer-in-dns hosts from ALL node group [production]
19:57 <brion> PLEASE SAY WHAT SERVER YOU'RE RUNNING BATCH PROCESSES ON IF THEY'RE NOT ON ZWINGER. thanks [production]
19:56 <RobH> power disconnection for primary routing rack in esams. power restored, and totally was not robh's fault regardless of what lies mark may say to the contrary. [production]
19:54 <brion> encoding issues reported with some old edits on dawiki. wondering if this is recompression-related? [production]
18:46 <brion> added PMTPA nameserver back in mayflower's resolv.conf so DNS actually works on it until things are fixed [production]
17:42 <brion> internal DNS for knams seems to be down (at least on mayflower), this is breaking at least SVN update notifications [production]
17:14 <brion> updating logo for pmswiki [[bugzilla:16587]] [production]
13:29 <Tim> starting recompressTracked.php on all wikis [production]
11:22 <mark> Shutting down knsq16-30 [production]
11:00 <mark> In case of overload problems, please move traffic to pmtpa (scenario knams-down) [production]