2451-2500 of 10000 results (12ms)
2012-06-28 §
05:08 <Tim> srv266 was flooding the fatal error log, complaining about a missing file. Killed apache and ran sync-common. [production]
05:03 <Tim> fixed fatal.log on fenari, socat was writing to a deleted file [production]
02:58 <LocalisationUpdate> completed (1.20wmf5) at Thu Jun 28 02:58:42 UTC 2012 [production]
02:30 <LocalisationUpdate> completed (1.20wmf6) at Thu Jun 28 02:30:03 UTC 2012 [production]
2012-06-27 §
23:50 <K4-7131> sync'd payments cluster to 592e0a5ba195 [production]
23:43 <preilly> synchronized wmf-config/InitialiseSettings.php 'add rule for mediawiki' [production]
22:37 <K4-7131> sync'd payments cluster to 7e9072c2d571c [production]
22:23 <binasher> temporarily pulling srv211 from pybal [production]
21:56 <RobH> mw1102 has no nic0, rather than troubleshoot it for a long time, reinstall! (rt 3058) [production]
21:01 <preilly> synchronized php-1.20wmf6/extensions/MobileFrontend 'fix CSRF' [production]
21:00 <preilly> synchronized php-1.20wmf5/extensions/MobileFrontend 'fix CSRF' [production]
20:55 <RobH> db1003 back online, replaced mgmt cable and mgmt is working now as well [production]
20:51 <LeslieCarr> rebooting srv266 as it is unresponsive [production]
20:44 <RobH> db1003 mgmt issue due to bad cable, system booting back up, replacing mgmt cable [production]
20:35 <RobH> clean mysql shutdown, db1003 now offline [production]
20:33 <RobH> db1003 mgmt is not responsible, I need to remove power and reboot. confirmed iwth asher this is an s3 slave and can do a short downtime without issues [production]
20:32 <maxsem> synchronized php-1.20wmf5/extensions/MobileFrontend/ 'MF fixes and logging' [production]
20:29 <maxsem> synchronized php-1.20wmf6/extensions/MobileFrontend/ 'MF fixes and logging' [production]
19:56 <reedy> synchronized wmf-config/InitialiseSettings.php 'disable LastModified and LastModified/E3Experiment' [production]
19:54 <reedy> synchronized php-1.20wmf6/maintenance/runJobs.php [production]
19:53 <reedy> synchronized php-1.20wmf5/maintenance/runJobs.php [production]
19:32 <reedy> synchronized php-1.20wmf6/extensions/Translate/specials/SpecialLanguageStats.php [production]
19:19 <reedy> synchronized php-1.20wmf6/extensions/Translate/ [production]
19:06 <reedy> synchronized php-1.20wmf6/extensions/Translate/ [production]
19:03 <reedy> rebuilt wikiversions.cdb and synchronized wikiversions files: meta back to wmf6, not cause of translate issues [production]
18:53 <reedy> rebuilt wikiversions.cdb and synchronized wikiversions files: closed to 1.20wmf6 [production]
18:51 <reedy> rebuilt wikiversions.cdb and synchronized wikiversions files: wikimedia wikis to 1.20wmf6 [production]
18:48 <reedy> rebuilt wikiversions.cdb and synchronized wikiversions files: wiktionary and wikiversity to 1.20wmf6 [production]
18:47 <reedy> rebuilt wikiversions.cdb and synchronized wikiversions files: wikisource and wikiquote to 1.20wmf6 [production]
18:47 <Jeff_Green> added several mobile hostnames to DNS for RT #2996 [production]
18:46 <reedy> rebuilt wikiversions.cdb and synchronized wikiversions files: wikinews and wikibooks to 1.20wmf6 [production]
18:44 <aaron> rebuilt wikiversions.cdb and synchronized wikiversions files: Moved metawiki back to 1.20wmf5 [production]
18:41 <K4-713> synchronized payments cluster to fundraising/1.20 de0256084a [production]
18:33 <aaron> rebuilt wikiversions.cdb and synchronized wikiversions files: Moved special wikis to php-1.20wmf6 [production]
18:29 <aaron> rebuilt wikiversions.cdb and synchronized wikiversions files: Moved en(wikibooks|wikinews|wikiquote|wikisource|wikiversity|wiktionary) to 1.20wmf6 [production]
18:05 <RobH> cp1017 memory replaced [production]
17:52 <RobH> cp1017 is offline due to memory error. replacement memory on site, pulling system for swap [production]
17:46 <preilly> synchronized php-1.20wmf6/extensions/ZeroRatedMobileAccess 'update for landing page' [production]
17:46 <asher> synchronized wmf-config/mc.php 'disabling wgMemCachedPersistent; lowering wgMemCachedTimeout to 2x client default from 30x default' [production]
17:45 <preilly> synchronized php-1.20wmf5/extensions/ZeroRatedMobileAccess 'update for landing page' [production]
17:44 <preilly> synchronized php-1.20wmf4/extensions/ZeroRatedMobileAccess 'update for landing page' [production]
17:20 <LeslieCarr> restarting apache2 on srv258 [production]
17:11 <maplebed> powercycled srv270 [production]
17:11 <mutante> powercycling srv277 (had to, frozen console) [production]
17:06 <LeslieCarr> rebooting srv287 [production]
17:05 <Ryan_Lane> rebooting srv280 [production]
17:03 <mutante> powercycling srv280 [production]
17:01 <paravoid> rebooting srv264, swapdeath [production]
16:52 <mark> Rebooting srv279, swapdeath [production]
16:52 <paravoid> rebooting srv275, swapdeath [production]