951-1000 of 10000 results (14ms)
2012-06-28 §
18:33 <hashar> srv190 and srv281 got ssh timeout [production]
18:31 <hashar> synchronized wmf-config/CommonSettings.php [production]
18:30 <hashar> did various tests using eval.php. Most important is $realm -> production. $cluster -> pmtpa. Syncing [production]
18:25 <hashar> updating mediawiki-config to grab a12545d edceb4c & eee97ad [production]
18:23 <RobHalsell> swapped bad psu out of ms1001-array3, redundant so no downtime [production]
15:40 <RobHalsell> pulling the following servers, relocating to payments rack: payments1001-1004, boron, beryllium, lithium [production]
15:34 <RobHalsell> dns updated [production]
15:31 <RobHalsell> boron appears to be unallocated, pulling IP allocation, rack allocation, moving to payments per 1227 [production]
14:40 <mutante> svn server is rebooting.brb [production]
14:38 <mutante> dist-upgrading formey (svn/gerrit), rebooting soon [production]
14:38 <Jeff_Green> manganese rebooted for kernel update [production]
14:38 <RobH> allocating yttrium to payments rack per rt 1227 [production]
14:25 <Jeff_Green> manganese dist-upgrade [production]
14:15 <Ryan_Lane> restarting apache on manganese [production]
14:15 <Ryan_Lane> restarting gerrit [production]
05:08 <Tim> srv266 was flooding the fatal error log, complaining about a missing file. Killed apache and ran sync-common. [production]
05:03 <Tim> fixed fatal.log on fenari, socat was writing to a deleted file [production]
02:58 <LocalisationUpdate> completed (1.20wmf5) at Thu Jun 28 02:58:42 UTC 2012 [production]
02:30 <LocalisationUpdate> completed (1.20wmf6) at Thu Jun 28 02:30:03 UTC 2012 [production]
2012-06-27 §
23:50 <K4-7131> sync'd payments cluster to 592e0a5ba195 [production]
23:43 <preilly> synchronized wmf-config/InitialiseSettings.php 'add rule for mediawiki' [production]
22:37 <K4-7131> sync'd payments cluster to 7e9072c2d571c [production]
22:23 <binasher> temporarily pulling srv211 from pybal [production]
21:56 <RobH> mw1102 has no nic0, rather than troubleshoot it for a long time, reinstall! (rt 3058) [production]
21:01 <preilly> synchronized php-1.20wmf6/extensions/MobileFrontend 'fix CSRF' [production]
21:00 <preilly> synchronized php-1.20wmf5/extensions/MobileFrontend 'fix CSRF' [production]
20:55 <RobH> db1003 back online, replaced mgmt cable and mgmt is working now as well [production]
20:51 <LeslieCarr> rebooting srv266 as it is unresponsive [production]
20:44 <RobH> db1003 mgmt issue due to bad cable, system booting back up, replacing mgmt cable [production]
20:35 <RobH> clean mysql shutdown, db1003 now offline [production]
20:33 <RobH> db1003 mgmt is not responsible, I need to remove power and reboot. confirmed iwth asher this is an s3 slave and can do a short downtime without issues [production]
20:32 <maxsem> synchronized php-1.20wmf5/extensions/MobileFrontend/ 'MF fixes and logging' [production]
20:29 <maxsem> synchronized php-1.20wmf6/extensions/MobileFrontend/ 'MF fixes and logging' [production]
19:56 <reedy> synchronized wmf-config/InitialiseSettings.php 'disable LastModified and LastModified/E3Experiment' [production]
19:54 <reedy> synchronized php-1.20wmf6/maintenance/runJobs.php [production]
19:53 <reedy> synchronized php-1.20wmf5/maintenance/runJobs.php [production]
19:32 <reedy> synchronized php-1.20wmf6/extensions/Translate/specials/SpecialLanguageStats.php [production]
19:19 <reedy> synchronized php-1.20wmf6/extensions/Translate/ [production]
19:06 <reedy> synchronized php-1.20wmf6/extensions/Translate/ [production]
19:03 <reedy> rebuilt wikiversions.cdb and synchronized wikiversions files: meta back to wmf6, not cause of translate issues [production]
18:53 <reedy> rebuilt wikiversions.cdb and synchronized wikiversions files: closed to 1.20wmf6 [production]
18:51 <reedy> rebuilt wikiversions.cdb and synchronized wikiversions files: wikimedia wikis to 1.20wmf6 [production]
18:48 <reedy> rebuilt wikiversions.cdb and synchronized wikiversions files: wiktionary and wikiversity to 1.20wmf6 [production]
18:47 <reedy> rebuilt wikiversions.cdb and synchronized wikiversions files: wikisource and wikiquote to 1.20wmf6 [production]
18:47 <Jeff_Green> added several mobile hostnames to DNS for RT #2996 [production]
18:46 <reedy> rebuilt wikiversions.cdb and synchronized wikiversions files: wikinews and wikibooks to 1.20wmf6 [production]
18:44 <aaron> rebuilt wikiversions.cdb and synchronized wikiversions files: Moved metawiki back to 1.20wmf5 [production]
18:41 <K4-713> synchronized payments cluster to fundraising/1.20 de0256084a [production]
18:33 <aaron> rebuilt wikiversions.cdb and synchronized wikiversions files: Moved special wikis to php-1.20wmf6 [production]
18:29 <aaron> rebuilt wikiversions.cdb and synchronized wikiversions files: Moved en(wikibooks|wikinews|wikiquote|wikisource|wikiversity|wiktionary) to 1.20wmf6 [production]