551-600 of 10000 results (14ms)
2012-01-09 §
21:36 <LeslieCarr> changing gmond source for ganglia3-tip [production]
21:19 <RobH> snapshot1001-1004 mgmt online [production]
21:05 <RobH> updating dns with snapshot1001-1004 primary ip info [production]
20:56 <RobH> updating dns with snapshot1001-1004 mgmt [production]
20:30 <catrope> synchronized php-1.18/extensions/ArticleFeedbackv5/modules/jquery.articleFeedbackv5/jquery.articleFeedbackv5.js '[[rev:108470|r108470]]' [production]
20:29 <catrope> synchronized php-1.18/extensions/ArticleFeedbackv5/api/ApiArticleFeedbackv5.php '[[rev:108470|r108470]]' [production]
20:25 <LeslieCarr> replacing the ops@ alias with the new ops list on mchenry as people keep forgetting to email the new list [production]
19:57 <nikerabbit> synchronized php-1.18/extensions/Translate/Translate.php 'Deploy [[rev:108469|r108469]] - bugfix for Translate' [production]
19:43 <RobH> torrus dead, kicking [production]
19:32 <nikerabbit> synchronized wmf-config/CommonSettings.php 'Updating Translate config 2/2' [production]
19:32 <nikerabbit> synchronized wmf-config/InitialiseSettings.php 'Updating Translate config 1/2' [production]
19:16 <nikerabbit> synchronized php-1.18/includes/parser/Parser.php 'Deploying [[rev:108461|r108461]]' [production]
19:07 <catrope> synchronized php-1.18/extensions/UploadWizard/resources/mw.UploadWizardLicenseInput.js '[[rev:108459|r108459]]' [production]
18:55 <nikerabbit> synchronized php-1.18/extensions/Translate/ 'Deploying translate [[rev:108451|r108451]]' [production]
18:54 <nikerabbit> synchronized php-1.18/languages/messages/MessagesEn.php 'Updating messagesEn' [production]
18:42 <nikerabbit> synchronized php-1.18/extensions/ParserFunctions/ParserFunctions.i18n.magic.php 'Deploying [[rev:108449|r108449]]' [production]
18:34 <nikerabbit> synchronized php-1.18/extensions/WikimediaMessages/WikimediaGrammarForms.php 'Deploying [[rev:108433|r108433]]' [production]
18:30 <nikerabbit> synchronized p/extensions/WebFonts/ 'Updating WebFonts [[rev:108447|r108447]]' [production]
18:21 <nikerabbit> synchronized php-1.18/extensions/Narayam/ 'Syncing Narayam' [production]
18:18 <Nikerabbit> running Narayam preference migration script [production]
17:30 <jeremyb> the time is now 17:30:30 UTC [production]
17:20 <RoanKattouw> Installing (!) NTP on wikitech [production]
17:16 <jeremyb> the time is now 17:19:30 UTC [production]
02:01 <LocalisationUpdate> completed (1.18) at Mon Jan 9 02:04:47 UTC 2012 [production]
2012-01-08 §
23:20 <Reedy> For some reason cp1001-10042 weren't listed in CommonSettings.php XFF, but (at least) 1042 was in service, meaning edits were attributed to it [production]
23:18 <reedy> synchronized wmf-config/CommonSettings.php 'Add cp1001-cp1041' [production]
23:10 <reedy> synchronized wmf-config/CommonSettings.php 'Add cp1042 to XFF' [production]
21:47 <rainman-sr> killed broken search indexer thread on searchidx1 (please note searchidx1 is no longer in use!), and restarted incremental indexing on searchidx2 which was somehow broken [production]
21:43 <rainman-sr> someone started incremental updating on searchidx1 ??!! [production]
14:54 <apergos> removed old puppet lockfile on brewster, ran by hand [production]
14:47 <apergos> cleared out some very large squid logs on brewster, (basically all of them) plus lighty logs, disk was full. restarted squid manually [production]
02:01 <LocalisationUpdate> completed (1.18) at Sun Jan 8 02:05:11 UTC 2012 [production]
00:43 <tfinc> killing long running show_bug.cgi procs on kaulen [production]
2012-01-07 §
22:30 <Reedy> Users reporting slowness while editing. dberror.log shows a few mysql errors for enwiki master and slaves. Few errors on other wikis, mainly enwiki [production]
02:01 <LocalisationUpdate> completed (1.18) at Sat Jan 7 02:05:09 UTC 2012 [production]
2012-01-06 §
23:22 <RobH> working rt1549 lvs1003 may flap, it is presently not in service due to possible hdd failure [production]
22:55 <binasher> db22 is back in s4 [production]
22:55 <asher> synchronized wmf-config/db.php 'adding db22 back to s4' [production]
21:41 <RobH> db1029 powering back up with ssd testing hardware installed [production]
21:35 <RobH> db1029 coming down for ssd testing [production]
21:26 <RobH> cp1014 and cp1019 hdd controller cables replaced (removed for testing controllers), both can be used normally [production]
21:19 <binasher> restoring db22 from a live hotbackup of db1038 [production]
21:18 <RobH> es1002 back ready for service use per #2220: replace original RAID card in es1002 [production]
21:05 <binasher> putting db51 into production as an s4 slave [production]
21:05 <asher> synchronized wmf-config/db.php 'adding db51 as an s4 slave' [production]
20:57 <binasher> started slaving db51 off of db31 [production]
20:21 <RobH> rt2226 - redeploy db22 for asher [production]
20:19 <RobH> db22 reinstalled and booting into OS. No puppet runs yet, now its Asher's problem ;] [production]
20:04 <RobH> db22 reinstalling [production]
19:24 <binasher> started innodb hot backup of db1038 to db51 [production]